A mechanism for representing data structures in data type engines and providing inline compression

Abstract

The present disclosure relates to a mechanism for representing a data structure of a data type engine and providing inline compression. Systems, methods and apparatus for facilitating a data structure of a data type engine and providing inline compression are provided. The system receives a command to read data from a host memory by a network interface card (NIC), wherein the command indicates a data type. The system generates a plurality of read requests including an offset from a base address and a corresponding length based on the data type. The system sends the plurality of read requests to the host memory to obtain data from the host memory. The system obtains a byte mask descriptor corresponding to the data type. The system performs on-the-fly compression of the obtained data based on the obtained data and the byte mask descriptor, thereby allowing the NIC to return a requested subset of the obtained data.

Description

Background Art

[0001] User applications can send data residing in non-continuous buffers. Similarly, these user applications can receive data in non-continuous buffers. In traditional systems, typical processes can involve copying or packaging data into a continuous send buffer, transmitting data, and copying or unpacking data from a continuous receive buffer to a suitable application location. These data copy operations (i.e., packing and unpacking) occur in software and are therefore serial with data transmission communications, which can significantly extend communication time. In addition, users may wish to transmit only a subset of data elements arranged in a "structure array" format. Transmitting only a subset may result in inefficient data access from a high-bandwidth network interface card (NIC). BRIEF DESCRIPTION OF THE DRAWINGS

[0002] Figure 1 A diagram illustrating an architecture that facilitates data structures of a datatype engine and provides inline compaction in accordance with aspects of the present application.

[0003] Figure 2 An exemplary code excerpt of an application using data types according to aspects of the present application is shown.

[0004] Figure 3 A diagram of an outbound packet engine with a data type engine as part of a transmit path is shown in accordance with aspects of the present application.

[0005] Figure 4A An exemplary code excerpt for sending a subset of elements defined in a structure array format according to aspects of the present application is shown.

[0006] Figure 4B An exemplary code excerpt for optimizing sending a subset of elements defined in a structure array format according to aspects of the present application is shown.

[0007] Figure 5 A diagram is shown describing a high-level view of an outbound packet engine, including a data type engine, storage of non-byte mask descriptors and byte mask descriptors, in accordance with aspects of the present application.

[0008] Figure 6 A diagram depicting a high-level view of a compression module according to aspects of the present application is shown.

[0009] Figure 7 A flow chart of a method for facilitating data structures of a data type engine and providing inline compression according to aspects of the present application is shown.

[0010] Figure 8An apparatus is shown that facilitates data structures of a data type engine and provides inline compression according to aspects of the present application.

[0011] In the drawings, like reference numerals refer to the same drawing elements. DETAILED DESCRIPTION

[0012] The description presented below enables those skilled in the art to make and use the various aspects and examples, and is provided in the context of a specific application and its requirements. Those skilled in the art will readily appreciate various modifications to the disclosed aspects, and the general principles defined herein may be applied to other aspects and applications without departing from the spirit and scope of the invention. Therefore, the aspects described herein are not limited to the aspects shown, but should be given the widest scope consistent with the principles and features disclosed herein.

[0013] User applications can send data residing in non-continuous buffers. Similarly, these user applications can receive data in non-continuous buffers. In conventional systems, typical streams can involve copying or packaging data into a continuous send buffer, transmitting data, and copying or unpacking data from a continuous receive buffer into a suitable application location. These data copy operations (i.e., packing and unpacking) occur in software, and therefore serially communicate with data transmission, which can significantly extend communication time. In addition, the user may wish to transmit only a subset of data elements arranged in a "structure array" format. Transmitting only a subset may result in inefficient data access from a high-bandwidth network interface card (NIC).

[0014] Current software interfaces can allow users to express data layout in its original form. A commonly used software interface is the message passing interface (MPI), in which users can use a base address to start transmission and a data type to apply the transmission. MPI also provides a mechanism called "derived data types", which can describe the data layout in the host memory and use the description for some MPI calls (e.g., MPI_Send and MPI_Recv). Historically, using derived data types in MPI may result in poor performance because the library must interpret the description so that the MPI_Send and MPI_Recv commands are executed to copy in and out of the internal buffer respectively. Therefore, packing and unpacking operations are still performed in software and are still connected in series with communication, which may result in inefficiency in the overall performance of data-related operations, such as data collection / scattering operations.

[0015] The described aspects provide a system that solves these problems by providing a data type engine (DTE) that performs a gather / scatter operation in hardware, thereby eliminating inefficient data duplication used in previous software solutions. The data type engine may reside in a network interface card (NIC) or an application-specific integrated circuit (ASIC) of the NIC, including as part of an outbound packet engine and an inbound packet engine. The data type engine may generate a series of addresses and lengths based on a data type descriptor. The data type descriptor may be an input / output vector (IOVEC) structure with relative addressing and may describe a multidimensional array defined with a count, a block size, and a stride. By using the data type descriptor as part of the data path, the data type engine in both the sending path (e.g., a read operation from a main memory is processed by an outbound packet engine) and the receiving path (e.g., a write operation to a main memory is processed by an inbound packet engine) may result in a reduction in the communication time involved in data transmission on the network. This results in more efficient message transmission between network entities, which in turn may result in more efficient resource usage, for example, to support high performance computing. The described aspects of the data type engine may perform gather / scatter operations while solving several challenges. One challenge is to ensure that the representation of the data type covers common use cases while remaining compact. This can be difficult when dealing with code that builds arrays of structures. The described aspects can represent the data type with a representation of three strides, three counts, and a base block. The system can use length, number of significant bytes, and a bit mask (e.g., as described below with Figure 5 and Figure 6 The data type can represent an arbitrary three-dimensional (3D) array of C structures (up to 256 bytes per structure) using less than 64 bytes of storage for each data type.

[0016] Another challenge involves the use of redundant memory resources as part of the compression. Reading data from memory and placing the read data into a temporary local buffer before compressing the valid bytes may require redundant memory resources. Current solutions may involve allocating a buffer, placing the results of the read into the buffer, and then compressing the data, which may result in inefficient use of memory resources. The described aspects address this challenge by compressing 256 bytes using a 256-bit byte mask to capture only the valid bytes in an eight-stage 2-to-1 multiplexer. The system can also utilize a population count (i.e., "POPCNT") instruction found in most processors to quickly and efficiently encode additional information to reduce the amount of logic required for each stage of compression. Figure 5 and Figure 6 A detailed description of the compression module in the outbound transport engine of the NIC is provided.

[0017] In addition to addressing these challenges to improve the performance of data transmission over the network and support high performance computing, the described system may include other aspects. One aspect may involve the ordering of data type commands (e.g., installing a data type descriptor in the DTE of the NIC) relative to payload commands (e.g., performing a user read / write operation). The data type must be created and set or installed before being used by a payload or user command. Commands can travel through the NIC via multiple paths (e.g., based on a series of parallel queues). The system can select a queue by performing a hash on various fields in the command or associated packet. Therefore, both the command to install the data type descriptor into the DTE of the NIC and the user command can have a flow hash that enables the two commands to take the same path. In addition, the system may include an event indicating that a data type command has been completed, such as described below with respect to Figure 1 The communication with the event engine is depicted in FIG. 1. Once a data type command is completed, the installed data type descriptor can be used by many separate subsequent commands.

[0018] Another aspect of the described system may involve data type selection and storage efficiency. The command to start a new message must indicate the data type it wishes to use. However, including a complete data type descriptor for each command may not be efficient or practical. In addition, data types may be frequently constructed to be the same between processes on a given node that are part of the same user job. The described aspects may include an allocation mechanism for data types that can lead to improved storage efficiency and the efficiency of new message commands. Software (e.g., an operating system (OS)) can allocate data type storage space and provide data type keys to users. The system can use a single key space for data type descriptors that can be shared between processes. Hardware can provide a mandatory mechanism that ensures that only processes that are part of the same application can use a given key. The user can include a given key in a new message to specify which data type to use. The system can use a null entry for a data type key to indicate that a command or transmission does not include a data type.

[0019] Another aspect of the described system may be related to the use of both absolute and relative addressing in the IOVEC structure used by the data type engine. Previous solutions may use absolute addressing in gather / scatter lists to support use cases involving OS kernel mode. However, interfaces to user-level software typically use relative addressing, which may create limitations when using the gather / scatter engine by MPI. In using the IOVEC structure in the described aspect, the DTE may use two different addressing modes to retrieve data from the host memory based on a list of addresses and lengths. The first mode is absolute addressing, in which the address retrieved from the host memory is the virtual address to be used, i.e., using a set of absolute address / length pairs in the IOVEC. The second mode is relative addressing, in which the address retrieved from the host memory is used as an offset from the base virtual address provided by the free user in the new message, where the offset can be obtained from the IOVEC (a list of offsets and lengths) or from the data type descriptor.

[0020] Another aspect of the described system relates to the efficiency of data type storage, IOVECs, and commands. In order to start a new message, previous solutions may add a pointer to an IOVEC to each command, which may result in an increased size of the command and a reduced efficiency of the command interface. The system may encode and set or install IOVECs in the same manner as other data types. However, some IOVEC use cases may be used only once, i.e., the system creates an IOVEC, uses it for one command ("only used once"), and discards it. Because command sequencing works in conjunction with the IOVEC method, the command sequencing logic described above may allow the system to maintain existing command formats while still ensuring an effective setup path for these "only used once" IOVECs.

[0021] Thus, aspects of the described system can address the above challenges by providing a data type engine (DTE) that generates a series of addresses and lengths based on a data type descriptor. The data type descriptor can be an IOVEC structure with relative addressing and can describe a multidimensional array defined with a count, block size, and stride, as shown below in conjunction with Figure 2 Described below. Figure 1 The overall architecture of a system (eg, NIC) including an outbound transmission engine and an inbound transmission engine is described below. Figure 3 Describes a detailed diagram of an outbound transport engine with a DTE module communicating with other modules.

[0022] Aspects of the described system may also provide a compression module in the outbound transmit engine of the NIC. The compression module may provide inline compression of requested read data by using byte masks and the currently existing software instruction POP_CNT to provide inline (i.e., "on-the-fly") compression, as described below in conjunction with Figure 5 and 6 The byte mask used can be any number of bits, such as a 128-bit byte mask, a 256-bit byte mask, or a 512-bit byte mask. Although the examples described herein use a 256-bit byte mask, the byte mask is not limited to 256 bits nor to a power of 2. The number of levels required to compress data using a byte mask can be determined as a logarithmic function of the number of bits in the byte mask (with a logarithmic base of 2), as shown below in conjunction with Figure 5 and 6 As described.

[0023] Thus, by using data types in a data type engine to represent data structures, and by providing inline compression based on byte masks and population counts, the described aspects can provide a system that addresses the above-mentioned challenges, including inefficient data duplication and use of redundant memory resources to perform compression. In addition, the system can provide a mechanism for hardware to read a block using a single read and transfer only the desired bytes (e.g., a subset of elements that a user wishes to send), which can result in more efficient transfers over a system host bus (e.g., a Peripheral Component Interconnect Express (PCIe) bus) than can be achieved by issuing multiple separate reads.

[0024] The terms "data type" and "data type descriptor" are used interchangeably in this disclosure and refer to descriptions of multidimensional arrays defined with counts, block sizes, and strides. The structure of a multidimensional array can be parameterized by up to three dimensions and a basic element structure that can span up to 256 bytes. In some aspects, more dimensions can be used and the basic element structure can span up to different numbers of bytes.

[0025] The terms "data type engine," "DTE," "data type engine module," and "data type engine unit" are used interchangeably in this disclosure and refer to a unit, module, or component that resides in a NIC or in a NIC ASIC. The data type engine may reside in both the outbound packet engine and the inbound packet engine of the NIC. The data type engine may use certain parameters to describe multi-dimensional arrays in host memory (i.e., data type descriptors).

[0026] The terms "unit" and "module" are used interchangeably in the present disclosure and refer to physical or logical components or groupings of operations performed by hardware, as described herein.

[0027] The terms "bit mask", "byte mask", and "byte mask descriptor" are used interchangeably in this disclosure and refer to a value having a number of bits that can be used as a byte mask on data up to a predetermined number of bytes. Each bit in the X-byte mask may correspond to a byte in an X-byte chunk or block of data. The size X of the byte mask may be any number of bits (such as 128, 256, or 256), and the size X of the data or the predetermined number of bytes is not limited to a power of 2. In addition, the following description may be combined with Figure 5 and 6 A logarithmic relationship between the number of bits in a byte mask and the number of levels required to compress the data is described.For purposes of illustration, some of the disclosed aspects use 256-bit byte masks and 256-byte blocks or data.

[0028] Example high-level architecture

[0029] Figure 1 Figure 100 of an architecture that facilitates a data type engine to support high performance computing according to aspects of the present application is shown. Figure 100 can depict a network interface card (NIC) including various components or modules and interfaces and communications. At one of the NICs, a host interface 110 exists between the components of the NIC and the host. The host interface 110 can handle communications with the host (via communication 142). At the other end, a port macro 132 exists between the components of the NIC and the network, and can handle communications with the network (via communication 186). Generally, the components on the left side of Figure 100 are related to the outbound path and can handle send operations related to performing read operations from the host memory (indicated by the bold line communications 146, 172, and 184 described below), while the components on the right side of Figure 100 are related to the inbound path and can handle receive operations related to performing write operations to the host memory (indicated by the bold line communications 188, 182, and 154).

[0030] The host interface 110 may receive data from various NIC components via system bus arbitration / host interface 112. The host interface 110 may also send command list pointers or commands with data to the command queue 114 (via communication 140), which may include a metadata (MD) table 115. The command queue 114 may send portal and Ethernet operations (e.g., related to read operations in the transmit path) to the outbound transfer engine 116 (via communication 156), which may include an outbound buffer 117 for storing data related to operations performed by the outbound transfer engine 116. The command queue 114 may also receive commands related to reading data (e.g., direct memory access (DMA) get commands) (via communication 144) and events from the event engine 118 (via communication 158). The command queue 114 may also send append commands to the list processing engine 122 (via communication 148).

[0031] The outbound transport engine 116 may include an instance of a data type engine ( Figure 1 The outbound transfer engine 116 may receive outbound DMA data from the host memory (via communication 146 from the host interface 112), which may be stored in the output buffer 117. Figure 3 A detailed description of the outbound transport engine 116 is provided.

[0032] After the data type engine of the outbound transfer engine 116 uses the IOVEC with relative addressing and describes the multidimensional array in the manner described herein, the outbound transfer engine 116 may output the request or command to the pending command table (transmit (TX)) 126 (via communication 172). The pending command table 126 may include a retry buffer 127. Once the corresponding command is completed, the pending command table 126 may send a completion notification to the event engine 118 (via communication 174). The pending command table 126 may send an outbound packet corresponding to the read command (via communication 184 to the port macro 132).

[0033] The NIC may receive inbound packets to be written to the host memory (via communication 188). The inbound packets may be sent to the pending command table (receive (RX)) 130. The pending command table 130 may then send information to the message status table 124 (via communication 180), and to the list processing engine 122 (via communication 178), which may include the list entry 123. The list processing engine 122 may send information to the message status table 124 (via communication 168), which may send information to the inbound transmission engine 120 (via communication 170). The inbound transmission engine 120 may include an input buffer 121. The list processing engine 122 may also send a memory or data type descriptor to the inbound transmission engine 120 (via communication 166) to be stored in a data type engine (not shown) of the inbound transmission engine 120. The list processing engine 122 may also send an event to the event engine 118 (via communication 164). The pending command table 130 may also send a payload associated with the inbound packet to the inbound transport engine 120 (via communication 182), which may be stored in the input buffer 121. The inbound transport engine 120 may include an instance of a data type engine ( Figure 1 154 to the host memory (via communication 154 to the host interface 112).

[0034] The inbound transfer engine 120 may receive data from the host interface 112 (via communication 150) and may also send completion notifications to the event engine 118 (via communication 162) when the corresponding command is completed. In addition to receiving completions from the pending command table 126 and the inbound transfer engine 120 (via communications 174 and 162, respectively), the event engine 118 may send event-related information to the host interface 112 (via communication 160). The pending command tables in the transmit and receive paths may communicate response status (tables 126 and 130 via communication 176), and the inbound transfer engine 120 may also send replies to the outbound transfer engine 116 (via communication 152). In the same manner, the outbound transfer engine 116 may form packets, and the DTE of the outbound transfer engine 116 may form multiple reads, and the list processing engine 122 may form an address for writing packets and pass the address to the DTE of the inbound transfer engine 120. The DTE of the inbound transport engine 120 may formulate multiple write requests including offsets from the address and corresponding lengths based on a given data type descriptor. It will be apparent to those skilled in the art that this mechanism may also be used for input processing of packets.

[0035] Data Type Descriptor Overview

[0036] In a typical NIC implementation, user software may provide a command indicating the virtual address and length of the data to be transferred. The NIC may receive the command and convert the virtual address. The NIC may then issue a read to the host memory (for sending data, such as in a Get operation) or a write to the host memory (for receiving data, such as in a Put operation). In conventional solutions, the reads and writes issued to the host memory are typically for large, contiguous blocks of data.

[0037] The described aspects provide a data type engine that generates a series of addresses and lengths based on a data type or "data type descriptor". The data type can natively express the data layout in the application memory, which can lead to the elimination of resource-consuming replication. That is, the data type is a way to describe the layout of native data in the memory. In the absence of data type descriptors as described herein, a central processing unit (CPU), a general processing unit (GPU), and an advanced memory system are used to perform the described data replication. However, since the operation of using these systems may be expensive (i.e., in general, CPUs, GPUs, and advanced memory systems are more suitable for other work), using these systems may be inefficient.

[0038] The data type descriptor can take two forms. The first form is a multidimensional array descriptor, where the data type engine uses parameters to describe an array in host memory, as described below in conjunction with the matrix conversion example in Figure 4. The multidimensional array can include: the number of elements in each dimension; the size of the block to be transferred; and the stride in each dimension. The data type descriptor can also include handles to the old data type name and the new data type name, as described below in conjunction with Figure 2 and 4. The second form is an input / output (IO) vector (IOVEC), in which the DTE obtains a list of addresses from host memory. IOVEC can use relative addressing or absolute addressing. In relative addressing, the DTE can obtain data based on an address that is an "offset" from a user-supplied base virtual address, while in absolute addressing, the DTE can obtain data based on an address that is a virtual address to be used.

[0039] Example code for an application using data types

[0040] Figure 2An exemplary code excerpt 200 of an application using data types according to aspects of the present application is shown. Portion 210 may indicate a data type "MPI_Type_contiguous," which allows data types to be copied into contiguous locations. Portions 212, 214, and 216 may indicate a data type "MPI_Type_vector," which allows data types to be copied into locations with equally spaced blocks, where each block can be obtained by concatenating the same number of copies of an old data type. The spacing between blocks may be a range of multiple old data types. "MPI_Type_vector" may indicate a structure including a multidimensional array, where the multidimensional array includes: a count corresponding to the number of blocks or elements in each dimension; a block size or length corresponding to the number of elements in each block; a stride in each dimension, corresponding to the number of elements between the start of each block; an indicator of an existing data type or an old data type; and a handle to a new data type.

[0041] Portion 210 may create a three element vector of type float and be used to construct a description of multiple faces (e.g., three) of a matrix. Using the data types indicated in portions 214 and 216 may result in sending data in large chunks or many large chunks, where chunks may be aggregated into messages and may result in efficient transmission. However, using the data type indicated in portion 212 may result in performing 12-byte gathers / scatters from many locations, which may not be as efficient as using the data types of portions 214 and 216. When comparing the performance of a NIC to that of a CPU and a GPU, a data type engine in a NIC having a data type descriptor may perform more efficiently when transmitting data in larger chunks (e.g., 128-byte segments) rather than in smaller chunks (e.g., 8-byte or 12-byte segments).

[0042] Another data type that can be created using the MPI standard is the data type "MPI_Type_indexed", which can build a structure similar to a traditional scatter / gather list. In some aspects of the described system, a user can create an "indexed" data type and generate an MPI_Send command using the created "indexed" data type, including a series of offsets from an address. Figure 3 This relative addressing is described in . In addition, a bit mask (e.g., a byte mask) can be applied to each element in the created list, which allows an application to provide a list of pointers to a structure in which only certain sub-elements of the structure are transferred. Figure 5 and 6 Describes the 256-bit byte mask used by the compression module in the NIC's outbound packet engine.

[0043] Furthermore, the array representations described herein can cover the most common types used in applications, namely one-dimensional (1D), two-dimensional (2D), and three-dimensional (3D) arrays. For each dimension, the described aspects can provide the number of elements and the stride between elements. Because a typical application may only send each "face" of a larger matrix, the described array representation can cover the 1D face of a 2D matrix, the 2D face of a 3D matrix, and the 3D face of a 4D matrix. Although the described aspects are described in a form that covers up to 4D matrices, the system can be extended to cover more dimensions.

[0044] Outbound Packet Engine and Send Path

[0045] As above combined Figure 1 As described, the data type engine can be integrated with the sending logic and receiving logic in the outbound packet engine and the inbound packet engine, respectively. Figure 3 A diagram 300 of an outbound packet engine 301 with a data type engine as part of a transmit path is shown in accordance with aspects of the present application. The outbound packet engine (OXE) 301 may include: a packet forming module 302; a data type engine (DTE) module 304; a direct memory access (DMA) module 306; and a post-processing module 308. The OXE 301 may take as input data from a command 320 and may output data packets to a reliability stage 322. In addition, the OXE 301 may communicate with an address translation unit (ATU) 310 and a host memory 312.

[0046] During operation, OXE 301 may receive commands, such as from a command queue of commands 320 (via communication 330). The command may be received via a message passing interface (MPI) and may be a command to read data from a host memory. In a standard path (i.e., where the data type descriptor does not exist and thus bypasses the DTE), the packet forming module 302 may form a packet referred to as a "non-DTE DMA command" that indicates a header, address, and length associated with the data to be read from the host memory. This standard non-DTE DMA command may be sent to DMA module 306 (via communication 334). DMA module 306 may obtain the corresponding address translation from ATU 310, such as by sending a virtual address 338 to ATU 310 and receiving a physical address 340 from ATU 310. DMA module 306 may then issue one or more read requests to host memory 312 (via communication 342) and receive corresponding one or more read responses from host memory 312 (via communication 344). The DMA module 306 may send the read responses to the post-processing module 308 (via communication 346 ), whose processed data may be sent as output to the reliability stage 322 (via communication 348 ).

[0047] In the event that a data type descriptor is present in the command (e.g., indicated as a handle to a previously stored data type descriptor), the packet forming module 302 may form a packet referred to as a "DTE command" that indicates or references the data type descriptor and a base address and length associated with the data to be read from the host memory. The DTE command may be sent to the DTE module 304 (via communication 332). The DTE module 304 may identify or determine that the data type descriptor indicated in the packet matches a data type descriptor stored in the DTE module 304 (e.g., as previously configured and stored by a user). Using the identified matching data type descriptors, the DTE module 304 may generate a plurality of read requests having offsets from the base address and corresponding lengths. The DTE module 304 may then send these generated read requests to the DMA module 306 (via communication 336 as DMA commands). The DMA module 306 may perform the above-described address translation (if necessary) and read retrieval operations associated with communications 338 , 340 , 342 , and 344 , as well as subsequent operations associated with the post-processing module 308 and the reliability stage 322 .

[0048] Example element array and data transfer optimization

[0049] Figure 4AAn exemplary code excerpt 400 for sending a subset of elements defined in a structure array format according to aspects of the present application is shown. Portion 410 may define a data structure (named "element") of four elements, the data structure comprising: an integer type "a" (4 bytes); a floating point type "b" (4 bytes); an unsigned integer type "c" (8 bytes) and a double type "d" (8 bytes). Therefore, the total size of all elements in the data structure is 24 bytes. Portion 410 may also define a three-dimensional data structure "element" array (named "AoE" or element array) with sizes of 80, 100, 200, and may also define three integers x, y and z as faces corresponding to the element array.

[0050] Portions 412, 414, and 416 may represent code for sending a subset of the elements of each of the three faces. For example, assume that a request to read data reads or sends only elements "b" and "d" of each face. Each portion includes nested iterations or loops, each of which traverses one of the two arrays corresponding to each face, where each iteration or loop specifically sends the exact element b or d that is desired to be read. For example, portion 416 indicates code for sending the b and d elements of face "zx", where the code traverses an outer loop of z from 0 to 80 with an inner loop of x from 0 to 200 (as shown in portion 418).

[0051] One aspect of the described system can provide optimization by utilizing the advantages of data arranged linearly in memory. Generally, many cells are less than 256 bytes in length and hardware can represent data up to 256 bytes. Optimization allows the system to expand a loop (e.g., such as the inner loop represented by part 418) into multiple "smaller sizes" elements that will be suitable for, for example, 256 byte blocks. Therefore, in the example of Fig. 4, where each such smaller size element is 24 bytes, the system can expand 10 of these 24-byte elements into a large 240-byte block, which can therefore be read as a single 240-byte block. Relative to reading 10 24-byte blocks through the host interface, reading a single 240-byte block once can reduce the communication on the host interface, which can result in a more efficient read operation.

[0052] As an example, assume that the elements described by the AoE in face zx are adjacent to each other in memory. The system can expand these 24-byte elements two at a time. Figure 4B The updated code obtained for this inner loop is described in . Figure 4B An exemplary code excerpt 430 is shown for optimizing sending a subset of elements defined in a structure array format according to aspects of the present application. Portion 432 may correspond to Figure 4APortion 416 and portion 434 may correspond to the updated inner loop. Portion 432 represents the expansion Figure 4A The code of inner loop 418 of 434 is changed so that the data is now read two blocks at a time, as indicated by the "x+2" and the additional two "send" lines in portion 434. Therefore, the system can now use a block size of 48 bytes as the returned data is obtained, rather than using a block size of 24 bytes (as in Figure 4A The system may also copy the 24-bit byte mask to obtain a 48-bit byte mask that may be used to perform compression on the corresponding 48-byte data returned.

[0053] If it can be Figure 4B As seen, unrolling the loop so that the maximum size does not exceed 256 bytes can provide optimization of the data transfer process because transferring larger sized data blocks (e.g., 48 bytes when unrolled by two or 240 bytes when unrolled by 10) over the host interface can be more efficient than multiple smaller sized elements (e.g., 10x 24-byte data blocks).

[0054] Outbound Packet Engine with Compression Module

[0055] Figure 5 A diagram 500 of a high-level view of an outbound packet engine (OXE) is shown, including a data type engine and storage of non-byte mask descriptors and byte mask descriptors, according to aspects of the present application. The diagram 500 may include a message slicing unit (MCU) module 510, a data type engine (DTE) module 520, and a direct memory access (DMA) module 530.

[0056] The MCU module 510 may include a request (Req) command processor 512, a send queue 516, and a response (Resp) command processor 514. The DTE module may include: a Get request (GRQ) engine 525; a Put request (PRQ) engine 526 and an issuer 527 (used to initiate a Put request to the DMA module 530 in the DTE phase); a Get reply (GRS) engine 528 and an issuer 529 (used to initiate a Get reply to the DMA module 530 in the DTE phase); and a shared descriptor memory including a non-byte mask descriptor memory 522 and a byte mask descriptor memory 524. The DMA module 530 may include: a request buffer (ReqMDB) 532; a response buffer (Resp MDB)

[0057] 534; initiator 538 (used to process requests in the DMA stage); initiator 539 (used to process responses in the DMA stage); arbitrator (ARB) 540; address translation (ATR) module 542; interface MDR1 544; interface MDR0

[0058] 546; and compression module 536.

[0059] The MCU module 510 may be divided into request (Req) and response (Resp) pipes. Each individual message segmentation unit in the MCU module 510 may manage commands by the process of building a packet header and collecting payload packets. When the MCU module 510 manages commands in which the message payload is defined by a data type (e.g., in which the command indicates a data type descriptor stored in the NIC), the MCU module 510 may pass packet-by-packet DMA instructions to the DTE module 520. The DTE module 520 may process the data type descriptor to generate a corresponding DMA instruction that obtains the packet payload from the host memory (as described above with respect to Figure 3 The DTE module 304, DMA module 306, ATU 310 and host memory 312 are related, including communications 336, 338, 340, 342 and 344).

[0060] There may be two similar interfaces between the MCU module 510 and the DTE module 520: one for processing the request pipeline (e.g., via the request command processor 512); and one for processing the response pipeline (e.g., via the response command processor 514). The request pipeline may include multiple MCUs that manage commands from the command queue (e.g., Put requests and Get requests) and commands from the list processing engine (LPE) (e.g., Get requests generated from the aggregated Put requests). The response pipeline may include multiple MCUs that process Put responses and Get response commands received from the inbound packet engine (IXE). When the request command processor 512 processes a command from a given MCU or when the response command processor 514 processes a Get response command, the processor 512 / 514 may pass instructions to the DTE module 520 for data type processing.

[0061] During operation, a user may create a data type (e.g., a multi-dimensional array structure), where the created data type is sent by the send queue 516 to be stored in the non-byte mask descriptor 522 (via communication 561). In the request pipeline, the request command processor 512 may send a command that does not indicate a data type descriptor directly to the initiator 538 of the DMA module 530 (via communication 573). The request command processor 512 may also send a command indicating a data type descriptor to the DTE module 520 (to the GRQ engine 525 (via communication 562) or to the PRQ engine 526 (via communication 563)). The GRQ engine 525 and the PRQ engine 526 may obtain the indicated data type descriptor from the non-byte mask descriptor storage 522 (via communications 565 and 566, respectively). The GRQ engine 525 may pre-process the data type descriptor and store the start context of a subsequent Get response packet in the request MDB 532 (via communication 568). The context is stored in the structure, and when the corresponding Get response packet arrives, the context is attached to the Get response packet and provided to the DTE in the inbound transfer engine. The PRQ engine 526 may send a set of DMA commands to the initiator 527 (via communication 570).

[0062] In the response pipeline, the response command processor 514 can send a command that does not indicate a data type descriptor directly to the initiator 539 of the DMA module 530 (via communication 576). The response command processor 514 can also send a command indicating a data type descriptor to the DTE module 520, to the GRS engine 528 (via communication 564), which obtains the indicated data type descriptor from the non-byte mask descriptor memory 522 (via communication 567). The GRS engine 528 can send a set of DMA commands to the initiator 529.

[0063] In the DMA module 530 (in either the request pipe or the response pipe), the ARB 540 may receive DMA commands from the initiators 538, 527, 529, and / or 539 (via communications 577, 578, 579, and / or 580, respectively). The ARB 540 may send commands to the ATR 542 for address translation (via communication 581), which may involve sending virtual addresses to the ATU 550 and receiving physical addresses from the ATU 550 (via communication 580). The DMA module 530 may include two interfaces to the host memory 552, the first interface may be managed by MDR0 546 and used for packet payload DMA reads (via communication 589). The second interface may be managed by MDR1 544 and used for low latency DMA reads using the CXL cache protocol (if available) (via communication 586). The DTE module 520 may use interface MDR1 546 to communicate with the host memory 552.

[0064] 544 to obtain the IOVEC entry and other information (via communication 569). ATR 542 can send the physical address to MDR1 544 (via communication 585), which can obtain the data at the given physical address from host memory 552 (via communication 586). ATR 542 can also send the physical address to MDR0 546 (via communication 584), which can obtain the data at the given physical address from host memory 552 (via communication 589).

[0065] When the DTE module 520 passes a DMA command to the DMA module 530, the command may include a bit that can be used after the ATR 542 to determine which MDR will manage the DMA request. For example, a DMA response managed by MDR1 544 (for IOVEC entries and other information) can be routed back to the DTE module 520 (via communications 569 and 571), while a DMA response managed by MDR0 546 can be routed to one of the packet buffers (532 and 534) via the compression module 536. The DMA command generated by the DTE module 520 may include information indicating whether compression needs to be performed and a data type descriptor. This information may determine the byte mask used for compression. For example, MDR0 546 may send a packet to the compression module 536 (via communication 588), and if the packet indicates that compression is to be performed, the compression module 536 may obtain the appropriate byte mask from the byte mask descriptor memory 524 (via communication 590). As described below, the compression module 536 may use the obtained byte mask to compress away invalid bytes of the DMA response data. Note that while byte mask descriptor memory 524 is depicted as being logically located in DTE module 520, it may be physically located near the compression logic performed by compression module 536 in DMA module 530. Given data (compressed if necessary) may be sent to the appropriate packet buffer, e.g., request buffer 532 or response buffer 534 (via communications 583 and 587, respectively).

[0066] Detailed description of the compression module

[0067] Recall that the size of the obtained byte mask ("bit_mask_size") can be any number of bits and is not necessarily limited to values ​​that are powers of 2 (such as 128, 526, 512). A 256-bit byte mask is used for illustrative purposes only. The compression module described herein can scan the obtained bit_mask_size byte mask and use the bit_mask_size byte mask to compress any read bit_mask_size bytes at a certain level (i.e., log2(bit_mask_size), rounded up as needed). For example: a 128-bit byte mask will use log2(128)=7 levels; a 129-bit byte mask will use log2(129)=8 levels (rounded up); and a 256-bit byte mask will use log2(256)=8 levels. As described above, if compression is required, the MDR0546 can obtain the appropriate byte mask from the byte mask descriptor memory 524 using the data type descriptor (i.e., data type handle) provided in the information of the DMA request received from the DTE module 520. In three cycles, the byte mask and other information can be obtained from the byte mask descriptor memory 524. The compression module 536 can scan the obtained 256-bit byte mask and use the byte mask in the eight-stage 2:1 multiplexer to compress any 256 bytes. The information obtained from the byte mask memory descriptor 524 may include the 256-bit byte mask and seven pre-calculated overall counts. These overall counts can be obtained from a non-expensive instruction called POPCOUNT running on the host processor. A single cycle instruction can calculate the number of valid bits in a 32-bit word. The overall count can be a progressive series of counts of ones from the least significant bit to the most significant bit in the byte mask at intervals of 32 bits. For example: the first pre-calculated overall count can be the number of ones in the byte mask [31:0]; the second pre-calculated overall count can be the number of ones in the byte mask [63:0]; etc. Therefore, storing only seven of these results can result in a simplification of the logic required to calculate the control bits of the eight-stage multiplexer.

[0068] When obtaining the byte mask and the pre-calculated total count from the byte mask descriptor memory 524, the MDR0 546 may also output the DMA response data and various information used to align the data and finally write the data to the target packet buffer (eg, 532 or 534).

[0069] Figure 6 A diagram 600 depicting a high-level view of a compression module according to aspects of the present application is shown. Diagram 600 may represent a module or modules of a network interface card (NIC), for example, a portion of an outbound packet engine in a NIC. The NIC module may include a bitwise inversion module 612 and a compression module 614 (which may correspond to Figure 5The compression module 536). The obtained 256-bit byte mask can be referred to as "byte_mask" and can be represented as "byte_mask[255:0]620". Byte_mask620 may include one bit corresponding to each byte of the 256 bytes of data to be compressed. The data to be compressed may include up to 256 bytes and may be represented as "data[255:0][7:0]640", where the first element indicates the number of bytes (out of 256) and the second element indicates the bits within each byte of the data.

[0070] Byte_mask 620 may be bit-wise inverted in the NIC by the bit-wise inversion module 612 for input to the first layer parity logic. Since the result may be some XNOR gates in the first layer parity logic instead of XOR gates, there is no hardware cost for this inversion. When there is no hardware cost, the present invention may also be performed by software instead (i.e., the software may pre-invert the byte mask). Thus, byte_mask[255:0] 620 may be bit-wise inverted by the bit-wise inversion module 612 (or software) to output "mask[255:0] 622".

[0071] In addition, "pop_cnts[x]" may represent seven overall counts calculated by software and provided to the compression module 614 along with the byte_mask 620 and the data 640. That is: pop_cnts[0] may represent the count of zeros in byte_mask[31:0]; pop_cnts[1] may represent the count of zeros in byte_mask[63:0]; pop_cnts[2] may represent the count of zeros in byte_mask[95:0]; pop_cnts[3] may represent the count of zeros in byte_mask[127:0]; pop_cnts[4] may represent the count of zeros in byte_mask[159:0]; pop_cnts[5] may represent the count of zeros in byte_mask[191:0]; and pop_cnts[6] may represent the count of zeros in byte_mask[223:0]. This can be represented by "pop_cnts[6:0][7:0] 630", where the first element indicates the overall count (of 7) and the second element indicates the position of the bit within that count value. Thus, in the example of a 256-bit byte mask, the POPCNT instruction can be called 7 times and summed as partial results.

[0072] The compression module 614 may take the following information as input: mask[255:0] 622; pop_cnts[6:0][7:0] 630; and data[255:0][7:0] 640. The compression module 614 may perform eight levels / layer compression using a 2:1 multiplexer, as described below. Upon completion of the eight levels / layer multiplexer (i.e., layers 1-8), the compression module 614 may output compressed data represented as "datc[255:0][7:0] 650".

[0073] In layer 1, each multiplexer can select from: data[i] or data[i+1] (where data[i+1] is zero for i>255); mask[i] or mask[i+1] (where mask[i+1] is zero for i>255). That is, the data and mask are compressed together. The control of multiplexor[i] in layer 1 is marked as sel0[i]. The compression module 536 can generate 256 multiplexer control bits based on the following steps: each control bit for (i%32)==0 and i>0 can be directly obtained from the pre-calculated pop_cnts bit (for example, pop_cnts[0][0] can be the value of "sel0

[32] ", pop_cnts[1][0] can be the value of "sel0

[64] "); and all other control bits can be the XOR of mask[i] and sel0[i-1] (where sel0[i] can be set to mask[i] when i<1). Therefore, the output of layer 1 can be dat1[255:0][7:0] and msk1[255:0]. The value of pop_cnts goes along with the data without modification.

[0074] In layer 2, each multiplexer can select from the output of layer 1: dat1[i] or dat1[i+2] (where dat1[i+2] is zero for i>253); msk1[i] or msk1[i+2] (where msk1[i+2] is zero for i>253). For 256 control bits: Each control bit for (i%32)==31 and i<224 can be obtained directly from the pre-calculated pop_cnts bits (e.g., pop_cnts[0][1] can be the value of "sel1

[31] ", and pop_cnts[1][1] can be the value of "sel1

[63] "). Note that the position of the bit obtained from each pop_cnt has been increased by 1. The position to which pop_cnts is applied has been shifted by the shift size in the previous stage. All other control bits can be the XOR of msk1[i] and sel1[i-2] (where control bit sel1[i] can be set to msk1[i] when i<2). Therefore, the output of layer 2 can be dat2[255:0][7:0] and msk2[255:0].

[0075] In layer 3, each multiplexer can select from the output of layer 2: dat2[i] or dat2[i+4] (where dat2[i+4] is zero for i>251); msk2[i] or msk2[i+4] (where msk2[i+4] is zero for i>251). For 256 control bits: each control bit for (i%32)==29 and i<222 can be directly obtained from the pre-calculated pop_cnts bit (for example, pop_cnts[0][2] can be the value of "sel2

[29] ", and pop_cnts[2][2] can be the value of "sel2

[93] "); and all other control bits can be the XOR of msk2[i] and sel2[i-4] (where control bit sel2[i] can be set to msk2[i] when i<4). Therefore, the output of layer 3 can be dat3[255:0][7:0] and msk3[255:0].

[0076] In layer 4, each multiplexer can select from the output of layer 3: dat3[i] or dat3[i+8] (where dat3[i+8] is zero for i>247); msk3[i] or msk3[i+8] (where msk3[i+8] is zero for i>247). For 256 control bits: each control bit for (i%32)==25 and i<218 can be directly obtained from the pre-calculated pop_cnts bit (for example, pop_cnts[0][3] can be the value of "sel3

[25] ", pop_cnts[3][3] can be the value of "sel3

[121] "); all other control bits can be the XOR of msk3[i] and sel3[i-8] (where control bit sel3[i] can be set to msk3[i] when i<8). Therefore, the output of layer 4 can be dat4[255:0][7:0] and msk4[255:0].

[0077] In layer 5, each multiplexer can select from the output of layer 4: dat4[i] or dat4[i+16] (where dat4[i+16] is zero for i>239); msk4[i] or msk4[i+16] (where msk4[i+16] is zero for i>239). For 256 control bits: each control bit for (i%32)==17 and i<210 can be directly obtained from the pre-calculated pop_cnts bit (for example, pop_cnts[0][4] can be the value of "sel4

[17] ", pop_cnts[4][4] can be the value of "sel4

[145] "); and all other control bits can be the XOR of msk4[i] and sel4[i-16] (where control bit sel4[i] can be set to msk4[i] when i<16). Therefore, the output of layer 5 can be dat5[255:0][7:0] and msk5[255:0].

[0078] In layer 6, each multiplexer can select from the output of layer 5: dat5[i] or dat5[i+32] (where dat5[i+32] is zero for i>223); msk5[i] or msk5[i+32] (where msk5[i+32] is zero for i>223). For 256 control bits: each control bit for (i%32)==1 and i<194 can be directly obtained from the pre-calculated pop_cnts bit (for example, pop_cnts[0][5] can be the value of "sel5[1]", pop_cnts[5][5] can be the value of "sel5

[161] "); and all other control bits can be the XOR of msk5[i] and sel5[i-32] (where control bit sel5[i] can be set to msk5[i] when i<32). Therefore, the output of layer 6 can be dat6[255:0][7:0] and msk6[255:0].

[0079] In layer 7, each multiplexer can select from the output of layer 6: dat6[i] or dat6[i+64] (where dat6[i+64] is zero for i>191); msk6[i] or msk6[i+64] (where msk6[i+64] is zero for i>191). For 256 control bits: each control bit for (i%32)==1 and i<162 can be directly obtained from the pre-calculated pop_cnts bit (for example, pop_cnts[1][6] can be the value of "sel6[1]", pop_cnts[6][6] can be the value of "sel6

[161] "); and all other control bits can be the XOR of msk6[i] and sel6[i-64] (where control bit sel6[i] can be set to msk6[i] when i<64). Therefore, the output of layer 7 can be dat7[255:0][7:0] and msk7[255:0].

[0080] In layer 8, each multiplexer can select from the output of layer 7: dat7[i] or dat7[i+128] (where dat7[i+128] is zero for i>127). For 256 control bits: for each control bit where (i%32)==1 and i<98, it can be obtained directly from the pre-calculated pop_cnts bit (for example, pop_cnts[4][7] can be the value of "sel7

[33] ", pop_cnts[5][7] can be the value of "sel7

[65] "); and all other control bits can be the XOR of msk7[i] and sel7[i-128] (where when i<128, the control bit sel7[i] can be set to msk7[i]). Therefore, the output of layer 8 can be compressed data, that is, Figure 6 datc[255:0][7:0]650 shown in .

[0081] Thus, the compression module 614 of the NIC can perform inline compression by taking as input the inversion of the 256-bit byte mask, seven population counts for the 7x 32-bit portions of the byte mask, and up to 256 bytes of data, performing eight levels of compression using a 2:1 multiplexer, and outputting the compressed data. By performing inline (i.e., "on the fly") compression in hardware, the described aspects can provide for more efficient transmission over a host interface (e.g., a PCIe bus).

[0082] Methods that facilitate the data structure of the data type engine and provide inline compression

[0083] Figure 7 A flowchart 700 is provided that illustrates a method of facilitating a data structure of a data type engine and providing inline compression according to aspects of the present application. During operation, a system receives a command from a network interface card (NIC) to read data from a host memory, wherein the command indicates a data type (operation 702). The system generates a plurality of read requests including an offset from a base address and a corresponding length based on the data type (operation 704). The system issues a plurality of read requests to the host memory to obtain data from the host memory (operation 706). The system obtains a byte mask descriptor corresponding to the data type (operation 708). The system performs on-the-fly compression on the obtained data based on the obtained data and the byte mask descriptor, thereby allowing the NIC to return a requested subset of the obtained data (operation 710).

[0084] A facility to facilitate the data structure of the data type engine and provide inline compression

[0085] Figure 8An apparatus 800 is shown that facilitates a data structure of a data type engine and provides inline compression according to aspects of the present application. The apparatus 800 may represent a network interface card (NIC) and may include: a command unit 802 for receiving a command to read data from a host memory, wherein the command indicates a data type; a data type management unit 808 for managing and storing data types (i.e., data type descriptors) in the NIC; a data type engine unit 804 for generating a plurality of read requests including an offset from a base address and a corresponding length based on the data type; a direct memory access unit 806 for issuing a plurality of read requests to the host memory to obtain data from the host memory; a byte mask management unit 810 for obtaining a byte mask descriptor corresponding to the data type and performing a bit-by-bit reversal in the NIC; and a data compression unit 812 for performing on-the-fly compression on the obtained data based on the obtained data and the byte mask descriptor, thereby allowing the NIC to return a requested subset of the obtained data.

[0086] The data type engine unit 804 may correspond to Figure 5 The direct memory access unit 806 may correspond to the DTE module 520. Figure 5 The data type management unit 808 may correspond to Figure 5 The non-byte mask descriptor memory 522 and the byte mask management unit 810 may correspond to Figure 5 The data compression unit 812 may correspond to the byte mask descriptor memory 524. Figure 5 The compression module 536 and Figure 6 The compression module 614 of the apparatus 800 may perform the operations described herein. Figure 5 and 6 Related to the above operations.

[0087] In general, the disclosed aspects provide systems, methods, apparatus, and non-transitory storage media for facilitating data structures of a data type engine and providing inline compression. In one aspect, a system receives a command from a network interface card (NIC) to read data from a host memory, wherein the command indicates a data type. The system generates a plurality of read requests including an offset from a base address and a corresponding length based on the data type. The system sends the plurality of read requests to the host memory to obtain data from the host memory. The system obtains a byte mask descriptor corresponding to the data type. The system performs on-the-fly compression on the obtained data based on the obtained data and the byte mask descriptor, thereby allowing the NIC to return a requested subset of the obtained data.

[0088] In a variation of this aspect, a data type engine of the NIC generates a plurality of read requests and sends the plurality of read requests to a direct memory access module of the NIC. The direct memory access module of the NIC receives the plurality of read requests, issues the plurality of read requests to a host memory, retrieves data from the host memory, and sends the obtained data to a compression module of the NIC. The compression module of the NIC receives the obtained data and a byte mask descriptor corresponding to the data type and performs on-the-fly compression of the obtained data.

[0089] In a further variation of this aspect, the system performs on-the-fly compression to obtain compressed data. The system sends the compressed data to a packet buffer of a direct memory access module, and the direct memory access module processes the response and returns the compressed data in response to the command.

[0090] In another variation, the system performs on-the-fly compression of the acquired data by processing the acquired data from the host memory using a first number of multiplexers (e.g., 8) and an inversion of the byte mask descriptor, wherein the first number of multiplexers is based on a logarithmic function of the number of bits in the byte mask (e.g., 256). The multiplexers in each layer select from a pair of data bytes and a pair of mask bits in increasing order based on a control bit, wherein the control bit is generated based on a second number overall count, wherein the second number (e.g., 7) is associated with the first number (e.g., 8).

[0091] In another variant, the second number of the population count is generated based on software instructions.The corresponding population count indicates the number of zeros in the inversion of the byte mask descriptor and corresponds to 32 bits in the obtained byte mask descriptor.

[0092] In another variation, the byte mask descriptor includes a plurality of bits (eg, 256), wherein each bit indicates whether a corresponding byte of the obtained data is to be retained or compressed.

[0093] In another variation, the data type indicates a data structure comprising a multi-dimensional array, wherein the multi-dimensional array comprises: a number of elements in each dimension; a size of a block to be transferred; and a stride in each dimension.

[0094] In another variation, the system unrolls a loop corresponding to a multidimensional array so that the total amount of data read by the unrolled loop is up to a predetermined number of bytes (eg, 256) in length.

[0095] In another variation, the byte mask descriptor is stored in a shared memory space, and descriptors of data types (including a first descriptor of the data type indicated in the command) are stored in the shared memory space.

[0096] On the other hand, a network interface card (NIC) is provided. The NIC includes a command module for receiving a command to read data from a host memory, wherein the command indicates a data type. The NIC also includes a data type engine module for generating multiple read requests including an offset from a base address and a corresponding length based on the data type. The NIC also includes a direct memory access module for issuing multiple read requests to the host memory to obtain data from the host memory. The NIC also includes a byte mask management module for obtaining a byte mask descriptor corresponding to the data type. The NIC also includes a data compression module for performing on-the-fly compression on the obtained data based on the obtained data and the byte mask descriptor, thereby allowing the NIC to return a requested subset of the obtained data.

[0097] Another aspect provides a non-transitory computer-readable storage medium storing instructions, wherein the instructions, when executed by a computer, cause the computer to perform a method as described above and herein.

[0098] The descriptions of the above aspects are given for the purpose of illustration and description only. They are not intended to be exhaustive or to limit the aspects described herein to the disclosed forms. Therefore, many modifications and variations will be apparent to those skilled in the art. In addition, the above disclosure is not intended to limit the aspects described herein. The scope of the aspects described herein is defined by the appended claims.

Claims

1. A computer-implemented method comprising: receiving, by a network interface card (NIC), a command to read data from a host memory, wherein the command indicates a type of data; generating a plurality of read requests based on the data type, the plurality of read requests comprising an offset from a base address and a corresponding length; issuing the plurality of read requests to the host memory to obtain the data from the host memory; Obtaining a byte mask descriptor corresponding to the data type; as well as On-the-fly compression of the obtained data is performed based on the obtained data and the byte mask descriptor, thereby allowing the NIC to return a requested subset of the obtained data.

2. The method according to claim 1, in, A data type engine of the NIC generates the plurality of read requests and sends the plurality of read requests to a direct memory access module of the NIC, wherein the direct memory access module of the NIC receives the multiple read requests, issues the multiple read requests to the host memory, obtains the data from the host memory, and sends the obtained data to the compression module of the NIC, and The compression module of the NIC receives the obtained data and the byte mask descriptor corresponding to the data type and performs the on-the-fly compression of the obtained data.

3. The method according to claim 1, wherein the on-the-fly compression is performed to obtain compressed data, and wherein the method further comprises: sending the compressed data to a packet buffer of a direct memory access module that processes responses; as well as The compressed data is returned in response to the command.

4. The method of claim 1 , wherein performing the on-the-fly compression of the obtained data comprises: processing the obtained data from the host memory using a multiplexer of a first number of layers and an inverse of the byte mask descriptor, wherein the first number of layers is based on a logarithmic function of the number of bits in the byte mask, wherein a multiplexer in each layer selects from a pair of data bytes and a pair of mask bits in increasing order based on a control bit generated based on a second number of the total count, wherein the second number is associated with the first number.

5. The method according to claim 4, in, the second number of overall counts being generated based on software instructions, and Therein, a corresponding population count indicates the number of zeros in the inversion of the byte mask descriptor and corresponds to 32 bits in the obtained byte mask descriptor.

6. The method according to claim 1, in, The byte mask descriptor includes a plurality of bits, wherein each bit indicates whether a corresponding byte of the obtained data is to be retained or compressed.

7. The method according to claim 1, in, The data type indicates a data structure including a multi-dimensional array, the multi-dimensional array including: The number of elements in each dimension; the size of the blocks to be transmitted; and The stride in each dimension.

8. The method according to claim 7, further comprising: A loop corresponding to the multidimensional array is unrolled so that a total amount of data read by the unrolled loop reaches at most a predetermined number of bytes in length.

9. The method according to claim 1, in, The byte mask descriptor is stored in a shared memory space, and The descriptors of the data types are stored in the shared memory space, wherein the descriptors of the data types include a first descriptor of the data type indicated in the command.

10. A network interface card, comprising: A command module, configured to receive a command to read data from a host memory, wherein the command indicates a data type; A data type engine module, configured to generate a plurality of read requests based on the data type, the plurality of read requests comprising an offset from a base address and a corresponding length; a direct memory access module, configured to issue the plurality of read requests to the host memory to obtain the data from the host memory; A byte mask management module, used for obtaining a byte mask descriptor corresponding to the data type; as well as A data compression module is configured to perform on-the-fly compression of the obtained data based on the obtained data and the byte mask descriptor, thereby allowing the network interface card (NIC) to return a requested subset of the obtained data.

11. The network interface card according to claim 10, in, The data type engine module is further configured to send the plurality of read requests to a direct memory access module of the NIC, The direct memory access module is further configured to receive the multiple read requests before issuing the multiple read requests to the host memory, obtain the data from the host memory, and send the obtained data to the data compression module, and The data compression module is further used to receive the obtained data and the byte mask descriptor corresponding to the data type.

12. The network interface card according to claim 10, in, The data compression module is further configured to obtain compressed data by performing the on-the-fly compression and to send the compressed data to a packet buffer of the direct memory access module that processes the response; and The direct memory access module is further configured to return the compressed data in response to the command.

13. The network interface card according to claim 10, wherein: The data compression module performs the instant compression on the obtained data by the following steps: processing the obtained data from the host memory using a multiplexer of a first number of layers and an inverse of the byte mask descriptor, wherein the first number of layers is based on a logarithmic function of the number of bits in the byte mask, wherein a multiplexer in each layer selects from a pair of data bytes and a pair of mask bits in increasing order based on a control bit, wherein the control bit is generated based on a second number of the total count, wherein the second number is associated with the first number, wherein the second number of the total count is generated based on software instructions, and Therein, a corresponding population count indicates the number of zeros in the inversion of the byte mask descriptor and corresponds to 32 bits in the obtained byte mask descriptor.

14. The network interface card according to claim 10, in, The byte mask descriptor includes a plurality of bits, wherein respective bits indicate that corresponding bytes of the obtained data are to be retained or compressed.

15. The network interface card according to claim 10, in, The data type indicates a data structure including a multi-dimensional array, the multi-dimensional array including: The number of elements in each dimension; The size of the block to be transferred; and The stride in each dimension.

16. The network interface card according to claim 15, The data type engine module is further used to expand the loop corresponding to the multidimensional array so that the total amount of data read by the expanded loop reaches at most a predetermined number of bytes in length.

17. The network interface card according to claim 10, in, The byte mask descriptor and the data type descriptor are stored in a shared memory space of the network interface card.

18. A non-transitory computer-readable storage medium storing instructions, wherein the instructions, when executed by a computer, cause the computer to perform a method comprising: storing, by a data type engine unit of a network interface card (NIC), one or more data type descriptors created by a user associated with the computer; receiving, by the NIC, a command to read data from a host memory, wherein the command indicates a data type descriptor; generating, by the data type engine unit, a plurality of read requests based on the data type, the plurality of read requests including an offset from a base address and a corresponding length; issuing, by a direct memory access unit of the NIC, the plurality of read requests to the host memory to obtain the data from the host memory; Obtaining, by a data compression unit of the NIC, a byte mask descriptor corresponding to the data type; as well as On-the-fly compression of the obtained data is performed by the data compression unit based on the obtained data and the byte mask descriptor, thereby allowing the NIC to return the requested subset of the obtained data.

19. The non-transitory computer-readable storage medium of claim 18, in, The byte mask descriptor includes a first number of bits, wherein each bit indicates whether a corresponding byte of the obtained data is to be processed in a first manner or in a second manner, and The data compression unit performs the instant compression of the obtained data by the following steps: processing the obtained data from the host memory using a multiplexer of a second number of layers and an inverse of the byte mask descriptor, wherein the second number of layers is based on a logarithmic function of the first number of bits in the byte mask, wherein a multiplexer in each layer selects from a pair of data bytes and a pair of mask bits in increasing order based on a control bit generated based on a third number of the overall count, wherein the third number of the total count is generated based on software instructions, and Therein, a corresponding population count indicates the number of zeros in the inversion of the byte mask descriptor and corresponds to 32 bits in the obtained byte mask descriptor.

20. The non-transitory computer-readable storage medium of claim 19, wherein: The first mode and the second mode include at least one of the following: retaining the corresponding bytes; compressing the corresponding bytes; and A predetermined action is performed on the corresponding byte.

Images