Remote Direct Memory Access with Offset

Abstract

The present disclosure relates to Remote Direct Memory Access (RDMA). This disclosure provides a device and corresponding method for RDMA that can initiate at least one RDMA operation on a memory device and implement effective zero-copy on a target device. For example, a target device for an RDMA operation is configured to receive a data packet comprising a first destination address and a destination key; obtain at least one offset value; and obtain a second destination address based on the first destination address, the destination key, and the at least one offset value. Furthermore, the target device is configured to initiate an RDMA operation on the memory device based on the second destination address.

Description

Technical Field

[0001] The present disclosure relates to Remote Direct Memory Access (RDMA). The present disclosure provides devices and corresponding methods for RDMA that can initiate at least one RDMA operation in memory, particularly in memory of a remote host. In particular, the present disclosure relates to achieving zero copy on the target. Background Art

[0002] RDMA is a method for communicating between hosts in a computer network. It typically requires specialized hardware support from at least one network interface card (NIC). Most common devices and methods conform to the widely accepted "Infiniband" standard, promulgated by the Infiniband Trade Association (IBTA), which covers hardware specifications, protocol formats, and practical implementations. It's worth noting that some NIC vendors have extended their implementations with additional non-standard features.

[0003] For example, RDMA is widely used in data centers and clusters, especially for low-latency, high-bandwidth networks. RDMA operations include the following types (where (1) represents a "single-sided operation" and (2) represents a "two-sided operation"):

[0004] 1. RDMA read operation (1): Given a memory location, this RDMA operation reads the memory location from the remote host's memory.

[0005] 2. RDMA write operation (1): Given data and a memory location, this RDMA operation writes the data at the memory location to the memory of the remote host.

[0006] 3. RDMA send operation (2): Given data, this RDMA operation writes the data to the next available buffer in the remote host queue.

[0007] 4. RDMA atomic operation (1): Given data and a memory location, this RDMA operation updates the remote host's memory with the data at the memory location (this operation is atomic, meaning it is guaranteed not to be interrupted).

[0008] like Figure 6 As an example, a conventional device for RDMA performs any of the above operations based on a data packet including at least a memory location (such as a virtual or physical address of the memory). Figure 6Schematically, a conventional target device 600 (also referred to as a physical target host) is shown. Target device 600 includes virtual storage 607, a NIC 609, and (physical) memory 605. Target device 600 performs an RDMA write operation in memory 605. The RDMA operation is based on an RDMA packet 601, which includes a destination virtual address (VA) 602, a memory key 603, data to be written 604, and a data length 606 of the data 604. The initiator device (not shown) of the RDMA operation provides packet 601. Conventionally, target device 600 does not control the location of the destination memory defined by destination VA 602.

[0009] Specifically, when data packet 601 arrives, the following steps may be performed at target device 600:

[0010] 1. Based on the memory key 603 contained in the data packet 601:

[0011] 1.1 Check memory area access permissions.

[0012] 1.2 Based on the destination VA 602 and data length 606 in the data packet 601, check whether the RDMA operation is within the range of the memory 605.

[0013] 1.3 Convert the destination VA 603 to the destination PA.

[0014] 2. According to the operation code 608 in the data packet 601, the RDMA operation type is indicated:

[0015] 2.1 RDMA write operation: write the message payload (data 604) to the calculated destination physical address (PA).

[0016] 2.2 RDMA read operation: reads data (of a given length) from the calculated destination PA and sends it back to the initiator device.

[0017] 2.3 RDMA atomic operation: Perform one of several standard atomic operations on the calculated destination PA.

[0018] The problem with this conventional RDMA operation, and therefore conventional devices and methods suitable for RDMA, is that it is difficult to achieve efficient communication via zero copy on the target device. In particular, zero copy on the target is more difficult than on the initiator side because it is difficult for the target device to control / anticipate incoming packets / data and their sizes. Notably, zero copy refers to a method in which each message (such as a packet containing data) is only passed once (a) from the initiator region to its NIC (zero copy send) and (b) from the target NIC to the target region (zero copy receive). Zero copy is desirable because it reduces communication latency.

[0019] Conventional ad hoc approaches, such as "tag matching", do not fully address the above issues. In this approach, the target device notifies the NIC of the expected message (and its area). Incoming messages are then identified based on a list of expected tags using some fixed-size prefix ("tag"). If the incoming message matches the tag, the corresponding storage area is used (zero copy). If it does not match, it is written to a temporary buffer (for later copying, so no zero copy is performed). The disadvantage of this approach is that it is not "transparent" to actual applications because the code needs to be modified to use it (specify the tag). In addition, it is slower than other approaches due to the high cost of software (SW)-hardware (HW) synchronization. In addition, this approach also requires the use of non-standard hardware. Summary of the Invention

[0020] In view of the above problems and shortcomings, embodiments of the present invention are intended to improve conventional devices, methods and approaches. The present invention is intended to provide a device and method for RDMA that can perform effective zero copy on the target.

[0021] This object is achieved by the embodiments of the invention as described in the appended independent claims, and advantageous implementations of the embodiments of the invention are further defined in the dependent claims.

[0022] In one embodiment of the present invention, based on any combination of the aforementioned RDMA operations (RDMA read, write, and atomic operations), atomic access is performed on at least one offset value (wherein the access may include incrementing at least one offset value based on the size of the RDMA read / write operation, but other types of access are also possible). This combination can be "atomic," meaning that the operation is guaranteed to be uninterrupted (which is beneficial for multiple concurrent operations).

[0023] In a first aspect of the present disclosure, a device for RDMA is provided, wherein the device is configured to perform an RDMA operation on a memory, the device being configured to: receive a data packet, the data packet including a first destination address and a destination key; obtain at least one offset value; obtain a second destination address based on the first destination address, the destination key, and the at least one offset value; and initiate the RDMA operation on the memory based on the second destination address.

[0024] Thus, a new RDMA operation is introduced, in particular an RDMA operation with offset. The device of the first aspect may be the target device of the RDMA operation. The new RDMA operation can achieve the above-mentioned zero copy on the target.

[0025] According to the implementation form of the first aspect, the data packet also includes at least one first offset address and at least one offset key, and the device is further used to obtain the at least one offset value based on the at least one first offset address and the at least one offset key.

[0026] According to an implementation form of the first aspect, the device is further configured to initiate modification of the at least one offset value after performing the RDMA operation on the memory.

[0027] According to an implementation form of the first aspect, the modification of the at least one offset value comprises increasing or decreasing the at least one offset value.

[0028] According to an implementation form of the first aspect, the data packet further includes a data length, and the device is configured to initiate modification of the at least one offset value based on the data length.

[0029] According to the implementation form of the first aspect, the first destination address is the virtual address of the virtual memory of the device, the second destination address is the physical address of the memory, and the device is used to convert the first destination address into the second destination address based on the base virtual address, the base physical address of the memory and the at least one offset value.

[0030] According to an implementation form of the first aspect, the device is further configured to obtain at least one second offset address based on the at least one first offset address, and to obtain the at least one offset value using the at least one second offset address.

[0031] According to the implementation form of the first aspect, the at least one first offset address is a virtual address of the virtual memory of the device, and the at least one second offset address is a physical address of the memory. The device is used to convert the at least one first offset address into the at least one second offset address based on at least one offset base virtual address and an offset base physical address of the memory.

[0032] According to an implementation form of the first aspect, the device is configured to obtain the at least one offset base virtual address and the offset base physical address based on the at least one offset key.

[0033] According to an implementation form of the first aspect, the at least one offset value is stored in the device, and / or the at least one offset value is stored in the memory, wherein the device includes the memory, or the memory is a remote memory, and the device is used to communicate with the memory.

[0034] For the device of the first aspect, "remote memory" is a memory of a remote host or device, that is, a memory that is neither constituted by nor located in the device of the first aspect. For example, the memory may be a memory of a third device.

[0035] A second aspect of the present disclosure provides a device for RDMA, wherein the device is used to initiate an RDMA operation in a remote memory, and the device is used to: provide a data packet to another device, the data packet including a destination address, a destination key, at least one offset address, and at least one offset key, wherein the destination address indicates the location of the RDMA operation in the remote memory, and the at least one offset address indicates the location of at least one offset value in the other device or the remote memory.

[0036] Thus, a new RDMA operation is introduced, in particular an RDMA operation with an offset. The device of the second aspect may be the initiator device of the RDMA operation. The new RDMA operation can achieve the above-mentioned zero copy on the target.

[0037] For the apparatus of the second aspect, "remote memory" is a memory of a remote host or device, i.e., a memory that is neither constituted by the apparatus of the first aspect nor located in the apparatus of the second aspect. For example, the memory may be constituted by the apparatus of the first aspect or located in the apparatus of the first aspect.

[0038] According to an implementation form of the second aspect, the RDMA operation is an RDMA write operation, or an RDMA read operation, or an RDMA atomic operation.

[0039] The RDMA operation may be determined or indicated by an operation code in the data packet.

[0040] A third aspect of the present disclosure provides a method for RDMA, wherein the method is used to perform an RDMA operation on a memory, the method comprising: receiving a data packet, the data packet including a first destination address and a destination key; obtaining at least one offset value; obtaining a second destination address based on the first destination address and the at least one offset value; and initiating the RDMA operation in the memory based on the second destination address.

[0041] According to the implementation of the third aspect, the data packet also includes at least one first offset address and at least one offset key, and the method is also used to obtain the at least one offset value based on the at least one first offset address and the at least one offset key.

[0042] According to an implementation of the third aspect, the method further includes initiating a modification of the at least one offset value after executing the RDMA operation in the memory.

[0043] According to an implementation manner of the third aspect, modifying the at least one offset value includes increasing or decreasing the at least one offset value.

[0044] According to an implementation of the third aspect, the data packet further includes a data length, and the method further includes initiating a modification of the at least one offset value based on the data length.

[0045] According to the implementation of the third aspect, the first destination address is a virtual address of a virtual memory, the second destination address is a physical address of the memory, and the method further includes converting the first destination address into the second destination address based on the base virtual address, base physical address and the at least one offset value of the memory.

[0046] According to an implementation manner of the third aspect, the method further includes obtaining at least one second offset address based on the at least one first offset address, and obtaining the at least one offset value using the at least one second offset address.

[0047] According to the implementation of the third aspect, the at least one first offset address is a virtual address of the virtual memory, and the at least one second offset address is a physical address of the memory. The method also includes converting the at least one first offset address into the at least one second offset address based on at least one offset base virtual address and offset base physical address of the memory.

[0048] According to an implementation of the third aspect, the method further includes obtaining the at least one offset base virtual address and the offset base physical address based on the at least one offset key.

[0049] According to the implementation of the third aspect, the at least one offset value is stored in the device, and / or the at least one offset value is stored in the memory, wherein the device includes the memory, or the memory is a remote memory, and the device is used to communicate with the memory.

[0050] The method of the third aspect provides the same advantageous benefits as the apparatus of the first aspect.

[0051] A fourth aspect of the present disclosure provides a method for RDMA, wherein the method is used to initiate an RDMA operation on a remote memory, the method comprising providing a data packet, the data packet comprising a destination address, a destination key, at least one offset address, and at least one offset key, wherein the destination address represents a location of the RDMA operation on the remote memory, and the at least one offset address represents a location of at least one offset value in a device storing the at least one offset value or a location on the remote memory.

[0052] According to an implementation of the fourth aspect, the RDMA operation is an RDMA write operation, or an RDMA read operation, or an RDMA atomic operation.

[0053] The method of the fourth aspect provides the same advantageous benefits as the apparatus of the second aspect.

[0054] A fifth aspect of the present disclosure provides a computer program, which includes program code. When the program code is run on a computer, the method according to the third aspect or the fourth aspect or any implementation thereof is executed.

[0055] A sixth aspect of the present disclosure provides a non-transitory storage medium storing executable program code, which, when executed by a processor, performs the method according to the third aspect or the fourth aspect or any implementation thereof.

[0056] In summary, various aspects and implementations (embodiments of the present invention) define a new class of RDMA operations (RDMA operations with offsets), all of which utilize at least one offset value, thereby expanding the list of currently available RDMA operation types. This list of operation types does not utilize offset values ​​(e.g., expanding RDMA write operations to RDMA write with offsets). Adding at least one offset value to the target device to be used or updated during / after any new RDMA operation addresses the aforementioned issues. With each data packet arriving, the actual storage destination in the target memory depends on the value of the at least one offset value (e.g., maintained in an offset register on the target device or memory). For example, by adding the data length of the data to the offset register, the next data item (using the same offset register, now containing the updated offset value) can be written immediately after the previous data item in memory. This new mechanism for calculating the destination memory for these RDMA operations can be used in practical applications, enabling zero-copy on the target without requiring any per-data software operations (perhaps requiring only a one-time setup).

[0057] It should be noted that all devices, elements, units and methods described in this application can be implemented in software or hardware elements or any combination thereof. All steps performed by the various entities described in this application and the functions performed by the various entities described are intended to indicate that the respective entities are applicable to or used to perform the corresponding steps and functions. Even if in the following description of a particular embodiment, a specific function or step performed by an external entity is not reflected in the description of the specific detailed elements of the entity that performs the specific step or function, it should be clear to those skilled in the art that these methods and functions can be implemented in respective software or hardware elements, or any combination thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The above aspects and implementations of the present invention (embodiments of the present invention) will be explained in the following description of specific embodiments in conjunction with the accompanying drawings, wherein

[0059] Figure 1 shows devices interacting with each other according to an embodiment of the present invention;

[0060] Figure 2 An RDMA write operation with explicit offset is shown;

[0061] Figure 3 The results of an RDMA write operation with explicit offset are shown;

[0062] Figure 4 An implicit RDMA write operation with offset is shown;

[0063] Figure 5 A method for performing RDMA operations according to an embodiment of the present invention is shown; and

[0064] Figure 6 A conventional RDMA write operation is shown. DETAILED DESCRIPTION

[0065] Figure 1 Device 100 and device 110 according to an embodiment of the present invention are shown. Both devices 100 and 110 are suitable for RDMA. Specifically, device 100 is a target device 100 of an RDMA operation, i.e., it is used to perform the RDMA operation, and device 110 is an initiator device 110 of the RDMA operation, i.e., it is used to initiate the RDMA operation.

[0066] Target device 100 can perform RDMA operations in (physical) memory 105. Target device 100 may include memory 105. However, memory 105 may also be "remote memory" of target device 100, such as memory belonging to another device. For initiator device 110, regardless of whether memory 105 is included in target device 100 or another device, memory 105 is "remote memory" in all cases. Hereinafter, only "memory 105" will be discussed.

[0067] To perform an RDMA operation in memory 105, target device 100 is configured to receive a data packet 101, where data packet 101 includes at least a first destination address 102 and a destination key 103. Target device 100 may receive data packet 101 from initiator device 110. Initiator device 110 is configured to provide data packet 101 to initiate an RDMA operation in memory 105. Destination address 102 indicates the location of the RDMA operation in remote memory 105.

[0068] In addition, the target device 100 is configured to obtain at least one offset value 104. Different methods for obtaining the at least one offset value 104 will be further described below. Next, the target device 100 is configured to obtain a second destination address 106 based on the first destination address 102, the destination key 103, and the at least one offset value 104. Then, the target device 100 is configured to initiate and / or perform an RDMA operation in the memory 105 based on the second destination address 106. Therefore, the RDMA operation can be an RDMA write operation, an RDMA read operation, or an RDMA atomic operation, that is, more precisely, an RDMA write operation with an offset, an RDMA read operation with an offset, or an RDMA atomic operation with an offset.

[0069] The target device 100 and / or the initiator device 110 may include a processor or processing circuit (not shown) for executing, performing, or initiating the various operations of the target device 100 and / or the initiator device 110 described herein. The processing circuit may include hardware and / or the processing circuit may be controlled by software. The hardware may include analog circuits or digital circuits, or both. The digital circuits may include components such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), or multi-function processors.

[0070] The target device 100 and / or the initiator device 110 may further include a storage circuit that stores at least one instruction executable by a processor or processing circuit, particularly under software control. For example, the storage circuit may include a non-transitory storage medium that stores executable software code that, when executed by the processor or processing circuit, is used to perform various operations of the target device 100 and / or the initiator device 110.

[0071] In one embodiment, the processing circuit includes at least one processor and a non-transitory memory connected to the at least one processor. The non-transitory memory may contain executable program code that, when executed by the at least one processor, causes the target device 100 and / or the initiator device 110 to perform, conduct, or initiate the operations or methods described herein.

[0072] Figure 2An example of an "explicit" offset-bearing RDMA write operation is shown (wherein the type of RDMA operation may be determined or indicated by an operation code 205 in a data packet 101), which may be performed by a target device 100 (also referred to as the "physical target host of the RDMA operation") and initiated by an initiator device 110. In particular, a data packet 101 is shown as being provided by an initiator device 110 (not shown). Based on this data packet 101, the target device 100 may perform an offset-bearing RDMA write operation in a memory 105. In this example, the target device 100 includes the memory 105. Furthermore, in this example, the target device 100 also includes a virtual memory 206 and a NIC 207. Because at least one offset value 104 to be used in this RDMA operation is indicated (explicitly) by the data packet 101, the RDMA operation is referred to as "explicit."

[0073] In particular, data packet 101 includes a first destination address 102 and a destination key 103. Figure 2 In the example of , the destination key 103 is a memory key of the memory 105, and the first destination address 102 is a destination VA. The destination VA 102 is an address of the virtual memory 206 and is associated with a buffer of the target device 100. The data packet 101 also includes a first offset address 201 and an offset key 202. Figure 2 In the example of FIG, first offset address 201 is offset VA. Offset VA 201 is an address of virtual memory 206 and is associated with the storage location of offset value 104. Offset VA 201 can represent a physical location 208 of offset value 104 in memory 105 of target device 100 (or alternatively, in remote memory 105 of another device used to communicate with target device 100 (alternative device not shown)).

[0074] Data packet 101 may further include data 204 (to be written). If the RDMA operation is an RDMA read operation with an offset, data packet 101 will not include data 204. Data packet 101 may also include a data length 203 of data 204. Finally, data packet 101 may further include an operation code 205 for identifying or indicating the RDMA operation to be performed.

[0075] The data 204 may be written by the target device 100 to a physical location 209 in the memory 105 (such as a physical location 209 in a specific storage area of ​​the memory 105, where the storage area may be indicated by the memory key 103). The physical location 209 corresponds to the second destination address 106 (in Figure 2In the example shown in FIG2 , the second destination address 106 is a destination physical address (PA) of the memory 105, which can be obtained by the target device 100 based on the destination VA 102 provided by the initiator device 110 and further based on the offset value 104. That is, the data 204 can be written to the destination PA 106. In particular, the data 204 can be first written using the destination VA 102 of the virtual memory 206, i.e., written to the buffer of the target device 100, and then written to the memory 105 by the target device 100 using the destination PA 106, i.e., written to the physical location 209.

[0076] Thus, both the initiator device 110 and the target device 100 can contribute to the generation of the second destination address 106. In particular, by adding the offset value 104 on the target device 100 side, the target device 100 can be made to control the destination of the write operation, thus changing the nature of the operation from single-sided (controlled only by the initiator device 110) to dual-sided (controlling both devices simultaneously).

[0077] exist Figure 2 In the example of , the target device 100 may convert the destination VA 102 into the destination PA 106 based on the base VA, the base PA of the memory 105 , and the at least one offset value 104 to obtain the destination PA 106 of the memory 105 .

[0078] The base VA and / or base PA may be known to the target device 100, or may be acquired or received by the target device 100. Figure 2 As shown, the base VA and base PA may be used or maintained by the NIC 207. For example, the target device 100, and more particularly the NIC 207, may calculate the target PA 106 based on the offset value 104 according to the following equation:

[0079] Destination PA = destination VA - base VA + base PA + offset value

[0080] Thus, the second offset address can be used to obtain the offset value 104. Figure 2 In the example, the second offset address is offset PA, according to the following formula:

[0081] Offset value = <memory contents of offset PA>

[0082] The offset PA identifies a physical location 208 in the memory 105 where the offset value 104 is stored and can be retrieved from the physical location 208 via the NIC 207 .

[0083] The target device 100 may calculate the offset PA based on the offset base VA and the offset base PA of the memory 105 (known and / or obtained by the target device 100, in particular, maintained by the NIC 207) and the destination offset VA 201 according to the following equation:

[0084] Offset PA = target offset VA - offset base VA + offset base PA

[0085] According to the following equation, after the target device 100 writes the data 204 using the destination PA 106, it may also initiate an increase in at least one offset value 104 based on the data length 203, such as modifying the offset value 104 currently stored in the memory 105:

[0086] <Offset PA memory content> = offset value + data length

[0087] That is, the data length 204 is increased to the current offset value 104 , and the new offset value 104 thus obtained is stored at the physical location 208 in the memory 105 .

[0088] In some embodiments, the offset value 104 may also be selected by the initiator device 110 (particularly for Figure 2 ), or the offset value 104 may be selected by the target device 100, such as when the data packet 101 arrives. However, the offset value 104 may be located on the target device 100 and may affect the calculation of the second destination address 106 in either manner.

[0089] The offset value 104 may generally be modified after any RDMA operation, such as by increasing or decreasing the offset value 104. As described above, the offset value 104 may be increased by the size of the incoming data 204 (e.g., as indicated by the data length 203, contained in the data packet 101) so that a subsequent RDMA write operation with the same offset (specifying the same destination address 102, such as the same destination VA 102, in the data packet 101) does not overwrite the previously written data 204, but instead writes the new data 204 immediately after the previously written data 204 in the memory 105 at a different physical location. In particular, according to Figure 2 The example and the above equation:

[0090] Destination PA = destination VA - base VA + base PA + offset value

[0091] If the offset value 104 changes (such as by being increased by the target device 100 ), the same destination VA 102 in the next data packet 101 will result in a different destination PA 106 , ie, the data 204 will be written to a different physical location 208 in the memory 105 .

[0092] Therefore, when the same another data packet 101 arrives, the other data 204 can be logically appended to the previously written data 204, that is, the other data 204 included in the same another data packet 101 can be placed after the previously written data 204 of the previous data packet 101, as shown in FIG. Figure 3 shown.

[0093] In particular, Figure 3 An example of an RDMA write operation with offset is shown (see operation code 205 ), where the value of destination VA 102 is 0x2111, the value of memory key 103 is 0x1, the value of destination VA 102 is 0x3010, the value of memory key 202 is 0x3, the value of data length is 0x5, and the data 204 is "ABCDE." Furthermore, the value of base VA is 0x2000, the value of base PA is 0x0, the value of offset base VA is 0x3000, and the value of offset base PA is 0x2000. Furthermore, the given values ​​are hexadecimal values, where 0x indicates a hexadecimal value.

[0094] According to the above given equation, the value of offset PA is calculated by using the above values ​​to be 0x2010 (i.e., offset PA = 0x3010 - 0x3000 + 0x2000). It is worth noting that offset VA 201 refers to the virtual memory 206 of the target device 100, and the NIC 207 converts offset VA 201 into offset PA. Figure 3 As schematically shown, the offset value 104 can be obtained from the physical location 208 of the memory 105 corresponding to the offset PA, and the value is 0x123.

[0095] Based on the offset value 104, the destination PA 106 can be calculated using the above values ​​to be 0x234 (ie, destination PA = 0x2111 - 0x2000 + 0x0 + 0x123).

[0096] Then, the target device 100, specifically the NIC 207, writes the data "ABCDE" to the physical location 209 of the memory 105 corresponding to the destination PA. Thus, the data 204 can be first written to the buffer of the destination device 100 using the destination VA 102, and then the data 204 can be written from the buffer to the physical location 209 using the destination PA 106.

[0097] After writing data 204, target device 100, particularly NIC 207, may increase offset value 104 by the length 103 of the data, ie, by a value of 0x5. Thus, new offset value 104 becomes 0x128 (ie, new offset value = 0x5 + 0x123).

[0098] If another data packet 101 now arrives from initiator device 110 (with the same destination VA 102), since the (new) offset value 104 is now different—i.e., it is now 0x128 instead of 0x123—then the data 204 of the other data packet 101 will not overwrite the previously written data 204. Specifically, the destination PA 106 of the next RDMA write operation can be calculated as: Destination PA = 0x2111–0x2000+0x0+0x128. Therefore, the data 204 of the other data packet 101 will logically be appended to the previously written data 204. Subsequently, the offset value 104 can be increased again to the value 0x12D = 0x128+0x5 (if the data length 203 of the other data 204 is again 0x5).

[0099] As described above for the RDMA write operation with an offset, the changes in the data packet 101 (compared to conventional systems and conventional RDMA write operations) and the calculation of the second destination address 106 can also be applied to other RDMA operation types. That is, the initiator device 110 and the target device 100 can also perform RDMA read operations with an offset and / or RDMA atomic operations with an offset (not shown in the figure).

[0100] Figure 4 Shown as Figure 2 and Figure 3 An alternative example of an "explicit" RDMA write with offset is shown in the example in . Figure 4 An “implicit” offset RDMA write operation is shown. This operation is called “implicit” because at least one offset value 104 is not indicated by the data packet 101 but is implicitly obtained by the target device 100. It is worth noting that Figure 2 and Figure 4 The same elements are represented by the same symbols.

[0101] like Figure 6 As shown in FIG. 1 , in this operation, data packet 101 remains unchanged compared to data packet 601 of a conventional RDMA write operation. That is, data packet 101 includes operation code 205, destination VA 102, storage key 103, data 204, and data length 203. However, as Figure 2 As shown, the data packet 101 does not include the destination offset VA 201 and the offset key 202. Nevertheless, the target device 100 still uses the offset value 104 (in the same manner as the offset value 104) when converting the destination VA 102 to the destination PA 106. Figure 2 The offset value 104 may be controlled by the target device 100, such as Figure 2 As described in "Explicit" RDMA write with offset. Figure 2As shown in the previous "explicit" variant, the offset value 104 can be stored in the target device 100, such as in the memory 105, in particular in the physical location 208. The target device 100 obtains the offset value 104 during its own movement without being given any information about this by the data packet 101.

[0102] RDMA operations with offsets can be further based on the following principles:

[0103] First, the NIC 207 of the target device 100 can directly access the memory 105, thereby not involving the CPU of the target device 100. This so-called "CPU offloading" saves CPU time, freeing it up for other operations.

[0104] Secondly, when the initiator device 110 sends the data 204 of the packet 101, the application can pass the instructions directly to the NIC 207 of the target device 100 without involving its operating system (such as the Linux kernel). This operation reduces a considerable amount of overhead.

[0105] Third, in order for the NIC 207 of the target device 100 to access any region of the memory 105, the region may first be "registered" with the NIC 207. This registration notifies the NIC 207 of the memory region's VA and PA. The VA and PA of the memory region may be the base VA and base PA described above. Typically, this registration also locks the region in place (until "registration is canceled").

[0106] Combining the above principles with offset-based RDMA operations enables a solution in which each data packet 101 is delivered only once: (a) from the initiator device 110 zone to its NIC (zero-copy send), and (b) from the target device 110's NIC 207 to the destination zone (zero-copy receive). This solution significantly reduces communication latency.

[0107] Figure 5 A method for performing an RDMA operation according to an embodiment of the present invention is shown. Method 500 may be performed by target device 100. For performing an RDMA operation on memory 105, method 500 includes: receiving 501 a data packet 101, data packet 101 including a first destination address 102 and a destination key 103; obtaining 502 at least one offset value 104; obtaining 503 a second destination address 106 based on the first destination address 102 and the at least one offset value 104; and initiating 504 an RDMA operation in memory 105 based on the second destination address 106.

[0108] The invention has been described with reference to various exemplary embodiments and various implementations. However, other variations may be understood, implemented, and practiced by a person skilled in the art by studying the drawings, the present disclosure, and the independent claims. In the claims and in the specification, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single element or other unit may perform the functions of several entities or items stated in the claims. The fact that certain means are cited in mutually different dependent claims does not indicate that a combination of these means cannot be used in an advantageous embodiment.

Claims

1. A device (100) for performing a Remote Direct Memory Access (RDMA) operation in a memory (105), wherein: The device (100) is used for: Receiving a data packet (101), the data packet (101) comprising a first destination address (102), a destination key (103), at least one first offset address (201) and at least one offset key (202); Obtaining at least one offset value (104) based on the at least one first offset address (201) and the at least one offset key (202); Obtaining a second destination address (106) based on the first destination address (102), the destination key (103), and the at least one offset value (104); and The RDMA operation is initiated in the memory (105) based on the second destination address (106).

2. The device (100) according to claim 1, further configured to: After performing the RDMA operation in the memory (105), a modification of the at least one offset value (104) is initiated.

3. The device (100) according to claim 2, wherein Modifying the at least one offset value (104) includes increasing or decreasing the at least one offset value (104).

4. The device (100) according to claim 2 or 3, wherein The data packet (101) further includes a data length (203); and The device (100) is further configured to initiate a modification of the at least one offset value (104) based on the data length (203).

5. The device (100) according to any one of claims 1 to 3, wherein The first destination address (102) is a virtual address of a virtual memory (206) of the device (100), and the second destination address (106) is a physical address of the memory (105); and The device (100) is further configured to convert the first destination address (102) into the second destination address (106) based on a base virtual address, a base physical address of the memory (105), and the at least one offset value (104).

6. The device (100) according to any one of claims 1 to 3, further configured to: Obtaining at least one second offset address based on the at least one first offset address (201); The at least one offset value is obtained using the at least one second offset address (104).

7. The apparatus (100) according to claim 6, wherein: The at least one first offset address (201) is a virtual address of a virtual memory (206) of the device (100), and the at least one second offset address is a physical address of the memory (105); and The device (100) is further configured to convert the at least one first offset address (201) into the at least one second offset address based on at least one offset base virtual address and an offset base physical address of the memory (105).

8. The device (100) according to claim 7, further configured to: The at least one offset base virtual address and the offset base physical address are obtained based on the at least one offset key (202).

9. The device (100) according to any one of claims 1 to 3, further configured to: storing the at least one offset value (104) in the device (100); and / or storing said at least one offset value (104) in said memory (105), in, The device (100) includes the memory (105), or the memory (105) is a remote memory, and the device (100) is configured to communicate with the memory (105).

10. The apparatus (100) according to any one of claims 1 to 3, wherein: The RDMA operation is an RDMA write operation, an RDMA read operation, or an RDMA atomic operation.

11. A device (110) for initiating a remote direct memory access (RDMA) operation in a remote memory (105), wherein the device (110) is configured to: A data packet (101) is provided to another device (100), the data packet (101) comprising a first destination address (102), a destination key (103), at least one first offset address (201), and at least one offset key (202), so that the other device (100) obtains at least one offset value (104) based on the at least one first offset address (201) and the at least one offset key (202), and obtains a second destination address (106) based on the first destination address (102), the destination key (103), and the at least one offset value (104), wherein: The second destination address (106) indicates the location of the RDMA operation in the remote memory (105), and The at least one first offset address (201) indicates a location of at least one offset value (104) in the other device (100) or the remote memory (105).

12. The apparatus (110) of claim 11, wherein: The RDMA operation is an RDMA write operation, an RDMA read operation, or an RDMA atomic operation.

13. A method (500) of performing a remote direct memory access (RDMA) operation in a memory (105), the method (500) comprising: Receiving (501) a data packet (101), the data packet (101) comprising a first destination address (102), a destination key (103), at least one first offset address (201) and at least one offset key (202); Obtaining (502) at least one offset value (104) based on the at least one first offset address (201) and the at least one offset key (202); obtaining (503) a second destination address (106) based on the first destination address (102), the destination key (103) and the at least one offset value (104); and The RDMA operation is initiated (504) in the memory (105) based on the second destination address (106).

14. A method of initiating a remote direct memory access (RDMA) operation in a remote memory (105), the method comprising: A device (110) provides a data packet (101) to another device (100), wherein the data packet (101) includes a first destination address (102), a destination key (103), at least one first offset address (201), and at least one offset key (202), so that the other device (100) obtains at least one offset value (104) based on the at least one first offset address (201) and the at least one offset key (202), and obtains a second destination address (106) based on the first destination address (102), the destination key (103), and the at least one offset value (104). wherein the second destination address (106) represents the location of the RDMA operation in the remote memory (105), and The at least one first offset address (201) indicates a location of at least one offset value (104) in the other device (100) or in the remote memory (105).

15. A computer program product comprising a program code, which is configured to execute the method according to claim 13 or 14 when the program code is run on a computer.

Images