Method and apparatus for implementing remote memory access on basis of ethernet

Abstract

Disclosed in the present application are a method, apparatus and system for implementing remote memory access on the basis of Ethernet, and an electronic device and a medium. The method comprises the following steps: at a transport layer, using a source port number field, a destination port number field and an RDMA base transport header to encapsulate a packet generated by an application layer; enabling a packet header of the packet at a network layer to comprise a source IP address field, a destination IP address field, a type of service field, a total length field and a protocol field, wherein the type of service field, the total length field and the protocol field are used for representing a frame priority, a frame length and an upper-layer protocol, respectively; and at a data link layer, using the source IP address field and the destination IP address field to replace a source MAC address field and a destination MAC address field, respectively, so as to address the packet. The present application can reduce communication overheads and improve the communication performance.

Description

Method and device for realizing remote memory access based on Ethernet

[0001] Cross-reference to Related Applications

[0002] This application claims priority to the Chinese patent application No. 202411389224.6, entitled "Method and device for realizing remote memory access based on Ethernet", filed on September 29, 2024, with the State Intellectual Property Office of China, the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] The present application relates to high-performance computing networks, in particular to a method, device, system, electronic equipment and medium for realizing remote memory access based on Ethernet. BACKGROUND

[0004] With the increasing requirements of high-performance computing networks on communication performance and delay, the traditional communication protocol has gradually exposed its limitations. In order to cope with this challenge, the Remote Direct Memory Access (RDMA) technology is widely adopted, among which the RoCEv2 (RDMA over Converged Ethernet version 2) protocol has become the mainstream choice due to its significant advantages. RoCEv2 has the following characteristics:

[0005] 1. Reducing data transmission delay: allowing data to be transmitted directly from the memory of one computer to the memory of another computer, bypassing the participation of the operating system kernel, thereby significantly reducing the delay of data transmission.

[0006] 2. Improving data processing efficiency: data transmission does not require the involvement of CPU, CPU can be more used for application processing rather than data transfer, which can improve the overall computing efficiency; at the same time, the protocol supports multicast communication, which is very effective for scenarios that need to send the same data to multiple nodes (such as data replication, data distribution, etc.), which can significantly improve the efficiency of data processing.

[0007] 3. Better compatibility: RoCEv2 is designed to consider compatibility with existing Ethernet and IP network devices, making deployment and use more flexible and convenient.

[0008] However, RoCEv2 is to encapsulate InfiniBand (IB) messages into Ethernet packets for transmission and reception, as shown in FIG. 1, the link layer uses the Ethernet data link layer protocol, the transport layer and the network layer use the UDP layer protocol and the IP layer protocol respectively, and the transport layer and the upper layer protocol thereof use the InfiniBand RDMA protocol, so that the RDMA application can realize data communication by using the Ethernet switching device, but compared with IB (i.e. the network card and switch specially designed for RDMA) in the same scenario, RoCEv2 will have a certain loss in performance.

[0009] In addition, in supercomputing applications, especially for short messages with a large proportion (such as less than or equal to 256 bytes), the existing data transmission format of the Ethernet data link layer, the transport layer (UDP layer) and the network layer (IP layer) cannot well meet the communication overhead and communication performance requirements in the data transmission process. SUMMARY

[0010] Based on the above status, the main purpose of the present application is to provide a method, device, system, electronic equipment and medium for realizing remote memory access based on Ethernet, which can effectively reduce the communication overhead of data transmission and help improve the communication performance by optimizing the data transmission frame structure of the data link layer when applied to remote direct memory access.

[0011] To achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0012] The first aspect of the present application provides a method for realizing remote memory access based on Ethernet, which comprises the following steps:

[0013] The message generated by the application layer is encapsulated using the source port number field, the destination port number field and the RDMA basic transport message header (BTH) in the transport layer;

[0014] The message header in the network layer includes the source IP address field, the destination IP address field, the service type (TOS) field, the total length (Total Length) field and the protocol (Protocol) field, wherein the service type field, the total length field and the protocol field are used to represent the priority of the frame, the length of the frame and the upper layer protocol respectively;

[0015] The source IP address field and the destination IP address field are used in the data link layer to replace the source MAC address field and the destination MAC address field for addressing respectively.

[0016] Preferably, the method further comprises the following steps:

[0017] The frame format of the message in the data link layer comprises a preamble field, a start field, a network layer packet field, and a frame check sequence (FCS) field, wherein the frame check sequence field is used to check the data transmission of the message in the Ethernet.

[0018] Preferably, the network layer packet field comprises a plurality of fields, including the source IP address field, the destination IP address field, the service type field, the total length field, the protocol field, and the transport layer datagram field.

[0019] Preferably, the upper layer protocol is a protocol above the UDP layer, representing a datagram format for remote direct memory access.

[0020] Preferably, the method further comprises the following steps:

[0021] The RDMA basic transport message header comprises an operation type (OpCode) field, a message sequence number (PSN) field, an ACK response requirement flag bit (ACK) field, and a reserved field.

[0022] The destination queue number (Destination QP) of the message is obtained from the destination port number field.

[0023] Preferably, the method further comprises the following steps:

[0024] The operation type field is used to identify the operation type of the remote direct memory access of the message, and the operation type comprises a direct operation between the memory of the local host and the remote host, and a direct operation between the processor and the memory of the local host and the remote host.

[0025] The second aspect of the present application provides an apparatus for implementing remote memory access in an Ethernet, the apparatus comprising:

[0026] A first unit is configured to encapsulate, in the transport layer, a message generated by an application layer using a source port number field, a destination port number field, and an RDMA basic transport message header (BTH);

[0027] A second unit is configured to make the message header in the network layer comprise a source IP address field, a destination IP address field, a service type (TOS) field, a total length (Total Length) field, and a protocol (Protocol) field, wherein the service type field, the total length field, and the protocol field are used to represent the priority of the frame, the length of the frame, and the upper layer protocol, respectively.

[0028] A third unit is configured to use the source IP address field and the destination IP address field of the message in the data link layer to replace the source MAC address field and the destination MAC address field for addressing, respectively.

[0029] Preferably, the third unit is further configured to make the frame format of the packet at the data link layer include a preamble field, a start-of-frame field, a network layer packet field, and a frame check sequence (FCS) field, wherein the data transmission of the packet in Ethernet is checked by the frame check sequence field.

[0030] Preferably, the network layer packet field includes a plurality of fields, including the source IP address field, the destination IP address field, the service type field, the total length field, the protocol field, and the transport layer datagram field.

[0031] Preferably, the upper layer protocol is a protocol above the UDP layer, to represent a data packet format for remote direct memory access.

[0032] Preferably, the RDMA basic transport packet header includes an operation type (OpCode) field, a packet sequence number (PSN) field, an acknowledgement response requirement flag bit (ACK) field, and a reserved field.

[0033] The destination queue number (Destination QP) of the packet is obtained from the destination port number field.

[0034] Preferably, the operation type field is used to identify the operation type of the remote direct memory access of the packet, including direct operations between the memories of the local host and the remote host, and direct operations between the processors and memories of the local host and the remote host.

[0035] The third aspect of the present application provides a method for implementing remote memory access based on Ethernet, the method comprising the following steps:

[0036] The packet generated by the application layer is encapsulated at the transport layer using a source port number field, a destination port number field, and an RDMA basic transport packet header (BTH), wherein the RDMA basic transport packet header only includes an operation type (OpCode) field, a packet sequence number (PSN) field, an acknowledgement response requirement flag bit (ACK) field, and a reserved field.

[0037] The destination queue number (Destination QP) of the packet is obtained from the destination port number field.

[0038] Preferably, the operation type field is used to identify the operation type of the remote direct memory access of the packet, including direct operations between the memories of the local host and the remote host, and direct operations between the processors and memories of the local host and the remote host.

[0039] The fourth aspect of the present application provides a device for implementing remote memory access based on Ethernet, the device comprising:

[0040] The device comprises:

[0041] The fourth unit is configured to encapsulate the message generated by the application layer using a source port number field, a destination port number field, and an RDMA basic transport message header (BTH) in a transport layer, wherein the RDMA basic transport message header only includes an operation type (OpCode) field, a message sequence number (PSN) field, an acknowledgement response requirement flag bit (ACK) field, and a reserved field.

[0042] The destination queue number (Destination QP) of the message is obtained from the destination port number field.

[0043] Preferably, the operation type field is used to identify the operation type of the remote direct memory access message, and the operation type includes direct operation between the memory of the local host and the remote host, and direct operation between the processor and the memory of the local host and the remote host.

[0044] The fifth aspect of the present application provides a system for implementing remote memory access based on Ethernet, the system comprising:

[0045] A communication interface; and

[0046] A processing circuit coupled to the communication interface and configured to:

[0047] Identify a plurality of fields of a frame format of a message for remote direct memory access, the plurality of fields being in turn a preamble field, a start-of-frame field, a network layer packet field, and a frame check sequence (FCS) field.

[0048] Preferably, the network layer packet field includes a plurality of fields in turn a source IP address field, a destination IP address field, a type of service (TOS) field, a total length (Total Length) field, a protocol (Protocol) field, and a transport layer datagram field.

[0049] Preferably, the transport layer datagram field includes a source port number field, a destination port number field, and an RDMA basic transport message header (BTH), and the format of the RDMA basic transport message header is in turn an operation type (OpCode) field, a message sequence number (PSN) field, an acknowledgement response requirement flag bit (ACK) field, and a reserved field.

[0050] Preferably, the operation type field is used to identify an operation type of the remote direct memory access of the packet, and the operation type comprises a direct operation between memories of a local host and a remote host, and a direct operation between processors and memories of the local host and the remote host.

[0051] The sixth aspect of the present application provides a system for implementing remote memory access based on Ethernet, the system comprising:

[0052] a communication interface; and

[0053] processing circuitry, coupled to the communication interface and configured to:

[0054] identify a plurality of fields of a frame format of a packet for remote direct memory access,

[0055] wherein the packet generated by an application layer is encapsulated using a source port number field, a destination port number field, an RDMA basic transport header (BTH) at a transport layer, and a format of the RDMA basic transport header is in turn an operation type (OpCode) field, a packet sequence number (PSN) field, an acknowledgement response requirement flag bit (ACK) field, and a reserved field.

[0056] a destination queue number (Destination QP) of the packet is obtained from the destination port number field.

[0057] Preferably, the operation type field is used to identify an operation type of the remote direct memory access of the packet, and the operation type comprises a direct operation between memories of a local host and a remote host, and a direct operation between processors and memories of the local host and the remote host.

[0058] The seventh aspect of the present application provides an electronic device, comprising: a processor; and a memory having a computer program stored thereon, wherein the computer program, when executed by the processor, can implement the method of the first aspect.

[0059] The eighth aspect of the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is used to run to implement the method of the first aspect.

[0060] The present application has obvious advantages and beneficial effects compared with the prior art, and at least has the following advantages:

[0061] The method for realizing remote memory access based on Ethernet of the application, aiming at the use scene of RDMA, improves the frame format used for data transmission of the existing Ethernet and the data message header of the network layer and the transport layer, uses the source IP address field and the destination IP address field to replace the source MAC address field and the destination MAC address field respectively for addressing at the data link layer, uses the service type field of the IP layer header to represent the priority of the frame, uses the total length field of the IP layer header to represent the length of the frame, uses the protocol field of the IP layer header to represent the upper layer protocol, and the IP layer header only retains the IP address field, the destination IP address field, the service type (TOS) field, the total length field and the protocol field, and the UDP layer header only retains the source port number field and the destination port number field, so that in the process of remote direct memory access, the data transmission adopts the simplified frame format, which can effectively reduce the communication overhead of data transmission and help improve the communication performance.

[0062] The device for realizing remote memory access based on Ethernet of the application encapsulates the message by using the source port number field and the destination port number field of the UDP layer header and the RDMA basic transmission message header at the transport layer, and the network layer message header is composed of the source IP address field, the destination IP address field, the service type (TOS) field, the total length field and the protocol field of the IP layer header, wherein the addressing of the message at the data link layer is performed by replacing the source MAC address field and the destination MAC address field with the source IP address field and the destination IP address field respectively, so that in the process of remote direct memory access, the data transmission can adopt the simplified frame format, which can effectively reduce the communication overhead of data transmission and help improve the communication performance.

[0063] The method and device for realizing remote memory access based on Ethernet of the application improve the RDMA basic transmission message header of the transport layer used for data transmission of the existing Ethernet for the use scene of RDMA, and the RDMA basic transmission message header only includes the operation type field, the message sequence number field, the response requirement mark bit field and the reserved field, and the destination queue number of the message is identified by the destination port number field of the UDP message header, wherein the operation type field is used to represent the operation type of the message, the message sequence number field is used to ensure the orderliness of message transmission, and the response requirement mark bit field is used to ensure no packet loss to ensure the reliability of data communication, so that by using the simplified RDMA basic transmission message header, the communication overhead is reduced and the communication performance is improved.

[0064] The system for realizing remote memory access based on Ethernet of the present application can identify the frame format of the message for remote direct memory access, which comprises a plurality of fields in sequence after simplified processing, i.e., a preamble field, a start field, a network layer packet field, and a frame check sequence field. The addressing of the message at the data link layer is performed through the source IP address field and the destination IP address field included in the network layer packet field, and the service type field, the total length field, and the protocol field contained in the network layer message header are used to respectively represent the priority of the frame, the length of the frame, and the upper layer protocol, and the frame check sequence field is used to uniformly check the data transmission of the message on the Ethernet. Thus, the simplified frame format is used to realize the remote direct memory access, which can effectively reduce the communication overhead and improve the communication performance.

[0065] The system for realizing remote memory access based on Ethernet of the present application can identify the source port number field and the destination port number field of the UDP message header used by the message for remote direct memory access at the transport layer, and the RDMA basic transport message header, the format of which comprises an operation type field, a message sequence number field, a response requirement flag bit field, and a reserved field in sequence, and the destination end queue number of the message is obtained from the destination port number field of the UDP message header. The simplified RDMA basic transport message header is used to realize the remote direct memory access, which can effectively reduce the communication overhead and improve the communication performance.

[0066] The electronic device and the computer readable storage medium of the present application can effectively reduce the communication overhead of data transmission while ensuring the correctness and reliability of data communication, and help to improve the communication performance by using the above method. BRIEF DESCRIPTION OF DRAWINGS

[0067] Fig. 1 is a schematic diagram of the protocol layer of RoCEv2;

[0068] Fig. 2 is a schematic diagram of the flow of one embodiment of the method for realizing remote memory access based on Ethernet of the present application;

[0069] Fig. 3 is a schematic diagram of the format of the existing conventional RDMA basic transport message header;

[0070] Fig. 4 is a schematic diagram of the format of the existing UDP layer message header;

[0071] Fig. 5 is a schematic diagram of the format of the existing IP layer message header;

[0072] Fig. 6 is a schematic diagram of the frame format of the existing data link layer;

[0073] Fig. 7 is a schematic diagram of the modules of one embodiment of the device of the present application;

[0074] Figure 8 is a frame format diagram of one embodiment of the system for implementing remote memory access based on Ethernet according to the present application;

[0075] Figure 9 is a basic transport header format diagram of one embodiment of the system for implementing remote memory access based on Ethernet according to the present application. DETAILED DESCRIPTION

[0076] To further clarify the technical means and effects taken by the present application to achieve the intended purpose, the specific embodiments, methods, steps, features and effects of the method, device, system, electronic equipment and computer readable storage medium according to the present application are described in detail below in conjunction with the drawings and preferred embodiments.

[0077] Referring to Figure 2, a method for implementing remote memory access based on Ethernet, the method comprising the following steps:

[0078] S100, the packet generated by the application layer is encapsulated using the source port number field, the destination port number field, and the RDMA basic transport header (BTH) at the transport layer;

[0079] S200, the packet header at the network layer includes a source IP address field, a destination IP address field, a service type (TOS) field, a total length (Total Length) field, and a protocol (Protocol) field, wherein the service type field, the total length field, and the protocol field are used to represent the priority of the frame, the length of the frame, and the upper layer protocol, respectively;

[0080] S300, the packet at the data link layer uses the source IP address field and the destination IP address field to replace the source MAC address field and the destination MAC address field for addressing, respectively.

[0081] Specifically, regarding step S100, in the RoCEv2 protocol stack, the RDMA basic transport header (Base Transport Header, BTH) defines multiple fields of the packet header of the IB transport layer, which contains the key information of the RoVEv2 packet, referring to Figure 3, for example, the OpCode field, the Destination QP field, and the PSN field.

[0082] In the transmission layer and network layer of the existing Ethernet, referring to FIG. 4, the UDP message header is generally 8 bytes, referring to FIG. 5, the IP message header (such as IPv4) is generally 20-60 bytes. In steps S100 and S200 of the embodiment, the message for RDMA uses only part of the fields of the UDP message header and the IP message header in the message header of the transmission layer and the network layer respectively, and is used in the frame format of the data link layer, so that the data transmission format is simplified to reduce the communication overhead.

[0083] Regarding step S300, referring to FIG. 6, in the data link layer of the existing Ethernet, the frame format of the data link layer generally needs to include a source MAC address field and a destination MAC address field. The frame format for RDMA is simplified in the embodiment by deleting the source MAC address field and the destination MAC address field, and using the source IP address field and the destination IP address field in the network layer message header to achieve addressing. In addition, considering that the use scenario is RDMA data communication, the Type / Length field used to indicate the upper layer protocol above the data link layer is deleted, and the message length is controlled by the total length field of the network layer.

[0084] Through the above steps, for the RDMA use scenario, the frame format for data transmission of the existing Ethernet and the data message header of the transmission layer and the network layer are improved. In the data link layer, the source IP address field and the destination IP address field are used to replace the source MAC address field and the destination MAC address field for addressing. The service type field of the IP layer message header is used to represent the priority of the frame, the total length field of the IP layer message header is used to represent the length of the frame, and the protocol field of the IP layer message header is used to represent the upper layer protocol. In addition, the IP layer message header only retains the IP address field, the destination IP address field, the service type field, the total length field, and the protocol field. The UDP layer message header only retains the source port number field and the destination port number field. Therefore, in the remote direct memory access process, the data transmission uses the simplified frame format, which can effectively reduce the communication overhead of data transmission and help improve the communication performance.

[0085] As an optional embodiment, the method further includes the following steps:

[0086] S400, the frame format of the message in the data link layer includes a preamble field, a start field, a network layer data packet field, and a frame check sequence (FCS) field. The message is uniformly checked by the frame check sequence field for data transmission in the Ethernet.

[0087] Specifically, as to step S400, the Preamble field, the Start of Frame Delimiter (SFD) field, and the FCS field are reserved in the frame format of the data link layer, and other protocol layer fields for checking or checksum are not reserved, and the integrity of the frame is checked by the FCS field of the data link layer.

[0088] Through the above steps, the frame format of the data link layer is further optimized, which not only ensures the correctness of data communication, but also meets the requirement of reducing communication overhead, thereby helping to improve the data transmission communication performance.

[0089] As a further improvement of the above embodiment, the network layer packet field includes a plurality of fields, including the source IP address field, the destination IP address field, the TOS field, the total length field, the protocol field, and the transport layer datagram field.

[0090] Referring to the above description of step S200, the existing RoCEv2 protocol message header at the network layer is optimized, the source IP address field, the destination IP address field, the TOS field, the total length field, and the protocol field are reserved in the IP message header, and the source port number field and the destination port number field are reserved in the UDP message header, wherein the source IP address field and the destination IP address field are used for addressing, the TOS field indicates the priority of the frame, the total length field indicates the length of the frame, and the protocol field indicates the upper layer protocol, which generally refers to the protocol above UDP.

[0091] As to the transport layer datagram field, it can include the source port number field, the destination port number field, and the RDMA datagram field. The destination port number field is used to indicate the destination QP (Destination QP) of the transport layer BTH, that is, the Queue Pair sequence number.

[0092] Through the above steps, the message header at the network layer for RDMA is simplified, which not only significantly reduces the communication overhead under the premise of meeting the RDMA protocol data transmission, but also significantly improves the communication performance.

[0093] As an optional embodiment, the upper layer protocol is a protocol above the UDP layer, to represent the datagram format for remote direct memory access.

[0094] In particular, the upper protocol herein can generally adopt a regular RDMA packet format or a simplified RDMA packet format, that is, the RDMA basic transport packet header used by the transport layer can be a regular BTH format or a simplified BTH format. The simplified RDMA packet header format can retain some fields of the regular BTH format to reduce data communication overhead.

[0095] For example, the protocol field is defined as 1 byte, which can be represented as follows:

[0096] 0x00: regular RDMA

[0097] 0x01: simplified RDMA

[0098] 0x1x: Reserve

[0099] As an optional embodiment, the RDMA basic transport packet header includes an operation type (OpCode) field, a packet sequence number (PSN) field, an acknowledgement response requirement flag bit (ACK) field, and a reserved field.

[0100] The method further includes the following steps:

[0101] The destination queue number (Destination QP) of the packet is obtained from the destination port number field.

[0102] In particular, as described above, the BTH of the transport layer for RDMA can only include the OpCode field, the PSN field, the ACK field, and the reserved field. The OpCode field is used to indicate the operation type of the packet, the PSN field is used to ensure the order of packet transmission, and the ACK field is used to ensure no packet loss to ensure the reliability of data communication. In addition, the Destination QP required by the transport layer can be represented using the destination port number field of the UDP layer packet header.

[0103] Thus, by using the simplified RDMA basic transport packet header, communication overhead is reduced and communication performance is improved.

[0104] As an optional embodiment, the method further includes the following steps:

[0105] The operation type field is used to identify the operation type of the remote direct memory access of the packet, and the operation type includes direct operations between the memories of the local host and the remote host, direct operations between the processors and memories of the local host and the remote host.

[0106] For example, the operation type can include Send, Read, Write, Atomic, Load, and Store.

[0107] Specifically, Send is generally used to send data to the receiving queue of the remote host QP, and the corresponding operation of Send is Receive; Read is generally used to read the memory of the remote host, and the caller specifies the remote virtual address, which is used to copy like a local memory address; Write is generally used to write data into the memory of the remote host; Atomic generally refers to a special operation of RDMA, which guarantees the atomicity when performing certain specific memory operations, usually including atomic get, atomic add, atomic compare and atomic exchange, which belongs to the extension of RDMA atomic operation.

[0108] On the basis of the above operation types, Load and Store operations are further added, which belong to the operations of a processor element (PE) on the memory. Among them, Load is to read the memory of the remote host and directly transmit the data to the local processor, and correspondingly, Read is to read the memory of the remote host and store the data into the local memory; Store is to directly write data from the local processor to the memory of the remote host, and correspondingly, Write is to write data from the local memory to the memory of the remote host.

[0109] Through the above steps, on the basis of the conventional RDMA operation type (direct data operation between host memories), two data transmission paths between the local host and the remote host are added, i.e. the reading and writing of data between the processors and memories of the local host and the remote host. For some control type packets that need to be processed in time, data transmission from the remote memory to the local processor or from the local processor to the remote memory can be realized, without the data transmission path from the remote memory to the local memory and then to the local processor, which can effectively reduce the delay of the existing data transmission path and improve the data communication performance.

[0110] Referring to FIG. 7, an apparatus for implementing remote memory access based on Ethernet, the apparatus comprises:

[0111] A first unit is configured to encapsulate the packet generated by the application layer using the source port number field, the destination port number field, and the RDMA basic transmission packet header (BTH) at the transport layer;

[0112] A second unit is configured to make the packet header at the network layer include a source IP address field, a destination IP address field, a service type (TOS) field, a total length (Total Length) field, and a protocol (Protocol) field, wherein the service type field, the total length field, and the protocol field are used to represent the priority of the frame, the length of the frame, and the upper layer protocol, respectively.

[0113] The third unit is configured to use a source IP address field and a destination IP address field to replace a source MAC address field and a destination MAC address field respectively for addressing of the packet at a data link layer.

[0114] Thus, for the RDMA use scenario, the frame format used by the existing Ethernet for data transmission and the data packet header of the transport layer and network layer are improved. The source IP address field and the destination IP address field are used to replace the source MAC address field and the destination MAC address field respectively for addressing of the packet at the data link layer. The service type field of the IP layer packet header is used to represent the priority of the frame. The total length field of the IP layer packet header is used to represent the length of the frame. The protocol field of the IP layer packet header is used to represent the upper layer protocol. The IP layer packet header only retains the IP address field, the destination IP address field, the service type field, the total length field and the protocol field. The UDP layer packet header only retains the source port number field and the destination port number field. Thus, the data transmission in the remote direct memory access process adopts the simplified frame format, which can effectively reduce the communication overhead of data transmission and help improve the communication performance.

[0115] As an optional embodiment, the third unit is further configured to make the frame format of the packet at the data link layer include a preamble field, a start-of-frame field, a network layer data packet field and a frame check sequence (FCS) field. The frame check sequence field is used to uniformly check the data transmission of the packet in the Ethernet.

[0116] Thus, the frame format of the data link layer is further optimized. The FCS field is retained in the frame format of the data link layer, and other protocol layer fields used for checking or checksum are not retained. The integrity of the frame is checked by the FCS field of the data link layer. The data communication correctness is ensured, and the requirement of reducing the communication overhead is also met, thereby helping to improve the data transmission communication performance.

[0117] As a further improvement of the above-mentioned embodiments, the network layer data packet field includes a plurality of fields, including the source IP address field, the destination IP address field, the service type field, the total length field, the protocol field and the transport layer data report field.

[0118] As to the transport layer data report field, it can include the source port number field, the destination port number field and the RDMA data report field. The destination port number field is used to represent the destination end queue pair number (Destination QP) of the transport layer BTH, that is, the destination end Queue Pair number.

[0119] Thus, the packet for RDMA is simplified in the network layer packet header. On the premise of meeting the RDMA protocol data transmission, the communication overhead can be significantly reduced, and the communication performance is obviously improved.

[0120] As an optional embodiment, the upper layer protocol is a protocol above the UDP layer, to represent a datagram format for remote direct memory access.

[0121] Wherein, the upper layer protocol can generally adopt a regular RDMA packet format or a simplified RDMA packet format.

[0122] As an optional embodiment, the RDMA basic transport packet header includes an operation type (OpCode) field, a packet sequence number (PSN) field, an ACK response requirement flag bit (ACK) field and a reserved field.

[0123] The destination queue number (Destination QP) of the packet is obtained from the destination port number field.

[0124] Thus, by using the simplified RDMA basic transport packet header, it is beneficial to reduce communication overhead and improve communication performance.

[0125] As an optional embodiment, the operation type field is used to identify the operation type of remote direct memory access of the packet, and the operation type includes direct operation between the memory of the local host and the remote host, direct operation between the processor and the memory of the local host and the remote host.

[0126] Specifically, as described above, the operation type can include Send, Read, Write, Atomic, Load and Store.

[0127] Thus, on the basis of the regular RDMA operation type (direct data operation between host memories), two data transmission paths between the local host and the remote host, i.e. reading and writing data between the processor and the memory of the local host and the remote host, can be added by Load and Store, so that data transmission from the remote memory to the local processor or from the local processor to the remote memory can be realized for some control type packets that need to be processed in time, without the data transmission path from the remote memory to the local memory and then to the local processor, which can effectively reduce the latency of the existing data transmission path and improve the data communication performance.

[0128] A method for implementing remote memory access based on Ethernet, the method comprising the following steps:

[0129] The packet generated by the application layer is encapsulated in the transport layer using the source port number field, the destination port number field and the RDMA basic transport packet header (BTH), wherein the RDMA basic transport packet header only includes an operation type (OpCode) field, a packet sequence number (PSN) field, an ACK response requirement flag bit (ACK) field and a reserved field.

[0130] The destination queue number (Destination QP) of the message is obtained from the destination port number field.

[0131] Specifically, the existing conventional BTH format is simplified, and the OpCode field, the PSN field, and the ACK field are reserved, wherein the OpCode field is used to indicate the operation type of the message, the PSN field is used to ensure the order of message transmission, and the ACK field is used to ensure no packet loss and ensure the reliability of data communication. The Destination QP required by the transmission layer can be represented by the destination port number field of the UDP layer message header.

[0132] It should be noted that the simplified BTH can be applied to the existing data link layer frame format, or the data link layer frame format simplified above.

[0133] By the above steps, using the simplified RDMA basic transport message header is conducive to reducing the communication overhead of remote direct memory access and improving communication performance.

[0134] As an optional embodiment, the operation type field is used to identify the operation type of the remote direct memory access of the message, and the operation type includes direct operation between the memory of the local host and the remote host, and direct operation between the processor and the memory of the local host and the remote host.

[0135] Through the above steps, data transmission from remote memory to local processor or from local processor to remote memory can be realized for some control type messages that need to be processed in time, without the data transmission path from remote memory to local memory to local processor, which can effectively reduce the existing data transmission path delay and improve the data communication performance.

[0136] An apparatus for implementing remote memory access based on Ethernet, the apparatus comprising:

[0137] A fourth unit is configured to encapsulate, at the transmission layer, a message generated by an application layer using a source port number field, a destination port number field, and an RDMA basic transport message header (BTH), wherein the RDMA basic transport message header only includes an operation type (OpCode) field, a message sequence number (PSN) field, an acknowledgement response requirement flag bit (ACK) field, and a reserved field.

[0138] The destination queue number (Destination QP) of the message is obtained from the destination port number field.

[0139] Thus, by using the simplified RDMA basic transport message header, it is conducive to reducing the communication overhead of remote direct memory access and improving communication performance.

[0140] As an optional embodiment, the operation type field is used to identify the operation type of the remote direct memory access of the packet, and the operation type includes a direct operation between the memory of the local host and the remote host, and a direct operation between the processor and the memory of the local host and the remote host.

[0141] Therefore, for some control type packets that need to be processed in time, data transmission from the remote memory to the local processor or from the local processor to the remote memory can be implemented without the data transmission path from the remote memory to the local memory and then to the local processor, which can effectively reduce the existing data transmission path delay and improve the data communication performance.

[0142] A system for implementing remote memory access based on Ethernet, the system comprising:

[0143] a communication interface; and

[0144] processing circuitry coupled to the communication interface and configured to:

[0145] identify a plurality of fields of a frame format of a packet for remote direct memory access, the plurality of fields being in order a preamble field, a start field, a network layer packet field, a frame check sequence (FCS) field.

[0146] Specifically, referring to FIG. 8, among the plurality of fields of the frame format for the data link layer identified by the system, only the preamble (Preamble) field, the start field (SFD) and the frame check sequence (FCS) field in the existing data link layer frame format are retained, and the source IP address field and the destination IP address field in the network layer packet header are used for addressing. Since the use scenario is fixed as RDMA data communication, the Type field is not needed to indicate the upper layer service of the data link layer, and the overall length of the packet can be controlled by the Total Length field of the IP layer packet header. The retained FCS field uniformly ensures the correctness of data transmission, and the checksum and verification and fields of other protocol layers can be deleted.

[0147] The network layer packet field includes a network layer packet header and data (Data), wherein the network layer packet header includes a source IP address field, a destination IP address field, a TOS field, a total length (Total Length) field, and a protocol (Protocol) field of an IP layer packet header.

[0148] Therefore, by simplifying the frame format for RDMA, the communication overhead of data transmission can be significantly reduced, and the communication performance can be improved.

[0149] As an optional embodiment, the network layer packet field comprises a plurality of fields, in sequence, a source IP address field, a destination IP address field, a service type (TOS) field, a total length (Total Length) field, a protocol (Protocol) field, and a transport layer datagram field.

[0150] For example, referring to FIG. 8, in this embodiment, the frame format for RDMA and the size of each field are as follows: an Ethernet frame gap (4-12 bytes), a preamble field (7 bytes), an SFD field (1 byte), a source IP address field (4 bytes), a destination IP address field (4 bytes), a TOS field (1 byte), a total length field (2 bytes), a protocol field (1 byte), an SPN field (2 bytes), a DPN field (2 bytes), data (8-16K bytes), and a FCS field (4 bytes).

[0151] Thus, the packet for RDMA uses only part of the UDP packet header and IP packet header at the transport layer and network layer, respectively, and is used in the frame format for the data link layer, so that the data transmission format is further simplified to reduce the communication overhead.

[0152] As an optional embodiment, the transport layer datagram field comprises a source port number field, a destination port number field, an RDMA basic transport packet header (BTH), and the format of the RDMA basic transport packet header is in sequence, an operation type (OpCode) field, a packet sequence number (PSN) field, an acknowledgement response requirement flag bit (ACK) field, and a reserved field.

[0153] Specifically, referring to FIG. 9, the BTH for RDMA at the transport layer comprises only an OpCode field, a PSN field, an ACK field, and a reserved (Reserved) field, wherein the OpCode field is used to indicate the operation type of the packet, the PSN field is used to ensure the orderliness of packet transmission, and the ACK field is used to ensure no packet loss to ensure the reliability of data communication. In addition, the Destination QP required by the transport layer is indicated by the destination port number field of the UDP layer packet header.

[0154] For example, in this embodiment, the simplified RDMA basic transport packet header format and the size of each field are as follows: an OpCode field (1 byte), a PSN field (3 bytes), an ACK field (1 bit), and a reserved field (31 bits).

[0155] Thus, in supercomputing applications, especially for short packets with a large proportion, the use of the simplified RDMA basic transport packet header can significantly reduce the communication overhead and improve the data transmission communication performance.

[0156] As an optional embodiment, the operation type field is used to identify an operation type of the remote direct memory access of the packet, and the operation type includes a direct operation between memories of a local host and a remote host, and a direct operation between processors and memories of the local host and the remote host.

[0157] Therefore, for some control packets requiring timely processing, data transmission from the remote memory to the local processor or from the local processor to the remote memory can be implemented without a data transmission path from the remote memory to the local memory and then to the local processor, which can effectively reduce the existing data transmission path delay and improve the data communication performance.

[0158] A system for implementing remote memory access based on Ethernet, the system comprising:

[0159] a communication interface; and

[0160] a processing circuit coupled to the communication interface and configured to:

[0161] identify a plurality of fields of a frame format of a packet for remote direct memory access,

[0162] wherein the packet generated by an application layer is encapsulated using a source port number field, a destination port number field, and an RDMA basic transport packet header (BTH) at a transport layer, and a format of the RDMA basic transport packet header is in turn an operation type (OpCode) field, a packet sequence number (PSN) field, an acknowledgement response requirement flag bit (ACK) field, and a reserved field.

[0163] a destination queue number (Destination QP) of the packet is obtained from the destination port number field.

[0164] Specifically, the BTH of the transport layer only includes the OpCode field, the PSN field, the ACK field, and the reserved field, wherein the OpCode field is used to indicate the operation type of the packet, the PSN field is used to ensure the orderliness of the packet transmission, and the ACK field is used to ensure no packet loss to ensure the reliability of the data communication.

[0165] In addition, the Destination QP required by the transport layer is indicated by using a destination port number field of a UDP layer packet header. It should be noted that if the reserved field is not used, the Destination QP can also use the reserved field to identify the destination QP sequence number.

[0166] Therefore, by using the simplified RDMA basic transport packet header, the communication overhead can be reduced and the communication performance can be improved.

[0167] As an optional embodiment, the operation type field is used to identify the operation type of the remote direct memory access of the packet, and the operation type includes a direct operation between the memory of the local host and the memory of the remote host, and a direct operation between the processor and the memory of the local host and the remote host.

[0168] Therefore, for some control type packets that need to be processed in time, data transmission from the remote memory to the local processor or from the local processor to the remote memory can be implemented, without the data transmission path from the remote memory to the local memory and then to the local processor, so that the existing data transmission path delay can be effectively reduced, and the data communication performance can be improved.

[0169] The application further provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the computer program is executed by the processor to implement the method in the above embodiments.

[0170] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is used to run to implement the method in the above embodiments.

[0171] The above is only a preferred embodiment of the application, and does not limit the application in any form. Although the application has been disclosed as the above preferred embodiment, it is not intended to limit the application. Any person skilled in the art can make some changes or modifications to the above disclosed technical content without departing from the scope of the application, and the equivalent embodiments with equivalent changes are equivalent to the above embodiments. Any simple modification, equivalent change and modification of the above embodiments according to the technical essence of the application are still within the scope of the application.

Claims

1. A method for implementing remote memory access over Ethernet, characterized in that, The method comprises the following steps: The message generated by the application layer is encapsulated by using a source port number field, a destination port number field and an RDMA basic transmission message header (BTH) in the transmission layer; The message header of the message in the network layer comprises a source IP address field, a destination IP address field, a service type (TOS) field, a total length (Total Length) field and a protocol (Protocol) field, wherein the service type field, the total length field and the protocol field are used to respectively represent the priority of the frame, the length of the frame and the upper layer protocol; The message in the data link layer uses the source IP address field and the destination IP address field to replace the source MAC address field and the destination MAC address field for addressing.

2. The method of claim 1, wherein, The method further comprises the following steps: The frame format of the message in the data link layer comprises a preamble field, a start field, a network layer data packet field and a frame check sequence (FCS) field, wherein the frame check sequence field is used to uniformly check the data transmission of the message in the Ethernet.

3. The method of claim 2, wherein, The network layer data packet field comprises a plurality of fields, which comprise the source IP address field, the destination IP address field, the service type field, the total length field, the protocol field and a transmission layer data report field.

4. The method of claim 1, wherein, The upper layer protocol is a protocol above the UDP layer, which represents the data message format for remote direct memory access.

5. The method of claim 1, wherein The RDMA basic transmission message header comprises an operation type (OpCode) field, a message sequence number (PSN) field, an ACK response requirement flag bit (ACK) field and a reserved field; The method further comprises the following steps: The destination queue number (Destination QP) of the message is obtained from the destination port number field.

6. The method of claim 5, wherein, The method further comprises the following steps: The operation type field is used to identify the operation type of the remote direct memory access of the message, and the operation type comprises a direct operation between the memories of the local host and the remote host, and a direct operation between the processors and the memories of the local host and the remote host.

7. An apparatus for implementing remote memory access over Ethernet, the apparatus comprising: The device comprises: A first unit for encapsulating a message generated by an application layer by using a source port number field, a destination port number field and an RDMA basic transmission message header (BTH) in the transmission layer; A second unit for making the message header of the message in the network layer comprise a source IP address field, a destination IP address field, a service type (TOS) field, a total length (Total Length) field and a protocol (Protocol) field, wherein the service type field, the total length field and the protocol field are used to respectively represent the priority of the frame, the length of the frame and the upper layer protocol; A third unit for using the source IP address field and the destination IP address field to replace the source MAC address field and the destination MAC address field for addressing the message in the data link layer.

8. The device of claim 7, wherein The third unit is further configured to cause the frame format of the message at the data link layer to include a preamble field, a start-of-frame field, a network layer packet field, and a frame check sequence (FCS) field, wherein the frame check sequence field is configured to uniformly check data transmission of the message on an Ethernet network.

9. The apparatus of claim 8, wherein, In some embodiments, The network layer packet field includes a plurality of fields, including the source IP address field, the destination IP address field, the service type field, the total length field, the protocol field, and the transport layer datagram field.

10. The apparatus of claim 7, wherein, The upper layer protocol is a protocol above the UDP layer, to represent a datagram format for remote direct memory access.

11. The apparatus of claim 7, wherein, In some embodiments, The RDMA basic transport header includes an operation type (OpCode) field, a message sequence number (PSN) field, an acknowledgement response requirement flag bit (ACK) field, and a reserved field. The destination queue number (Destination QP) of the message is obtained from the destination port number field.

12. The apparatus of claim 11, wherein, The operation type field is configured to identify an operation type of remote direct memory access of the message, including direct operations between memories of a local host and a remote host, and direct operations between processors and memories of the local host and the remote host.

13. A method for implementing remote memory access over Ethernet, the method comprising: The method includes the following steps: The message generated by the application layer is encapsulated at the transport layer using a source port number field, a destination port number field, and an RDMA basic transport header (BTH), wherein the RDMA basic transport header only includes an operation type (OpCode) field, a message sequence number (PSN) field, an acknowledgement response requirement flag bit (ACK) field, and a reserved field. The destination queue number (Destination QP) of the message is obtained from the destination port number field.

14. The method of claim 13, wherein, The operation type field is configured to identify an operation type of remote direct memory access of the message, including direct operations between memories of a local host and a remote host, and direct operations between processors and memories of the local host and the remote host.

15. An apparatus for implementing remote memory access over Ethernet, the apparatus comprising: The apparatus includes: A fourth unit configured to encapsulate, at the transport layer, a message generated by an application layer using a source port number field, a destination port number field, and an RDMA basic transport header (BTH), wherein the RDMA basic transport header only includes an operation type (OpCode) field, a message sequence number (PSN) field, an acknowledgement response requirement flag bit (ACK) field, and a reserved field. The destination queue number (Destination QP) of the message is obtained from the destination port number field.

16. The apparatus of claim 15, wherein, The operation type field is configured to identify an operation type of remote direct memory access of the message, including direct operations between memories of a local host and a remote host, and direct operations between processors and memories of the local host and the remote host.

17. A system for implementing remote memory access over Ethernet, the system comprising: The system includes: a communication interface; and processing circuitry coupled to the communication interface and configured to: A plurality of fields of a frame format of a packet for remote direct memory access are identified, in order: a preamble field, a start-of-frame field, a network layer packet field, and a frame check sequence (FCS) field.

18. The system of claim 17, wherein, The network layer packet field includes a plurality of fields, in order: a source IP address field, a destination IP address field, a type of service (TOS) field, a total length (Total Length) field, a protocol (Protocol) field, and a transport layer datagram field.

19. The system of claim 18, wherein, The transport layer datagram field includes a source port number field, a destination port number field, an RDMA base transport header (BTH), the RDMA base transport header having a format in order: an operation code (OpCode) field, a packet sequence number (PSN) field, an acknowledgement response required flag (ACK) field, and a reserved field.

20. The system of claim 19, wherein, The operation code field is used to identify an operation type of remote direct memory access of the packet, the operation type including a direct operation between memories of a local host and a remote host, and a direct operation between processors and memories of the local host and the remote host.

21. A system for implementing remote memory access over Ethernet, the system comprising: The system includes: a communication interface; and processing circuitry coupled to the communication interface and configured to: identify a plurality of fields of a frame format of a packet for remote direct memory access, wherein a packet generated by an application layer is encapsulated at a transport layer using a source port number field, a destination port number field, an RDMA base transport header (BTH), the RDMA base transport header having a format in order: an operation code (OpCode) field, a packet sequence number (PSN) field, an acknowledgement response required flag (ACK) field, and a reserved field; a destination queue pair (Destination QP) of the packet is obtained from the destination port number field.

22. The system of claim 21, wherein, The operation code field is used to identify an operation type of remote direct memory access of the packet, the operation type including a direct operation between memories of a local host and a remote host, and a direct operation between processors and memories of the local host and the remote host.

23. An electronic device, comprising: comprises: a processor; and a memory having stored thereon a computer program that, when executed by the processor, is capable of implementing the method of any one of claims 1 to 6. The computer program is for execution to implement the method of any one of claims 1 to 6.

24. A computer readable storage medium having stored thereon a computer program, characterized in that, ​

Images