RDMA link establishment method based on QUIC protocol

Abstract

A remote direct memory access (RDMA) link establishment method based on a QUIC protocol comprises the following operation steps: (1) realizing kernel of a user mode protocol QUIC by modifying an open source msque into a kernel mode code; (2) embedding a kernel-based QUIC protocol code into a source code of an RDMA network card driver; (3) establishing a link by calling a QUIC API (Application Program Interface) and using a link establishment logic of a QUIC protocol, and realizing 0-RTT; according to the method provided by the invention, the link establishment time is greatly reduced, and the communication performance of the whole network is improved in a scene requiring frequent link establishment, such as large-model multi-machine multi-card distributed training.

Description

Technical Field

[0001] The present invention relates to an RDMA link building method based on the QUIC protocol, belonging to the field of information technology, in particular to the field of RDMA network card technology. Background Art

[0002] Although RDMA network card technology can provide an efficient and low-latency data transmission method, there are still some problems in practical applications. For example, in scenarios such as large-scale and frequent link establishment, such as large-scale model multi-machine and multi-card distributed training, the long RDMA network card link establishment time often causes network congestion.

[0003] Designing a more efficient link building method and achieving more efficient link building has become a technical problem that urgently needs to be solved in the current RDMA network card technology field. Summary of the invention

[0004] In view of this, the purpose of the present invention is to modify the RDMA network card communication connection establishment method by modifying the RDMA network card driver source code protocol stack, so as to achieve more efficient link establishment.

[0005] In order to achieve the above object, the present invention proposes a RDMA link establishment method based on the QUIC protocol, and the method includes the following steps:

[0006] (1) By modifying the open source msquic into kernel code, the user-mode protocol QUIC is kernelized;

[0007] (2) Embed the kernelized QUIC protocol code into the source code of the RDMA network card driver;

[0008] (3) By calling the QUIC API, the link establishment logic of the QUIC protocol is used to establish a link, and 0-RTT can be achieved; the 0-RTT means that an encrypted channel will be formed after the first connection is established, and then when the connection is established again, the first data packet can carry valid transmission data.

[0009] The specific method of kernelizing the user-mode protocol QUIC by modifying the open source msquic (msquic is the Microsoft implementation of the IETF QUIC protocol) into kernel-mode code is: modifying the code portion of the library function of the user library part into the Linux kernel library or redesigning the relevant library functions, including but not limited to memory application functions, memory release functions, data type conversion functions, etc.

[0010] The specific method of embedding the kernelized QUIC protocol code into the source code of the RDMA network card driver is: first download the source code of the network card driver from the official website, and then embed the kernelized QUIC code, specifically including:

[0011] (1) Adapt the RDMA network card driver code to the QUIC protocol;

[0012] (2) The part of establishing a connection with the RDMA network card is modified so that the link establishment part is changed from the original TCP protocol link establishment to the QUIC protocol link establishment.

[0013] The specific method of establishing a link by calling the QUIC API and using the link establishment logic of the QUIC protocol to achieve 0-RTT is: using the open source GnuTLS (GNU Transport Layer Security library) and writing a QUIC API similar to SocketAPI to establish a link and achieve 0-RTT; after modifying the link establishment method, compared with the link establishment method before the modification, each time a connection is established, 2 round-trip interaction times are reduced.

[0014] The beneficial effects of the present invention are that by kernelizing the user-mode protocol QUIC and embedding it into the RDMA network card driver code, the RDMA network card can use the user-mode QUIC protocol stack; by modifying the RDMA network card link establishment process from Socket link establishment to QUIC link establishment, the link establishment time is greatly reduced, and the communication performance of the overall network is improved in scenarios where frequent link establishment is required, such as large-model multi-machine and multi-card distributed training. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flowchart of a RDMA link establishment method based on the QUIC protocol proposed by the present invention; DETAILED DESCRIPTION

[0016] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings.

[0017] See also Figure 1 , introduces an RDMA link establishment method based on the QUIC protocol proposed by the present invention, the method comprising the following steps:

[0018] (1) By modifying the open source msquic into kernel code, the user-mode protocol QUIC is kernelized;

[0019] (2) Embed the kernelized QUIC protocol code into the source code of the RDMA network card driver;

[0020] (3) By calling the QUIC API, the link establishment logic of the QUIC protocol is used to establish a link, and 0-RTT can be achieved; the 0-RTT means that an encrypted channel will be formed after the first connection is established, and then when the connection is established again, the first data packet can carry valid transmission data.

[0021] The specific method of kernelizing the user-mode protocol QUIC by modifying the open source msquic into kernel-mode code is: modifying the code portion of the library function of the user library part into the Linux kernel library or redesigning the relevant library functions, including but not limited to memory application functions, memory release functions, data type conversion functions, etc.

[0022] For example: in the embodiment, the kernel state cannot use the malloc function, and the kmalloc function and the vmalloc function provided by the kernel must be used when applying for memory; the kernel state cannot use the printf function provided by the standard library to output information, and must use the printk function provided by the kernel instead.

[0023] The specific method of embedding the kernelized QUIC protocol code into the source code of the RDMA network card driver is: first download the source code of the network card driver from the official website, and then embed the kernelized QUIC code, specifically including:

[0024] (1) Adapt the RDMA network card driver code to the QUIC protocol. In the embodiment, the adaptation method is mainly to integrate the code with repeated functions. For example, the QUIC protocol uses GnuTLS and the driver itself also uses TLS. The code with similar repeated functions is integrated and adapted.

[0025] (2) The part of establishing a connection with the RDMA network card is modified so that the link establishment part is changed from the original TCP protocol link establishment to the QUIC protocol link establishment.

[0026] The specific method of calling the QUIC API, using the link building logic of the QUIC protocol to establish a link, and achieving 0-RTT is: using the open source GnuTLS (GNU Transport Layer Security library) and writing a QUIC API similar to SocketAPI to establish a link and achieve 0-RTT; after modifying the link building method, compared with the link building method before the modification, each time a connection is established, 2 round-trip interaction times are reduced.

[0027] In an embodiment, when the TCP protocol establishes a connection for the first time, the client sends a SYN request, the server replies with a SYN-ACK, and the client sends an ACK confirmation, completing 1 RTT to enter the ESTABLISHED state. Subsequently, the TLS handshake begins: the client sends a ClientHello message, the server replies with a ServerHello and a certificate, the client verifies the certificate and sends a Pre-master Secret, the server and the client negotiate to generate a session key, and both parties send a Finished message to ensure that the encrypted channel is established, which requires a total of 2 additional RTTs, so the entire process requires 3 RTTs. During the first connection establishment of the QUIC protocol, the client sends an initial data packet with encrypted handshake data to the server, including ClientHello. After receiving it, the server immediately returns ServerHello and encryption settings, and transmits the certificate at the same time. After the client verifies the server certificate, the key exchange is completed and the session key is generated. QUIC combines the encrypted handshake and data transmission within one RTT. After the encrypted channel is established, TCP requires 2RTT time, while QUIC implements 0-RTT transmission.

[0028] The inventors have conducted a large number of experiments on the system proposed in the present invention, and the experimental results are as follows:

[0029] Perftest (an open source tool for testing the performance between RDMA network cards) was used to test between two servers equipped with Mellanox Connectx-5100G network cards. The test simulated an environment with frequent link establishment and set up 1000 link establishments. By changing the total throughput, the comparison of parameters such as bandwidth and average link establishment time under different throughput environments was tested. The experimental results are shown in Table 1.

[0030] Table 1

[0031]

[0032] In Table 1, QUIC represents the method of the present invention. The experimental results prove that the method proposed in the present invention is feasible and effective.

Claims

1. A RDMA link establishment method based on the QUIC protocol, characterized in that: The method comprises the following steps: (1) By modifying the open source msquic into kernel code, the user-mode protocol QUIC is kernelized; (2) Embed the kernelized QUIC protocol code into the source code of the RDMA network card driver; (3) By calling the QUIC API, the link establishment logic of the QUIC protocol is used to establish a link, and 0-RTT can be achieved; the 0-RTT means that an encrypted channel will be formed after the first connection is established, and then when the connection is established again, the first data packet can carry valid transmission data.

2. According to the RDMA link establishment method based on the QUIC protocol according to claim 1, it is characterized in that: The specific method of kernelizing the user-mode protocol QUIC by modifying the open source msquic into kernel-mode code is: modifying the code portion of the library function of the user library part into the Linux kernel library or redesigning the relevant library functions, including but not limited to memory application functions, memory release functions, data type conversion functions, etc.

3. According to the RDMA link establishment method based on the QUIC protocol according to claim 1, it is characterized in that: The specific method of embedding the kernelized QUIC protocol code into the source code of the RDMA network card driver is: first download the source code of the network card driver from the official website, and then embed the kernelized QUIC code, specifically including: (1) Adapt the RDMA network card driver code to the QUIC protocol; (2) The part of establishing a connection with the RDMA network card is modified so that the link establishment part is changed from the original TCP protocol link establishment to the QUIC protocol link establishment.

4. According to the RDMA link establishment method based on the QUIC protocol according to claim 1, it is characterized in that: The specific method of calling the QUIC API, using the link building logic of the QUIC protocol to establish a link, and achieving 0-RTT is: using the open source GnuTLS (GNU Transport Layer Security library) and writing a QUIC API similar to SocketAPI to establish a link and achieve 0-RTT; after modifying the link building method, compared with the link building method before the modification, each time a connection is established, 2 round-trip interaction times are reduced.

Images