A method for long-distance data transmission based on RDMA protocol

Abstract

The present application relates to the technical field of long-distance data transmission, in particular to a long-distance data transmission method based on RDMA protocol, comprising: a sending end and a receiving end establish a reliable connection, transmit relevant information of a file, and exchange key parameters required for subsequent communication; the receiving end designs an adaptive address buffer pool and a suitable sliding window size; the receiving end and the sending end create necessary resources before RDMA transmission, and then start file transmission; a batch processing mechanism of queue elements is designed, a plurality of completion information is combined and processed, and performance is further optimized; the sending end reads and blocks file contents, requests available addresses from a receiving end address pool, writes the file blocks into a specified memory of the receiving end, writes the memory into a specified position of a hard disk after writing is completed, and reuses the address; after file transmission is completed, the created relevant resources are released, and the speed and efficiency of RDMA data transmission are significantly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of long-distance data transmission, and in particular to a long-distance data transmission method based on the RDMA protocol. Background Art

[0002] With the continuous advancement of cloud computing technology, distributed architecture has been widely used in high-performance computing and storage due to its high performance, high reliability, and easy scalability. As an important infrastructure supporting distributed business, data centers have developed rapidly in recent years.

[0003] To meet the transmission needs of upper-layer services, data center networks are constantly upgrading. Link rates have increased from 10 Gbps in the early days to 100 Gbps today, and a new generation of hardware devices supporting 400 Gbps has been released. For example, NVIDIA's fourth-generation intra-GPU communication link, NV-Link, boasts a total bandwidth of 900 Gbps for intra-GPU communication per GPU. At the same time, services are placing increasingly stringent demands on basic transmission latency, targeting microsecond levels. For example, cloud storage needs to provide performance close to that of local storage. Alibaba Cloud's enhanced SSD, for example, guarantees a latency of less than 200 microseconds for small 4KB messages. For distributed machine learning tasks, to efficiently train models and process data, the infrastructure network needs to compress the tail latency of short messages to less than 50 microseconds.

[0004] However, the traditional TCP network protocol stack can no longer meet these requirements. Regarding link speed, the TCP protocol stack requires approximately 1Hz of CPU frequency to process a 1-bit message. For a 100Gbps link, the TCP protocol stack would consume approximately 100GHz of CPU resources, placing a significant burden on service providers. Furthermore, when sending and receiving messages, the TCP protocol requires the kernel to perform multiple context switches and data copies. Each context switch takes approximately 5 to 10 microseconds, resulting in processing delays of tens of microseconds, creating a performance bottleneck.

[0005] Against this backdrop, Remote Direct Memory Access (RDMA) technology has been introduced to data centers. RDMA allows direct access to memory without requiring intervention from the operating systems of either communicating party. Its kernel-bypassing, zero-load, and zero-copy nature provide high throughput and low latency, making it a key technology for improving data center network performance. RDMA can reduce protocol stack processing latency to 1 millisecond and CPU utilization to 5% at 40Gbps transmission speeds, effectively overcoming the performance limitations of the TCP protocol stack at high link rates.

[0006] There are currently three main hardware implementations for RDMA: Infiniband, iWARP, and RoCE. RoCE is capable of performing RDMA communications in traditional Ethernet networks, requiring only a special network card, significantly reducing deployment costs. Currently, the RoCE protocol includes two implementation versions: RoCEv1 and RoCEv2. It uses a priority-based flow control mechanism and integrates a Go-Back-N retransmission mechanism within the network card. RoCEv1 is implemented based on the Ethernet link layer and only allows communication between two hosts in the same broadcast domain. RoCEv2 is implemented based on the Ethernet UDP layer. The main difference from RoCEv1 is that it can use the UDP and IP network layers, enabling RoCEv2 devices to support IP routing and communication between hosts in different broadcast domains.

[0007] SoftRoCE is a software implementation of RoCEv2. It is compatible with standard devices without RDMA network cards, enabling RDMA services to be deployed on devices lacking dedicated network cards, significantly reducing the cost of deploying RDMA networks. SoftRoCE works in any Ethernet environment, without relying on NICs, switches, or L2QoS support. It is primarily intended for testing, application development, and environments where RDMA applications need to be deployed despite lacking RDMA hardware. Its implementation consists of two parts: user space coupling with the API provided by libibverbs and kernel space coupling with the TCP / IP protocol stack through a driver. Users transmit RoCE data to virtual RDMA devices through a UDP tunnel on an Ethernet card. Summary of the Invention

[0008] The object of the present invention is to provide a long-distance data transmission method based on the RDMA protocol to solve the problems raised in the above background technology.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0010] A long-distance data transmission method based on the RDMA protocol, the method comprising:

[0011] S100, when the file transfer starts, the sending end establishes a reliable connection with the receiving end through the RDMACM method, transmits the relevant information of the file, and exchanges the key parameters required for subsequent communication;

[0012] S200: The receiving end designs an adaptive address buffer pool based on the file size and available memory capacity, and designs an appropriate sliding window size based on the network bandwidth, packet loss rate, and delay to optimize transmission performance.

[0013] S300, the receiving end and the sending end create the necessary resources before RDMA transmission, including Queue Pair, Protection Domain, Memory Region and Completion Queue, and then start the file transfer;

[0014] S400. Design a batch processing mechanism for completion queue elements to merge multiple completion messages and further optimize performance.

[0015] S500: The sender reads the file content and divides it into blocks, requests an available address from the address pool of the receiver, and writes the file blocks into the designated memory of the receiver via RDMAWRITE. After the writing is completed, the memory is written to the designated location on the hard disk via RDMACM communication and the address is reused.

[0016] S600: After the file transfer is completed, the created related resources are released.

[0017] Preferably, the address buffer pool designed in S200 according to the file size and available memory capacity includes:

[0018] S201-1. Determine the available memory capacity by performing a memory specification or performance test to obtain the available memory capacity.

[0019] S201-2. When the file size is less than or equal to the available memory capacity / 4, allocate an address buffer pool equivalent to the file size;

[0020] S201-3. When the file size is greater than the available memory capacity / 4, an address buffer pool of the available memory capacity / 4 is used.

[0021] Preferably, in S200, a suitable sliding window size is designed based on the network bandwidth, packet loss rate, and delay, including:

[0022] S202-1. Determine the network bandwidth, packet loss rate, and latency, and obtain the bandwidth, packet loss rate, and latency through performance testing;

[0023] S202-2. Since network bandwidth determines maximum throughput, latency determines network response speed and user experience, and packet loss rate determines network reliability, set an appropriate sliding window size based on the application scenario and system resources:

[0024] Set the delay to RTT (unit: ms), the packet loss rate to P, and the bandwidth to B (unit: bps);

[0025] Based on the delay and packet loss rate, the window size is set to:

[0026] If bandwidth B is considered, the window size should meet the network throughput requirement as follows:

[0027] Based on this, combined with bandwidth, packet loss rate and delay, the sliding window size is obtained as

[0028] S202-3. Develop strategies based on different scenarios adjusted , real-time monitoring of packet loss rate and delay, dynamic adjustment of sliding window size, balance of throughput and stability: W adjusted =W×(1-P);

[0029] When the packet loss rate is high and P>0.01, reduce W to avoid congestion;

[0030] In high-latency scenarios with RTT greater than 100ms, balance bandwidth and latency according to the formula and increase W.

[0031] Preferably, the batch processing mechanism for completing the queue elements in S400 includes:

[0032] S401. Set a fixed value CQ_MAX_SIZE. When the number of entries in the CQ reaches this value, the process is performed. Otherwise, the message "CQ is empty" is returned.

[0033] S402. Set a timeout value CQ_TIME_THRESHOLD. When no CQE is generated for a period of time, the system will proactively obtain CQEs smaller than CQ_MAX_SIZE in the CQ.

[0034] S403. The processed CQE is used to notify the server to write the data into the hard disk. Therefore, CQ_MAX_SIZE is determined by the block size and the address buffer pool size, and CQ_TIME_THRESHOLD is determined by the network bandwidth and the file size. The specific formula is as follows:

[0035] Assume that the block size is P and the address buffer pool size is M, then CQ_MAX_SIZE = M / P*10;

[0036] Assuming the network bandwidth is B and the file size is F, we can obtain CQ_TIME_THRESHOLD = F / B*5.

[0037] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned long-distance data transmission method based on the RDMA protocol.

[0038] A computer device includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the program, the steps of the above-mentioned long-distance data transmission method based on the RDMA protocol are implemented.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The present invention fully reuses memory resources by designing an efficient address buffer pool at the receiving end, and flexibly adjusts the sliding window according to the network packet loss rate and delay, thereby reducing the impact of the network environment on transmission efficiency. In addition, the design of batch completion queue elements reduces CPU utilization, thereby significantly improving the speed and efficiency of RDMA data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0042] Figure 1 This is a flow chart of a long-distance data transmission method based on the RDMA protocol of the present invention;

[0043] Figure 2 This is an RDMACM link establishment process of a long-distance data transmission method based on the RDMA protocol of the present invention;

[0044] Figure 3 The invention provides an RDMA resource creation process of a long-distance data transmission method based on the RDMA protocol;

[0045] Figure 4 The invention discloses an RDMA resource release process of a long-distance data transmission method based on an RDMA protocol. DETAILED DESCRIPTION

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0047] See also Figure 1-Figure 4 , the present invention provides a technical solution:

[0048] Example 1:

[0049] S100, when the file transfer starts, the sending end establishes a reliable connection with the receiving end through the RDMACM method, transmits the relevant information of the file, and exchanges the key parameters required for subsequent communication;

[0050] The specific steps of establishing a reliable connection with the receiving end using the RDMACM method in step S100 are as follows:

[0051] like Figure 2 As shown, the sender process is as follows:

[0052] 1. Create an event channel

[0053] Open a channel for reporting communication events. Asynchronous events will be reported to users through the event channel. The corresponding method is

[0054] struct rdma_event_channel*rdma_create_event_channel(void)

[0055] 2. Create a connection ID

[0056] Create an identifier for tracking communication information, the corresponding method is

[0057] int rdma_create_id(struct rdma_event_channel*channel,struct rdma_cm_id**id,void*context,enum rdma_port_space ps)

[0058] 3. Bind address

[0059] Associate the source address with the rdma_cm_id. The corresponding method is

[0060] int rdma_bind_addr(struct rdma_cm_id*id,struct sockaddr*addr)

[0061] 4. Create QP

[0062] Allocate the QP associated with the specified rdma_cm_id and convert it for transmission and reception. The corresponding method is

[0063] int rdma_create_qp(struct rdma_cm_id*id,struct ibv_pd*pd,struct ibv_qp_init_attr*qp_init_attr)

[0064] 5. Create Listener and return port / address

[0065] Initialize the Listener for incoming connection requests or data packet service lookups. The corresponding method is

[0066] int rdma_listen(struct rdma_cm_id*id,int backlog)

[0067] 6. Wait for connection request

[0068] Retrieve communication events until the RDMA_CM_EVENT_CONNECT_REQUEST event is received. The corresponding method is

[0069] int rdma_get_cm_event(struct rdma_event_channel*channel,struct rdma_cm_event**event)

[0070] 7. Accept the connection request

[0071] Waiting for the receiver to establish a connection, the corresponding method is

[0072] int rdma_accept(struct rdma_cm_id*id,struct rdma_conn_param*conn_param)

[0073] 8. Wait for the connection to be established

[0074] Retrieve communication events until the RDMA_CM_EVENT_CONNECT_REQUEST event is received, and the connection establishment is complete;

[0075] The receiving process is as follows:

[0076] 1. Create an event channel

[0077] Same as sender

[0078] 2. Create a connection ID

[0079] Same as sender

[0080] 3. Bind address

[0081] Resolve the destination address and optional source address from the IP address to an RDMA address. If successful, the specified rdma_cm_id will be bound to the local device. The corresponding method is

[0082] int rdma_resolve_addr(struct rdma_cm_id*id,struct sockaddr*src_addr,struct sockaddr*dst_addr,int timeout_ms)

[0083] 4. Wait for address resolution

[0084] Retrieve communication events until the RDMA_CM_EVENT_ADDR_RESOLVED event is received, the address resolution is completed, and the RDMA route pointing to the target address is resolved to establish a connection. The corresponding method is

[0085] int rdma_resolve_route(struct rdma_cm_id*id,int timeout_ms)

[0086] 5. Wait for routing resolution

[0087] Retrieve communication events until the RDMA_CM_EVENT_ROUTE_RESOLVED event is received, indicating that route resolution is complete.

[0088] 6. Create QP

[0089] Same as the sender

[0090] 7. Establish a connection

[0091] Request to establish a connection with the sender. The corresponding method is

[0092] int rdma_connect(struct rdma_cm_id*id,struct rdma_conn_param*conn_param)

[0093] 8. Wait for the connection to be established

[0094] Same as the sending end.

[0095] S200: The receiving end designs an adaptive address buffer pool based on the file size and available memory capacity, and designs an appropriate sliding window size based on the network bandwidth, packet loss rate, and delay to optimize transmission performance.

[0096] The specific steps for designing the address buffer pool size in step S200 are as follows:

[0097] S201-1. It is necessary to determine the available memory capacity, and obtain the available memory capacity through memory specifications or performance testing;

[0098] S201-2. When the file size is less than or equal to the available memory capacity / 4, an address buffer pool corresponding to the file size is allocated;

[0099] S201-3. When the file size is greater than the available memory capacity / 4, an address buffer pool of the available memory capacity / 4 is used.

[0100] In step S2, the specific steps for designing an appropriate sliding window size based on the network's packet loss rate and delay are as follows:

[0101] S202-1. It is necessary to determine the network bandwidth, packet loss rate, and latency. The bandwidth, packet loss rate, and latency are obtained through performance testing.

[0102] S202-2. Since network bandwidth determines the maximum throughput, latency determines the network response speed and user experience, and packet loss rate determines network reliability, an appropriate sliding window size can be set based on the application scenario and system resource availability.

[0103] S202-3. Assume that the delay is R (unit: ms), the packet loss rate is P, and the bandwidth is B (unit: bps)

[0104] According to the delay and packet loss rate, the window size is set to:

[0105] If bandwidth B is considered, the window size should meet the network throughput requirement as follows:

[0106] Based on this, combined with bandwidth, packet loss rate and delay, the sliding window size is obtained as

[0107] S202-3. Develop strategies based on different scenarios adjusted , real-time monitoring of packet loss rate and delay, dynamic adjustment of sliding window size, balance of throughput and stability: W adjusted =W×(1-P);

[0108] When the packet loss rate is high and P>0.01, reduce W to avoid congestion;

[0109] In high-latency scenarios with RTT greater than 100ms, balance bandwidth and latency according to the formula and increase W.

[0110] S300, the receiving end and the sending end create the necessary resources before RDMA transmission, including Queue Pair, Protection Domain, Memory Region and Completion Queue, and then start the file transfer;

[0111] like Figure 3As shown, in step S3, the receiving end and the sending end create the necessary resources before RDMA transmission. The specific steps are as follows:

[0112] S301. Allocate PD:

[0113] Protection domains (PDs) are used to group and isolate resources (such as queue pairs, memory areas, etc.) to ensure that operations between different queue pairs and memory areas do not interfere with each other. A protection domain is allocated to the device by calling the ibv_alloc_pd() method.

[0114] S302. Create CQ

[0115] The Completion Queue (CQ) is a core component that manages completion notifications for RDMA operations. It records the completion status of RDMA operations (such as sends, receives, and writes). When an RDMA operation completes, a Completion Queue Entry (CQE) is written to the associated CQ. The program queries the CQE to confirm the successful completion of the operation and creates a completion queue by calling the ibv_create_cq() method.

[0116] S303, create QP

[0117] Queue Pair (QP) is one of the core components of RDMA communication, responsible for the management and execution of data sending, receiving and related operations. RDMA communication can be abstracted as communication between QPs, and queue pairs are created by calling the ibv_create_qp() method.

[0118] S304, Register MR

[0119] Memory Regions (MRs) are a key concept in RDMA operations. MRs bridge direct memory access between applications and RDMA network hardware, providing secure, efficient, and direct access to memory. Registered Memory RDMA requires registering the memory where data resides before transmission. This is done by calling the ibv_reg_mr() method.

[0120] S305. Create threads and related resources

[0121] The sending end creates a thread for reading files, a sending thread for RDMA WRITE, and a completion queue thread. The receiving end creates a thread for writing files and an address pool.

[0122] S400. Design a batch processing mechanism for completion queue elements to merge multiple completion messages and further optimize performance.

[0123] The design scheme of the batch processing mechanism based on the completion queue elements in step S400 is as follows:

[0124] After each packet is sent, the network card sends a work completion message to the kernel. This is relatively inefficient and the system needs to spend a lot of time polling for work completion information. If the completion information can be sent to the kernel at one time after a fixed number of packets are sent, the system's work efficiency will be greatly improved.

[0125] S401. A fixed value CQ_MAX_SIZE is set. When the number of entries in the CQ reaches this value, processing is performed. Otherwise, the message "CQ is empty" is returned.

[0126] S402. A timeout value CQ_TIME_THRESHOLD is designed. When no CQE is generated for a period of time, the system will actively obtain CQEs less than CQ_MAX_SIZE in the CQ.

[0127] S403. The processed CQE is mainly used to notify the server to write the memory to the hard disk. Therefore, CQ_MAX_SIZE is determined by the block size and the address buffer pool size, and CQ_TIME_THRESHOLD is determined by the network bandwidth and the file size. The specific formula is as follows:

[0128] Assume that the block size is P and the address buffer pool size is MCQ_MAX_SIZE=M / P*10

[0129] Assume the network bandwidth is B and the file size is FCQ_TIME_THRESHOLD = F / B*5

[0130]

[0131] S500: The sender reads the file content and divides it into blocks, requests an available address from the address pool of the receiver, and writes the file blocks into the designated memory of the receiver via RDMAWRITE. After the writing is completed, the memory is written to the designated location on the hard disk via RDMACM communication and the address is reused.

[0132] The transmission design scheme in step S500 is as follows:

[0133] S501, the reading thread reads file information in blocks;

[0134] S502: Request address information from the receiving end address pool;

[0135] S503, write the file information into the receiving end memory via RDMA WRITE;

[0136] S504: The completion queue receives the completion information of writing into the memory from the receiving end and notifies the receiving end to write into the hard disk;

[0137] S505: Write the address to the specified location on the hard disk and add the address to the address pool again.

[0138] S600: After the file transfer is completed, the created related resources are released.

[0139] like Figure 4 As shown, the steps for releasing resources in S600 are as follows:

[0140] S601: Clean up work requests and buffers, complete or cancel all outstanding requests in the send queue and receive queue, and release the data buffer allocated to the RDMA operation;

[0141] S602. Deregister the memory and use ibv_dereg_mr() to release the memory region (Memory Region, MR) registered by ibv_reg_mr().

[0142] S603. Release the completion queue (CQ) and call ibv_destroy_cq() to release the completion queue resources.

[0143] S604. Destroy the queue pair (QP) and use ibv_destroy_qp() to release the queue pair (Queue Pair, QP).

[0144] S605: Release the protection domain (PD) by using ibv_dealloc_pd();

[0145] S606. Release the completion event channel. Use ibv_destroy_comp_channel() to release the completion event channel.

[0146] S607: Close the device context. Use ibv_close_device() to close the RDMA device context.

[0147] Example 2:

[0148] The computer-readable storage medium of this embodiment stores a computer program thereon, and when the program is executed by a processor, the steps of the long-distance data transmission method based on the RDMA protocol in embodiment 1 are implemented.

[0149] The computer-readable storage medium of this embodiment may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal; the computer-readable storage medium of this embodiment may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, etc. equipped on the terminal; further, the computer-readable storage medium may also include both an internal storage unit of the terminal and an external storage device.

[0150] The computer-readable storage medium of this embodiment is used to store computer programs and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0151] Example 3:

[0152] The computer device of this embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the long-distance data transmission method based on the RDMA protocol in Example 1 are implemented.

[0153] In this embodiment, the processor can be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The memory can include read-only memory and random access memory, and provide instructions and data to the processor. A part of the memory can also include non-volatile random access memory. For example, the memory can also store information about the device type.

[0154] Those skilled in the art will appreciate that the disclosed contents of the embodiments may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0155] The present solution is described with reference to the flowcharts and / or block diagrams of the methods and computer program products according to the embodiments of the present solution. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as the combination of the processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions; these computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or methods Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0156] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or methods Figure 1 The function specified in one or more boxes.

[0157] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or methods Figure 1 A step that specifies a function in one or more boxes.

[0158] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0159] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A long-distance data transmission method based on the RDMA protocol, characterized by: The method comprises: S100, when the file transfer starts, the sending end establishes a reliable connection with the receiving end through the RDMACM method, transmits the relevant information of the file, and exchanges the key parameters required for subsequent communication; S200: The receiving end designs an adaptive address buffer pool based on the file size and available memory capacity, and designs an appropriate sliding window size based on the network bandwidth, packet loss rate, and delay to optimize transmission performance. S300, the receiving end and the sending end create necessary resources before RDMA transmission, including Queue Pair, ProtectionDomain, Memory Region and Completion Queue, and then start the file transfer; S400. Design a batch processing mechanism for completion queue elements to merge multiple completion messages and further optimize performance. S500: The sender reads the file content and divides it into blocks, requests an available address from the address pool of the receiver, and writes the file blocks into the designated memory of the receiver via RDMA WRITE. After the write is completed, the memory is written to the designated location on the hard disk via RDMA CMM communication and the address is reused. S600: After the file transfer is completed, the created related resources are released.

2. A long-distance data transmission method based on the RDMA protocol as claimed in claim 1, characterized in that: The address buffer pool adapted to the file size and available memory capacity is designed in S200, including: S201-1. Determine the available memory capacity by performing a memory specification or performance test to obtain the available memory capacity. S201-2. When the file size is less than or equal to the available memory capacity / 4, allocate an address buffer pool equivalent to the file size; S201-3. When the file size is greater than the available memory capacity / 4, an address buffer pool of the available memory capacity / 4 is used.

3. The long-distance data transmission method based on the RDMA protocol according to claim 1, characterized in that: In S200, a suitable sliding window size is designed based on the network bandwidth, packet loss rate, and delay, including: S202-1. Determine the network bandwidth, packet loss rate, and latency, and obtain the bandwidth, packet loss rate, and latency through performance testing; S202-2. Set an appropriate sliding window size based on the application scenario and system resources: Set the delay to RTT, the packet loss rate to P, and the bandwidth to B; Based on the delay and packet loss rate, the window size is set to: If bandwidth B is considered, the window size should meet the network throughput requirement as follows: Based on this, combined with bandwidth, packet loss rate and delay, the sliding window size is obtained as S202-3. Develop strategies based on different scenarios adjusted , real-time monitoring of packet loss rate and delay, dynamic adjustment of sliding window size, balance of throughput and stability: W adjusted =W×(1-P); When the packet loss rate is high and P>0.01, reduce W to avoid congestion; In high-latency scenarios with RTT greater than 100ms, balance bandwidth and latency according to the formula and increase W.

4. The long-distance data transmission method based on the RDMA protocol according to claim 1, characterized in that: The batch processing mechanism for completing the queue elements in S400 includes: S401. Set a fixed value CQ_MAX_SIZE. When the number of entries in the CQ reaches this value, the process is performed. Otherwise, the message "CQ is empty" is returned. S402. Set a timeout value CQ_TIME_THRESHOLD. When no CQE is generated for a period of time, the system will proactively obtain CQEs smaller than CQ_MAX_SIZE in the CQ. S403. The processed CQE is used to notify the server to write the data into the hard disk. Therefore, CQ_MAX_SIZE is determined by the block size and the address buffer pool size, and CQ_TIME_THRESHOLD is determined by the network bandwidth and the file size. The specific formula is as follows: Assume that the block size is P and the address buffer pool size is M, then CQ_MAX_SIZE = M / P*10; Assuming the network bandwidth is B and the file size is F, we can obtain CQ_TIME_THRESHOLD = F / B*5.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the long-distance data transmission method based on the RDMA protocol as claimed in any one of claims 1 to 4 are implemented.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the steps of the long-distance data transmission method based on the RDMA protocol as described in any one of claims 1 to 4 are implemented.

Images