RDMA network transmission packet loss recovery method and system

Abstract

The invention provides an RDMA (Remote Direct Memory Access) network transmission packet loss recovery method and system. The method comprises the following steps: a sending end determines that a data stream to be transmitted is a larger or smaller data stream based on a multiple of a bandwidth delay product of an RDMA network; performing redundant coding and code marking on the smaller data stream, generating a data stream carrying a code mark and a redundant data packet, and transmitting the data stream to a receiving end; dividing a larger data stream into a non-tail BDP part and a tail BDP part based on the multiple of the bandwidth delay product, performing redundant coding and coding marking on the tail BDP part, and generating the tail BDP part carrying a coding mark and a redundant data packet; and transmitting a data stream formed by the non-tail BDP part and the tail BDP part carrying the coding mark and the redundant data packet to a receiving end, so that the receiving end carries out packet loss and coding mark detection on the received data stream, and if the packet loss and the coding mark exist, the lost data packet is recovered based on the carried redundant data packet. And if there is packet loss and there is no coding mark, requesting the sending end to select to retransmit and recover the lost data packet.

Description

Technical Field

[0001] The present invention relates to the field of communication technologies, and in particular, to a method and system for recovering lost packets in RDMA network transmission. Background Art

[0002] With the rapid development of information technology, the demand for long-distance cross-domain data transmission has increased significantly. In scientific research, disciplines such as astronomy, meteorology, and geography need to transmit massive amounts of observation data from the collection site to the analysis center; in geographically distributed machine learning, task collaboration depends on the rapid data transmission between different regions to achieve efficient computing and model optimization; in the field of artificial intelligence large model training, a single computing cluster has difficulty meeting the huge computing power requirements, and multi-cluster cross-domain joint training has gradually become a new trend. Long-distance transmission usually uses optical fiber as the transmission medium, and random packet loss problems may occur during data transmission due to factors such as mechanical vibration, optoelectronic conversion, and equipment failure. Packet loss not only seriously affects the throughput of long-distance transmission but also significantly increases the flow completion time of data interaction. There is an urgent need for an effective packet loss recovery solution to ensure the reliability of transmission.

[0003] TCP (Transmission Control Protocol) can maintain the integrity of data transmission through mechanisms such as congestion control and flow management in a lossy network. However, the protocol stack of TCP is relatively complex, especially when dealing with large-scale data streams, its protocol overhead increases significantly. Packet transmission requires frequent context switching and system interrupts, occupying a large amount of CPU resources and making it difficult to support high-speed data transmission. RDMA (Remote Direct Memory Access) technology has been widely used within the intelligent computing center due to its ability to bypass the operating system kernel and directly access remote memory. Through mechanisms such as zero-copy and remote memory mapping, RDMA can significantly reduce the latency of data transmission and achieve throughput performance close to the hardware limit. The high performance of RDMA highly depends on the support of the underlying lossless network. By deploying flow control mechanisms such as PFC (Priority Flow Control), it ensures the lossless transmission of packets within the intelligent computing center. However, in a lossy long-distance transmission environment, deploying PFC will cause the forwarding suspension of a certain node to quickly spread to the entire long-distance network, resulting in serious performance losses. In the case where the underlying network cannot provide a lossless environment, packet loss will cause a sharp decrease in the throughput of long-distance transmission, thereby affecting the flow completion time.

[0004] Therefore, for the random packet loss problem in a lossy long-distance RDMA network, it is necessary to specifically design an adaptive and low-overhead packet loss recovery solution. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method and system for recovering lost packets in RDMA network transmission to eliminate or improve one or more defects existing in the prior art.

[0006] The first aspect of the present invention provides a method for recovering lost packets in RDMA network transmission, and the method includes the following steps:

[0007] The sending end determines, based on a multiple of the bandwidth-delay product of the RDMA network, whether the data stream to be transmitted is a relatively large data stream to be transmitted or a relatively small data stream to be transmitted, and the data stream includes multiple data packets;

[0008] Redundantly encode and encode-mark the relatively small data stream to be transmitted to generate a first data stream, and transmit the first data stream to the receiving end, where the first data stream is a data stream carrying an encoding mark and redundant data packets;

[0009] Divide the relatively large data stream to be transmitted into a non-tail bandwidth-delay product part and a tail bandwidth-delay product part based on the multiple of the bandwidth-delay product, redundantly encode and encode-mark the tail bandwidth-delay product part to generate a tail bandwidth-delay product part carrying an encoding mark and redundant data packets;

[0010] Transmit a second data stream formed by the non-tail bandwidth-delay product part and the tail bandwidth-delay product part carrying an encoding mark and redundant data packets to the receiving end, so that the receiving end detects lost packets and encoding marks in the received data stream. If there are lost packets and there is an encoding mark, recover the lost data packets based on the carried redundant data packets. If there are lost packets and there is no encoding mark, request the sending end to select and retransmit to recover the lost data packets.

[0011] In some embodiments of the present invention, the method further includes:

[0012] Periodically count the lost packet frequency. If the lost packet frequency is greater than a preset threshold, increase the redundancy encoding ratio and reduce the size of the redundancy encoding block;

[0013] If the lost packet frequency is less than the preset threshold, reduce the redundancy encoding ratio and increase the size of the redundancy encoding block.

[0014] In some embodiments of the present invention, before transmitting the first data stream to the receiving end, the method further includes:

[0015] Prioritize the transmission of the first data stream by increasing the transmission priority of the first data stream.

[0016] In some embodiments of the present invention, the sending end determines, based on a multiple of the bandwidth-delay product of the RDMA network, whether the data stream to be transmitted is a relatively large data stream to be transmitted or a relatively small data stream to be transmitted, including:

[0017] The sender compares the size of the data stream to be transmitted with a multiple of the bandwidth-delay product of the RDMA network. If the size of the data stream to be transmitted is greater than the multiple of the bandwidth-delay product, the data stream to be transmitted is a relatively large data stream to be transmitted;

[0018] If the size of the data stream to be transmitted is less than or equal to the multiple of the bandwidth-delay product, the data stream to be transmitted is a relatively small data stream to be transmitted.

[0019] The second aspect of the present invention provides an RDMA network transmission packet loss recovery method, which includes the following steps:

[0020] The receiver receives the first data stream or the second data stream from the sender. The first data stream is a data stream carrying a coding mark and a redundant data packet obtained by performing redundant coding and coding marking on a relatively small data stream to be transmitted determined based on a multiple of the bandwidth-delay product of the RDMA network. The second data stream is formed by a non-tail bandwidth-delay product part and a tail bandwidth-delay product part carrying a coding mark and a redundant data packet. The non-tail bandwidth-delay product part and the tail bandwidth-delay product part carrying a coding mark and a redundant data packet are obtained by dividing a relatively large data stream to be transmitted determined based on the multiple of the bandwidth-delay product into a non-tail bandwidth-delay product part and a tail bandwidth-delay product part and performing redundant coding and coding marking on the tail bandwidth-delay product part;

[0021] Perform packet loss and coding mark detection on the received data stream;

[0022] If there is packet loss and there is a coding mark, recover the lost data packets in the received data stream based on the carried redundant data packets;

[0023] If there is packet loss and there is no coding mark, request the sender to select and retransmit to recover the lost data packets in the received data stream.

[0024] In some embodiments of the present invention, in the case of continuous packet loss during the transmission of the tail bandwidth-delay product part carrying a coding mark and a redundant data packet in the second data stream and the first data stream, request the sender to select and retransmit to recover the lost data packets in the received data stream.

[0025] In some embodiments of the present invention, requesting the sender to select and retransmit to recover the lost data packets in the received data stream includes:

[0026] Based on each lost data packet, feedback a selective acknowledgment message to the sender so that the sender retransmits the lost data packets to the receiver based on the received selective acknowledgment message.

[0027] In some embodiments of the present invention, in the step of feeding back selective acknowledgment messages to the sender based on each lost data packet, cumulative selective acknowledgment is used for feedback.

[0028] The third aspect of the present invention provides an RDMA network transmission packet loss recovery system, which includes: a computer device, the computer device includes a processor and a memory, computer instructions are stored in the memory, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method described in the foregoing first aspect, or implements the steps of the method described in the foregoing second aspect.

[0029] The fourth aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the method described in the foregoing first aspect, or implements the steps of the method described in the foregoing second aspect.

[0030] The fifth aspect of the present invention provides a computer program product, including computer instructions, which implement the steps of the method described in the foregoing first aspect, or implement the steps of the method described in the foregoing second aspect when executed by a processor.

[0031] The RDMA network transmission packet loss recovery method and system of the present invention can provide an adaptive and low-overhead packet loss recovery scheme for the random packet loss problem in the lossy long-distance transmission process of the RDMA network, breaking the inherent situation that the original coding bandwidth and computational overhead increase linearly with the increase of the transmitted data volume. It organically combines the low-overhead advantage of selective retransmission and the fast packet loss recovery advantage of redundant coding, and while ensuring the reliability of data stream transmission, reduces the bandwidth resource overhead as much as possible, and ensures reliable long-distance transmission with extremely low overhead, meeting the cross-domain transmission requirements of high throughput, low latency, and anti-packet loss. And without actively sending probes to sense the link state, it adaptively adjusts the redundant coding rate to further save long-distance transmission bandwidth resources.

[0032] The additional advantages, objectives, and features of the present invention will be partially described below, and will become partially apparent to those of ordinary skill in the art after studying the following text, or can be learned from the practice of the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in the specification and the drawings.

[0033] Those skilled in the art will understand that the objectives and advantages that can be achieved by the present invention are not limited to the above specifically described, and the above and other objectives that the present invention can achieve will be more clearly understood according to the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings described herein are used to provide a further understanding of the present invention, form a part of this application, and do not limit the present invention.

[0035] Figure 1 It is a schematic flowchart of a method for recovering lost packets in RDMA network transmission in an embodiment of the present invention;

[0036] Figure 2 It is a schematic flowchart of a method for recovering lost packets in RDMA network transmission in another embodiment of the present invention;

[0037] Figure 3 It is a specific schematic flowchart of a method for recovering lost packets in RDMA network transmission in an embodiment of the present invention. Detailed implementation manners

[0038] To make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below in combination with the implementation manners and the accompanying drawings. Herein, the illustrative implementation manners of the present invention and their descriptions are used to explain the present invention, but do not limit the present invention.

[0039] Herein, it also needs to be noted that in order to avoid obscuring the present invention due to unnecessary details, only the structures and / or processing steps closely related to the solution of the present invention are shown in the accompanying drawings, while other details less related to the present invention are omitted.

[0040] It should be emphasized that the term "comprising / including" when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0041] Herein, it also needs to be noted that if not specifically stated, the term "connection" in this article can not only refer to a direct connection, but also represent an indirect connection with an intermediate.

[0042] In the following, embodiments of the present invention will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0043] Regarding the problem of packet loss recovery in the network, there are currently two solutions. One is the packet retransmission-based solution, and the other is the packet protection solution based on redundant coding. The basic idea of the packet retransmission-based solution is to detect packet loss through an intermediate switch or the receiving end and feedback the packet loss situation to the sending end. The sending end retransmits the lost packets according to the packet loss feedback. Common retransmission algorithms include GBN (Go-Back-N) and SR (Selective Repeat). The basic idea of the packet protection solution based on redundant coding is that when the sending end sends data, it generates redundant packets through FEC (Forward Error Correction) coding methods such as exclusive OR calculation. When the receiving end detects packet loss, it recovers the lost packets through the received redundant packets.

[0044] The packet retransmission-based solution is the main means of packet loss recovery at present, and reliable transmission can be achieved without additional bandwidth overhead. In the RDMA network, GBN is used to ensure the reliability of data transmission. However, when a packet is lost during transmission, the receiving party does not separately request the retransmission of the lost packet, but the sending party needs to retransmit all subsequent packets starting from the lost packet. This method results in high bandwidth consumption and inefficient retransmission. Especially in a network environment with poor link quality or high latency, the performance of RDMA will decrease significantly. In selective repeat, only when a packet is lost, the receiving party requests the retransmission of the specific lost packet, rather than retransmitting all subsequent packets together, thereby improving bandwidth efficiency and reducing unnecessary retransmissions. The NVIDIA commercial network card ConnectX-6D supports the selective repeat function to avoid the performance loss caused by GBN. IRN and SRNIC (an extensible architecture for RDMA network cards) implement selective repeat based on hardware. The receiving end maintains a packet bitmap to record the received packet sequence, and notifies the sending end of the packet sequence that has not been received by sending a SACK (Selective Acknowledgment) message. After receiving the retransmitted packet, the bitmap is recounted and the complete data block is delivered to the upper-layer application. One solution comprehensively implements selective repeat and reliability assurance based on software on the basis of using commercial network card hardware without modifying the network card hardware. Another solution further moves the packet loss feedback from the end side to the switch. After the intermediate switch detects packet loss, it generates corresponding packet loss feedback and quickly feeds it back to the sending end for packet retransmission recovery.

[0045] Although the existing packet loss recovery scheme based on selective packet retransmission does not require additional transmission bandwidth overhead, it cannot quickly feedback SACK in a long-distance transmission network environment, which causes the receiving end to wait at least one RTT (Round-Trip Time) after detecting packet loss to successfully receive the retransmitted packet. Then, for small flows and latency-sensitive services, the long-distance transmission latency brought by this scheme greatly affects the flow transmission time, service completion time, and application performance. In addition, if the SACK or retransmitted packet is lost, the waiting time will be further increased, which will seriously affect the network transmission performance.

[0046] The packet protection scheme based on redundant coding requires additional data transmission bandwidth, but the receiving end can perform packet loss recovery without waiting for an additional RTT. Among them, FEC coding is mostly used for redundant coding. FEC is a technology that improves the reliability of data transmission by adding redundant information at the sending end, allowing the receiving end to still recover the original data even when some packets are lost, thereby reducing the impact of network packet loss on network transmission quality. The core idea of FEC is to provide sufficient error correction capabilities through redundant information, so that even if packets are lost or damaged during transmission, the receiving end can correctly recover the original data. In WebRTC (Real-Time Communication Technology), there are mainly two FEC implementation methods: ULPFEC and FlexFEC. ULPFEC protects the integrity of a single packet by adding redundant information to the packet and is suitable for network environments with low packet loss; FlexFEC is an extension of ULPFEC that can correct errors for multiple packets. By combining multiple packets into a coding block and adding redundant information, it improves the error correction ability under high packet loss rates or unstable network conditions. There is also a scheme that adds an intermediate layer between the RDMA transport layer and the UDP / IP layer, and adds redundant packets through exclusive OR calculation to perform redundant coding on all long-distance transmission packets.

[0047] Although the existing packet protection and recovery scheme based on redundant coding has certain advantages in data transmission latency, it needs to encode all long-distance transmission packets, that is, regardless of whether the link state is good or bad, redundant protection will be performed on all packets in the transmitted data stream, occupying additional bandwidth resources. And in the same redundant coding block, only the loss of one packet can be recovered. If multiple packets are lost continuously, the lost packets cannot be recovered. This scheme must make a trade-off between bandwidth loss and packet protection degree, and no matter how the coding ratio is adjusted, it will cause bandwidth loss, thus wasting precious long-distance transmission bandwidth resources.

[0048] Based on this, in order to solve the problem of random packet loss in lossy long-distance transmission of RDMA networks, embodiments of the present invention propose an RDMA network transmission packet loss recovery method and system that can adapt to the lossy long-distance transmission environment of RDMA networks. Its goal is to combine the low overhead advantage of selective repeat and the fast packet loss recovery advantage of redundant coding and be able to adapt to the lossy long-distance transmission network environment of RDMA, ensuring the transmission reliability of data against packet loss, while minimizing the bandwidth overhead as much as possible and shortening the flow transmission time and the corresponding service completion time. In the process of achieving low overhead and adapting to packet loss recovery in lossy long-distance transmission of RDMA networks, the present invention needs to overcome two challenges. Among them, Challenge 1 lies in how to organically combine selective repeat and redundant coding and how to design their switching mechanism to give full play to the advantages of both packet loss recovery schemes; Challenge 2 lies in how to design an adaptive redundant coding algorithm that can adaptively adjust the coding ratio according to the states of different transmission links. To address the above two challenges, the present invention deeply analyzes the disadvantages of the two packet loss recovery schemes of selective repeat and redundant coding and the root causes of these disadvantages, that is, packet loss in the tail BDP (Bandwidth-Delay Product) part of the selective repeat data stream during transmission will extremely affect the flow transmission time and the corresponding service completion time, and all data packets in the redundant coding data stream will extremely waste the transmission link bandwidth resources. In response to this, the RDMA network transmission packet loss recovery method and system designed and proposed by the present invention include a tail BDP-based packet loss recovery mechanism and an adaptive coding mechanism that organically combine the two packet loss recovery schemes of selective repeat and redundant coding, which can address the above two challenges and overcome the above disadvantages, and ensure the reliable transmission of data in the lossy long-distance transmission environment of RDMA networks at as low a cost as possible.

[0049] Figure 1 and Figure 3 are respectively the schematic flowchart and the specific flowchart of the RDMA network transmission packet loss recovery method in an embodiment of the present invention. As Figure 1 and Figure 3 shown, the execution subject of this method is the sending end, including the following steps:

[0050] Step S110, the sending end determines whether the data stream to be transmitted is a larger data stream to be transmitted or a smaller data stream to be transmitted based on a multiple of the bandwidth-delay product of the RDMA network, and the data stream includes multiple data packets.

[0051] The bandwidth-delay product of an RDMA network is a metric for measuring the performance of an RDMA network, representing the product of the propagation delay and the bandwidth of the link. The size of the delay-bandwidth product of different RDMA networks may vary. In the specific implementation process, when the present method issues a work request for cross-domain transmission service data stream at the sending end, it calculates the size of the BDP according to the maximum bandwidth allowed for a single data stream in the RDMA network. First, the sending end compares the size of the data stream to be transmitted in the RDMA network with the multiple of the delay-bandwidth product of the RDMA network to determine whether the data stream to be transmitted is a larger data stream or a smaller data stream. If the size of the data stream to be transmitted in the RDMA network is greater than the multiple of the delay-bandwidth product of the RDMA network, it is determined that the data stream to be transmitted is a larger data stream (which can be called a large stream); if the size of the data stream to be transmitted in the RDMA network is less than or equal to the multiple of the delay-bandwidth product of the RDMA network, it is determined that the data stream to be transmitted is a smaller data stream (which can be called a small stream).

[0052] Step S120: Perform redundant encoding and encoding marking on the smaller data stream to be transmitted to generate a first data stream, and transmit the first data stream to the receiving end. The first data stream is a data stream carrying an encoding mark and redundant data packets.

[0053] Since smaller data streams often have higher requirements for transmission delay, the present method transmits smaller data streams in a redundant encoding manner and recovers lost data packets during the transmission process, which can ensure that the services of smaller data streams can be completed as soon as possible and meet the delay requirements of smaller data streams. In the specific implementation process, a part of the data packets in all the data packets of the smaller data stream to be transmitted are combined into encoding data blocks, so as to obtain multiple encoding data blocks. Then, exclusive OR operations can be performed on each encoding data block to obtain redundant data packets, so as to obtain multiple redundant data packets, and the multiple redundant data packets are added as redundant information to the original data stream to be transmitted. At the same time, encoding identification information is added and encapsulated in the packet headers of the data packets in the data stream to be transmitted to perform encoding marking on the data stream to be transmitted, and finally a data stream carrying encoding identification / marking and redundant data packets is obtained, so that the redundant information, encoding identification / marking and the original data stream to be transmitted are transmitted from the sending end to the receiving end together. In the redundant encoding process of other embodiments, FEC encoding methods such as RS codes or convolutional codes can also be used to replace the exclusive OR operation to obtain the above redundant information.

[0054] Step S130: Divide the larger data stream to be transmitted into a non-tail bandwidth-delay product part and a tail bandwidth-delay product part based on the multiple of the bandwidth-delay product, and perform redundant encoding and encoding marking on the tail bandwidth-delay product part to generate a tail bandwidth-delay product part carrying an encoding mark and redundant data packets.

[0055] Since redundant encoding of all packets in a large data stream incurs additional bandwidth overheads in both good and poor network link conditions, and packet loss recovery for the tail BDP part of a large data stream by selective retransmission has a significant impact on the streaming transmission time and the corresponding service completion time, this method uses selective retransmission to recover packet loss for the non-tail BDP part of a large data stream and redundant encoding to quickly recover packet loss for the tail BDP part of a large data stream, which can minimize bandwidth overheads and shorten the streaming transmission time. Specifically, the tail bandwidth-delay product part (tail BDP part) of a large data stream to be transmitted refers to the part composed of multiple packets with a length that is a multiple of the last BDP at the end of the large data stream to be transmitted, and the part composed of multiple packets in front of the tail BDP part in the large data stream to be transmitted is the non-tail BDP part. The specific implementation process of redundant encoding for the tail BDP part is similar to the way of redundant encoding for a small data stream in step S120. Since a large data stream includes two blocks, an encoded part (tail BDP part) and an unencoded part (non-tail BDP part), different markings are needed for the tail BDP part and the non-tail BDP part in the large data stream to distinguish these two blocks. Specifically, encoding identification information can be added and encapsulated only in the packet headers of the packets in the tail BDP part, so as to obtain the tail BDP part carrying the encoding identification / marking and redundant packets, and the non-tail BDP part without any identification / marking. It is also possible to add and encapsulate encoding identification information in the packet headers of the packets in the tail BDP part, and add and encapsulate non-encoding identification information in the packet headers of the packets in the non-tail BDP part, so as to obtain the tail BDP part carrying the encoding identification / marking and redundant packets, and the non-tail BDP part carrying the non-encoding identification / marking, that is, the tail BDP part carrying the encoding identification / marking and redundant packets, and the non-tail BDP part without the encoding identification / marking. When adding and encapsulating identification information, the marking method agreed upon by both the sending end and the receiving end is adopted. For example, the encoding identification and the non-encoding identification can be 1 and 0 respectively, that is, 1 represents redundant encoding and 0 represents selective retransmission; or only the encoding identification is 1, that is, 1 represents redundant encoding, and packets without any identification represent selective retransmission.

[0056] In this method, by determining whether the data stream to be transmitted is a large stream or a small stream based on multiples of the BDP, and locating the start packet of the tail BDP part in the large stream to divide the large stream into the tail BDP part and the non-tail BDP part, it is possible to better address the problem and application scenario where the BDP may change due to delay jitter that may occur in long-distance links in the RDMA network. In the specific implementation process, the range of multiples of the BDP is set to 1 to 2 times (including the two endpoints 1 and 2).

[0057] Step S140: Transmit a second data stream formed by the non-tail bandwidth-delay product part and the tail bandwidth-delay product part carrying coded tags and redundant data packets to the receiving end, so that the receiving end can detect packet loss and coded tag in the received data stream. If there is packet loss and there are coded tags, the lost data packets are recovered based on the carried redundant data packets. If there is packet loss and there are no coded tags, the lost data packets are recovered by requesting selective retransmission from the sending end.

[0058] The sending end transmits the data stream formed by the tail BDP part carrying coded identifiers / tags and redundant data packets and the non-tail BDP part without carrying coded identifiers / tags, that is, the second data stream, to the receiving end. After receiving the smaller first data stream or the larger second data stream sent by the sending end, the receiving end first updates the bitmap to record the packet status of the received first or second data stream, and continuously detects whether there is packet loss in the received first or second data stream. When no packet loss is detected, that is, no packet loss occurs, the receiving end directly outputs the received data stream or packet sequence and performs subsequent corresponding operations on the received data stream according to specific requirements; when packet loss is detected, that is, packet loss occurs, the receiving end further detects whether these lost data packets carry coded tags. If the lost data packets carry coded tags, the lost data packets are recovered using the received redundant data packets to ensure data integrity. If the lost data packets do not carry coded tags, a packet loss recovery scheme of selective retransmission is adopted to recover the lost data packets.

[0059] In some embodiments, the method further includes the following steps:

[0060] Periodically count the packet loss frequency. If the packet loss frequency is greater than a preset threshold, increase the redundancy coding ratio and reduce the size of the redundancy coding block;

[0061] If the packet loss frequency is less than the preset threshold, reduce the redundancy coding ratio and increase the size of the redundancy coding block.

[0062] During the actual transmission process, when packet loss occurs in the non-tail BDP part of a large data stream, the receiving end feeds back a SACK packet to the sending end. The sending end periodically counts the number of SACK packets received within each period, that is, the packet loss frequency. According to the random packet loss occurrence probability of the lossy long-distance network link, i.e., the packet loss frequency, the redundancy coding ratio is dynamically and adaptively adjusted, and the size of the corresponding coded data block is calculated. If packet loss occurs frequently within a certain period (the packet loss frequency is greater than the preset threshold), it indicates that the transmission state of the network link is poor and there are many interference factors. Then, more redundant data packets are required in the subsequent redundancy coding stage for the tail BDP part, that is, the size of the coded data block within this period needs to be reduced and the redundancy coding ratio needs to be increased. On the contrary, when less packet loss occurs within a certain period (the packet loss frequency is less than the preset threshold), it indicates that the network link state is good, which means that not too many redundant data packets are needed. The redundancy coding ratio can be reduced, the size of the redundancy coding block can be increased, and the bandwidth resource overhead for data protection can be reduced. For example, when one packet is lost for every 8 packets transmitted, redundancy coding is performed on every 8 packets, and these 8 packets are used as a coded data block, with a redundancy coding ratio of 1 / 8. When 2 - 3 or even more packets are lost for every 8 packets transmitted, the size of the coded data block needs to be reduced and the redundancy coding ratio needs to be increased. The coded data block can be set to 4 packets, and the redundancy coding ratio is 1 / 4. This adaptive coding mechanism can dynamically adjust the coding ratio according to the state of the network link, ensure accurate packet loss recovery while minimizing the additional bandwidth resource overhead, and improve the packet loss resistance ability under different network link states.

[0063] In some embodiments, before transmitting the first data stream to the receiving end, the method further includes the following steps:

[0064] The first data stream is preferentially transmitted by increasing the transmission priority of the first data stream.

[0065] Another objective of this method is to optimize the transmission efficiency of smaller data streams, reduce the flow completion time of small flows, and thus improve the overall network performance. Small flows are often more sensitive to latency. Therefore, in a long-distance transmission environment, special processing is required to avoid additional latency overhead caused by redundancy coding or flow control mechanisms. For this purpose, first, the smaller data streams are marked in real time at the sending end and divided into high-priority flows, which are processed separately from large flows. For these high-priority small flows, the transmission of small flows is preferentially arranged in the data forwarding queue to ensure that small flows can preferentially obtain bandwidth resources, avoid being delayed due to excessive queuing waiting time, and thus shorten the flow completion time of small flows. While ensuring the fast transmission of small flows, the adverse impact of redundancy coding on the transmission performance of small flows is also minimized to provide more efficient support for latency-sensitive long-distance transmission services.

[0066] Figure 2 This is a schematic flowchart of a method for recovering packet loss in RDMA network transmission in another embodiment of the present invention. As Figure 2 and Figure 3 shown, the execution subject of this method is the receiving end, including the following steps:

[0067] Step S210, the receiving end receives the first data stream or the second data stream from the sending end. The first data stream is a data stream carrying coding marks and redundant data packets obtained by performing redundant coding and coding marking on a smaller data stream to be transmitted determined based on a multiple of the bandwidth-delay product of the RDMA network. The second data stream is formed by a non-tail bandwidth-delay product part and a tail bandwidth-delay product part carrying coding marks and redundant data packets. The non-tail bandwidth-delay product part and the tail bandwidth-delay product part carrying coding marks and redundant data packets are obtained by dividing a larger data stream to be transmitted determined based on a multiple of the bandwidth-delay product into a non-tail bandwidth-delay product part and a tail bandwidth-delay product part according to a multiple of the bandwidth-delay product and performing redundant coding and coding marking on the tail bandwidth-delay product part;

[0068] Step S220, detect packet loss and coding marks in the received data stream;

[0069] Step S230, if there is packet loss and there are coding marks, recover the lost data packets in the received data stream based on the carried redundant data packets;

[0070] Step S240, if there is packet loss and there are no coding marks, request the sending end to select and retransmit to recover the lost data packets in the received data stream.

[0071] In some embodiments, in the case of continuous packet loss during the transmission of the tail bandwidth-delay product part carrying coding marks and redundant data packets in the second data stream and the first data stream, request the sending end to select and retransmit to recover the lost data packets in the received data stream.

[0072] In some embodiments, requesting the sending end to select and retransmit to recover the lost data packets in the received data stream includes the following steps:

[0073] Based on each lost data packet, feedback a selective acknowledgment message to the sending end, so that the sending end retransmits the lost data packets to the receiving end based on the received selective acknowledgment message.

[0074] When there is a situation of continuous packet loss in the data stream, the packet loss recovery scheme based on redundant coding, that is, redundant data packets, cannot be used to recover the continuously lost data packets. The receiving end immediately adopts the selective repeat packet loss recovery scheme to recover the continuously lost data packets, sends a SACK feedback to the sending end, and requests the retransmission of the lost data packets, so as to realize the switching between the two packet loss recovery schemes of redundant coding and selective repeat. In addition, the result of decoding recovery is also fed back to the sending end for optimizing the coding strategy.

[0075] In some embodiments, in the step of feeding back a selective acknowledgment message to the sending end based on each lost data packet, cumulative selective acknowledgment is adopted for feedback. That is, each time feedback is made, the sequence of data packets that should have been received but were actually not received in the bitmap is fed back, which can avoid the situation that the corresponding lost data packets cannot be retransmitted and thus cannot be recovered due to the loss of SACK during a certain feedback process.

[0076] In summary, the packet loss recovery method for RDMA network transmission in the embodiments of the present invention can provide an adaptive and low-overhead packet loss recovery scheme for the random packet loss problem in the lossy long-distance transmission process of the RDMA network, breaking the inherent situation that the original coding bandwidth and computational overhead increase linearly with the increase of the amount of transmitted data, organically combining the low-overhead advantage of selective repeat and the fast packet loss recovery advantage of redundant coding, minimizing the bandwidth resource overhead while ensuring the reliability of data stream transmission, ensuring reliable long-distance transmission with extremely low overhead, and meeting the cross-domain transmission requirements of high throughput, low latency, and anti-packet loss. And without actively sending probes to sense the link state, the redundant coding code rate is adaptively adjusted to further save the long-distance transmission bandwidth resources.

[0077] Correspondingly to the above method, the present invention also provides an RDMA network transmission packet loss recovery system, which includes a computer device. The computer device includes a processor and a memory. Computer instructions are stored in the memory, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the foregoing method.

[0078] The embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the foregoing method are implemented. The computer-readable storage medium may be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0079] An embodiment of the present invention also provides a computer program product, including computer instructions which, when executed by a processor, implement the steps of the foregoing method.

[0080] Those of ordinary skill in the art should understand that the various exemplary components, systems, and methods described in connection with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Specifically, whether to implement in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, and so on. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave on a transmission medium or a communication link.

[0081] It should be clear that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present invention.

[0082] In the present invention, the features described and / or illustrated for one embodiment can be used in the same or similar manner in one or more other embodiments, and / or combined with the features of other embodiments or replace the features of other embodiments.

[0083] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, various changes and variations can be made to the embodiments of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for recovering packet loss in RDMA network transmission, characterized in that: The method comprises: The transmitting end determines whether the data stream to be transmitted is a larger data stream to be transmitted or a smaller data stream to be transmitted based on multiples of the bandwidth-delay product of the RDMA network, wherein the data stream includes multiple data packets; Redundantly encode and encode the smaller data stream to be transmitted to generate a first data stream, and transmit the first data stream to a receiving end, where the first data stream is a data stream carrying the encoding mark and redundant data packets; Dividing the larger data stream to be transmitted into a non-tail bandwidth-delay product part and a tail bandwidth-delay product part based on multiples of the bandwidth-delay product, performing redundant coding and coding marking on the tail bandwidth-delay product part, and generating a tail bandwidth-delay product part carrying the coding mark and redundant data packets; A second data stream formed by the non-tail bandwidth-delay product part and the tail bandwidth-delay product part carrying the coding mark and the redundant data packet is transmitted to the receiving end, so that the receiving end performs packet loss and coding mark detection on the received data stream. If there is packet loss and there is a coding mark, the lost data packet is restored based on the carried redundant data packet; if there is packet loss and there is no coding mark, the lost data packet is restored by requesting selective retransmission to the sending end.

2. The method according to claim 1, characterized in that The method further comprises: Periodically count the packet loss frequency. If the packet loss frequency is greater than a preset threshold, increase the redundant coding ratio and reduce the size of the redundant coding block. If the packet loss frequency is less than the preset threshold, the redundant coding ratio is reduced and the size of the redundant coding block is increased.

3. The method according to claim 1, characterized in that Before transmitting the first data stream to the receiving end, the method further includes: Prioritizing transmission of the first data stream by increasing the transmission priority of the first data stream; The sending end determines whether the data stream to be transmitted is a larger data stream to be transmitted or a smaller data stream to be transmitted based on multiples of the bandwidth-delay product of the RDMA network, including: The sending end compares the size of the data stream to be transmitted with a multiple of the bandwidth-delay product of the RDMA network. If the size of the data stream to be transmitted is greater than the multiple of the bandwidth-delay product, the data stream to be transmitted is a larger data stream to be transmitted. If the size of the data stream to be transmitted is less than or equal to a multiple of the bandwidth-delay product, the data stream to be transmitted is a smaller data stream to be transmitted.

4. A method for recovering packet loss in RDMA network transmission, characterized in that: The method comprises: The receiving end receives a first data stream or a second data stream from a sending end, wherein the first data stream is a data stream carrying coding marks and redundant data packets obtained by redundantly encoding and coding marking a smaller data stream to be transmitted determined by a multiple of a bandwidth-delay product based on an RDMA network, and the second data stream is formed by a non-tail bandwidth-delay product part and a tail bandwidth-delay product part carrying coding marks and redundant data packets, wherein the non-tail bandwidth-delay product part and the tail bandwidth-delay product part carrying coding marks and redundant data packets are obtained by dividing a larger data stream to be transmitted determined by a multiple of the bandwidth-delay product into a non-tail bandwidth-delay product part and a tail bandwidth-delay product part based on the multiple of the bandwidth-delay product, and performing redundant encoding and coding marking on the tail bandwidth-delay product part; Perform packet loss and encoding mark detection on the received data stream; If there is packet loss and there is a coding mark, recover the lost data packets in the received data stream based on the carried redundant data packets; If there is packet loss and no coding mark, the lost data packets in the received data stream are recovered by requesting selective retransmission to the sending end.

5. The method according to claim 4, characterized in that In the case where the tail bandwidth-delay product part of the coding mark and redundant data packets is carried in the second data stream and continuous packet loss occurs during the transmission of the first data stream, the lost data packets in the received data stream are recovered by requesting selective retransmission to the sending end.

6. The method according to claim 4 or 5, characterized in that: Restoring lost data packets in the received data stream by requesting selective retransmission from the transmitting end, comprising: A selective confirmation message is fed back to the sending end based on each lost data packet, so that the sending end retransmits the lost data packet to the receiving end based on the received selective confirmation message.

7. The method according to claim 6, characterized in that In the step of feeding back a selective confirmation message to the sending end based on each lost data packet, a cumulative selective confirmation method is used for feedback.

8. A RDMA network transmission packet loss recovery system, comprising a processor, a memory, and computer instructions stored in the memory, characterized in that: The processor is used to execute the computer instructions, and when the computer instructions are executed, the system implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

Images