Rdma network transmission packet loss recovery method and system

Abstract

The application provides an RDMA network transmission packet loss recovery method and system, the method comprising: determining, by a sending end, a data stream to be transmitted as a large or small data stream based on a multiple of a bandwidth-delay product of an RDMA network; performing redundancy encoding and encoding marking on the small data stream, generating a data stream carrying the encoding marking and redundancy data packets, and transmitting the data stream to a receiving end; dividing the large data stream into a non-tail BDP part and a tail BDP part based on the multiple of the bandwidth-delay product, performing redundancy encoding and encoding marking on the tail BDP part, and generating a tail BDP part carrying the encoding marking and redundancy data packets; transmitting, to the receiving end, a data stream formed by the non-tail BDP part and the tail BDP part carrying the encoding marking and redundancy data packets, so that the receiving end performs packet loss and encoding marking detection on the received data stream, recovers lost data packets based on the carried redundancy data packets if there is packet loss and there is the encoding marking, and requests the sending end to perform selective repeat transmission to recover the lost data packets if there is packet loss and there is no encoding marking.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of communication technology, in particular to a RDMA network transmission packet loss recovery method and system. BACKGROUND

[0002] With the rapid development of information technology, the demand for long-distance cross-domain data transmission has significantly increased. In scientific research, disciplines such as astronomy, meteorology, and geography need to transmit massive amounts of observation data from collection sites to analysis centers; in geographically distributed machine learning, task collaboration relies on fast data transmission between different regions to achieve efficient computing and model optimization; in the field of artificial intelligence large model training, a single computing cluster has been difficult to meet the demand for massive computing power, and multi-cluster cross-domain joint training has gradually become a new trend. Long-distance transmission usually uses optical fiber as the transmission medium, and random packet loss problems may occur during data transmission due to mechanical vibration, photoelectric conversion, equipment failure, and other factors. Data packet loss not only seriously affects long-distance transmission throughput, but also significantly increases the flow completion time of data interaction, and there is an urgent need for effective packet loss recovery solutions to ensure transmission reliability.

[0003] TCP (Transmission Control Protocol) can maintain the integrity of data transmission through retransmission and window adjustment through mechanisms such as congestion control and flow management in lossy networks. However, the TCP protocol stack is complex, especially when handling large-scale data streams, its protocol overhead increases significantly. Data packet transmission requires frequent context switching and system interruption, occupying a large amount of CPU resources, making it difficult to support high-speed data transmission. RDMA (Remote Direct Memory Access) technology has been widely used in intelligent computing centers due to its ability to bypass the operating system kernel and directly access remote memory. Through mechanisms such as zero-copy and remote memory mapping, RDMA can significantly reduce data transmission latency and achieve near-hardware limit throughput performance. The high performance of RDMA highly depends on the support of underlying lossless networks, and through the deployment of PFC (Priority Flow Control) and other flow control mechanisms, data packets are ensured to be transmitted losslessly within the intelligent computing center. However, in a lossy long-distance transmission environment, the deployment of PFC will cause the forwarding pause of a node to rapidly spread to the entire long-distance network, resulting in serious performance loss. In the case where the underlying network cannot provide a lossless environment, data packet loss will cause the long-distance transmission throughput to drop sharply, thereby affecting the flow completion time.

[0004] Therefore, for the random packet loss problem of lossy long-distance RDMA networks, an adaptive and low-overhead packet loss recovery solution needs to be designed. SUMMARY

[0005] In view of this, the embodiments of the present application provide an RDMA network transmission packet loss recovery method and system to eliminate or improve one or more defects in the prior art.

[0006] The first aspect of the present application provides an RDMA network transmission packet loss recovery method, which comprises the following steps:

[0007] The sending end determines the to-be-transmitted data stream to be a larger to-be-transmitted data stream or a smaller to-be-transmitted data stream based on a multiple of a bandwidth-delay product of the RDMA network, and the data stream comprises a plurality of data packets;

[0008] The smaller to-be-transmitted data stream is redundantly encoded and encoded marked to generate a first data stream, and the first data stream is transmitted to the receiving end, wherein the first data stream is a data stream carrying encoded marks and redundant data packets;

[0009] The larger to-be-transmitted data stream is divided into a non-tail bandwidth-delay product part and a tail bandwidth-delay product part based on the multiple of the bandwidth-delay product, and the tail bandwidth-delay product part is redundantly encoded and encoded marked to generate a tail bandwidth-delay product part carrying encoded marks and redundant data packets;

[0010] A second data stream formed by the non-tail bandwidth-delay product part and the tail bandwidth-delay product part carrying encoded marks and redundant data packets is transmitted to the receiving end, so that the receiving end detects packet loss and encoded marks for the received data stream, recovers the lost data packets based on the carried redundant data packets if there is packet loss and encoded marks, and recovers the lost data packets by requesting selective repeat transmission from the sending end if there is packet loss and no encoded marks.

[0011] In some embodiments of the present application, the method further comprises:

[0012] Periodically counting packet loss frequency, and if the packet loss frequency is greater than a preset threshold, increasing the redundancy encoding ratio and reducing the size of the redundancy encoding block;

[0013] If the packet loss frequency is less than the preset threshold, the redundancy encoding ratio is reduced and the size of the redundancy encoding block is increased.

[0014] In some embodiments of the present application, before transmitting the first data stream to the receiving end, the method further comprises:

[0015] The first data stream is preferentially transmitted by increasing the transmission priority of the first data stream.

[0016] In some embodiments of the present application, the sending end determines the to-be-transmitted data stream to be a larger to-be-transmitted data stream or a smaller to-be-transmitted data stream based on a multiple of a bandwidth-delay product of the RDMA network, comprising:

[0017] comparing, by the sending end, a size of a to-be-transmitted data stream with a multiple of a bandwidth-delay product of the RDMA network, wherein if the size of the to-be-transmitted data stream is greater than the multiple of the bandwidth-delay product, the to-be-transmitted data stream is a larger to-be-transmitted data stream;

[0018] if the size of the to-be-transmitted data stream is less than or equal to the multiple of the bandwidth-delay product, the to-be-transmitted data stream is a smaller to-be-transmitted data stream.

[0019] A second aspect of the present application provides an RDMA network transmission packet loss recovery method, which comprises the following steps:

[0020] receiving, by the receiving end, a first data stream or a second data stream from the sending end, wherein the first data stream is a data stream carrying encoding marks and redundant data packets obtained by performing redundancy encoding and encoding marking on a smaller to-be-transmitted data stream determined based on a multiple of a bandwidth-delay product of the RDMA network, and the second data stream is formed by a non-tail bandwidth-delay product part and a tail bandwidth-delay product part carrying encoding marks and redundant data packets, wherein the non-tail bandwidth-delay product part and the tail bandwidth-delay product part carrying encoding marks and redundant data packets are obtained by dividing a larger to-be-transmitted data stream determined based on the multiple of the bandwidth-delay product into the non-tail bandwidth-delay product part and the tail bandwidth-delay product part based on the multiple of the bandwidth-delay product and performing redundancy encoding and encoding marking on the tail bandwidth-delay product part;

[0021] detecting packet loss and encoding marks in the received data stream;

[0022] if there is packet loss and encoding marks, recovering the lost data packets in the received data stream based on the carried redundant data packets;

[0023] if there is packet loss and no encoding marks, recovering the lost data packets in the received data stream by requesting selective repeat transmission from the sending end.

[0024] In some embodiments of the present application, in the case that the tail bandwidth-delay product part carrying encoding marks and redundant data packets in the second data stream and the first data stream are continuously lost during transmission, the lost data packets in the received data stream are recovered by requesting selective repeat transmission from the sending end.

[0025] In some embodiments of the present application, the lost data packets in the received data stream are recovered by requesting selective repeat transmission from the sending end, which comprises:

[0026] feeding back selective acknowledgement messages to the sending end based on each lost data packet, so that the sending end retransmits the lost data packets to the receiving end based on the received selective acknowledgement messages.

[0027] In some embodiments of the present application, in the step of feeding back the selective acknowledgement message to the sending end based on each lost data packet, the cumulative selective acknowledgement mode is adopted for feedback.

[0028] The third aspect of the present application provides an RDMA network transmission packet loss recovery system, comprising: a computer device, the computer device comprising a processor and a memory, the memory storing computer instructions, and the processor being configured to execute the computer instructions stored in the memory, so that the system implements the steps of the method of the first aspect or the steps of the method of the second aspect.

[0029] The fourth aspect of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the method of the first aspect or the steps of the method of the second aspect.

[0030] The fifth aspect of the present application provides a computer program product, comprising computer instructions, which are executed by a processor to implement the steps of the method of the first aspect or the steps of the method of the second aspect.

[0031] The RDMA network transmission packet loss recovery method and system of the present application can provide an adaptive and low-overhead packet loss recovery scheme for the random packet loss problem in the lossy long-distance transmission process of the RDMA network, break the inherent situation that the original encoding bandwidth and the calculation overhead increase linearly with the increase of the transmission data volume, and organically combine the low-overhead advantage of selective retransmission and the fast recovery advantage of redundant encoding of the packet loss, so as to reduce the bandwidth resource overhead as much as possible while ensuring the reliability of the data stream transmission, and to ensure the long-distance reliable transmission with extremely low overhead, to meet the cross-domain transmission requirements of high throughput, low latency and anti-packet loss. And without actively sending probes to perceive the link state, the redundant encoding code rate is adaptively adjusted, further saving the bandwidth resources of long-distance transmission.

[0032] Additional advantages, objects, and features of the application will be set forth in part by the description that follows, and will become apparent to those skilled in the art upon examination of the following figures and detailed description thereof or can be learned by practice of the application. The objects and other advantages of the application can be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

[0033] Those skilled in the art will understand that the objects and advantages of the present application are not limited to the above specifically described, and the above and other objects that can be achieved by the present application will be more clearly understood according to the following detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0034] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the principles of the application.

[0035] Figure 1 Flowchart of the method for recovering lost packets in RDMA network transmission according to an embodiment of the present application;

[0036] Figure 2 Flowchart of the method for recovering lost packets in RDMA network transmission according to another embodiment of the present application;

[0037] Figure 3 Flowchart of the method for recovering lost packets in RDMA network transmission according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the embodiments and drawings. Here, the illustrative embodiments of the present application and their descriptions are used to explain the present application, but are not used to limit the present application.

[0039] It should be noted that, in order to avoid the present application being obscured by unnecessary details, only the structures and / or processing steps closely related to the solutions according to the present application are shown in the drawings, and other details not closely related to the present application are omitted.

[0040] It should be emphasized that the term "comprises / comprising" when used in this text means the presence of the stated features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0041] It should be noted that, if not specifically stated, the term "connected" in this text can not only mean direct connection, but also mean indirect connection with an intermediate object.

[0042] In the following, embodiments of the present application will be described with reference to the drawings. In the drawings, the same reference numerals represent the same or similar parts or the same or similar steps.

[0043] There are two existing solutions for the network packet loss recovery problem. One is the packet retransmission-based solution, and the other is the redundant coding-based packet protection solution. The basic idea of the packet retransmission-based solution is to detect packet loss by an intermediate switch or a receiving end, and feed back the packet loss to the sending end. The sending end retransmits the lost packets according to the packet loss feedback. Common retransmission algorithms include GBN (Go-Back-N) and SR (Selective Repeat). The basic idea of the redundant coding-based packet protection solution is that the sending end generates redundant packets through XOR calculation and other FEC (Forward Error Correction) coding methods when sending data. The receiving end recovers the lost packets through the received redundant packets when detecting packet loss.

[0044] The packet retransmission-based solution is the main packet loss recovery method at present, which can achieve reliable transmission without additional bandwidth overhead. In the RDMA network, GBN is used to ensure the reliability of data transmission. However, when the data packet is lost during transmission, the receiver does not request retransmission of the lost data packet alone, but the sender needs to retransmit all subsequent data packets starting from the lost data packet. This approach results in high bandwidth consumption and inefficient retransmission, especially in poor link quality or high delay network environments, where the performance of RDMA will be significantly reduced. Selective repeat only requests retransmission of the specific lost data packet from the receiver when the data packet is lost, rather than retransmitting all subsequent data packets together, thereby improving bandwidth efficiency and reducing unnecessary retransmission. The ConnectX-6D commercial network card of NVIDIA supports the selective repeat function to avoid the performance loss caused by GBN. IRN and SRNIC (a scalable architecture for RDMA network cards) implement selective repeat based on hardware. The receiving end maintains a packet bitmap to record the received data packet sequence, feeds back the SACK (Selective Acknowledgment) message to inform the sender of the unreceived data packet sequence, and re-counts the bitmap after receiving the retransmitted data packet, and delivers the complete data block to the upper layer application. One solution comprehensively implements selective repeat and reliability guarantee based on software on the basis of using commercial network card hardware, without modifying the network card hardware. Another solution further moves the packet loss feedback from the end side to the switch to implement it. The intermediate switch detects packet loss and generates corresponding packet loss feedback to quickly feed back to the sending end for packet retransmission recovery.

[0045] Although the existing packet selection retransmission-based packet loss recovery scheme does not need to pay extra transmission bandwidth, it cannot quickly feed back SACK in a long-distance transmission network environment, so that the receiving end needs to wait at least one RTT (Round-Trip Time) to successfully receive the retransmission packet after detecting the packet loss. Then, for small flows and delay-sensitive services, the long-distance transmission delay caused by this scheme greatly affects the flow transmission time, service completion time and application performance. In addition, if SACK or retransmission packet is lost, the waiting time will be further increased, thereby seriously affecting the network transmission performance.

[0046] The existing packet protection scheme based on redundancy coding needs to occupy extra data transmission bandwidth, but the receiving end does not need to wait for extra RTT for packet loss recovery. Among them, redundancy coding mostly uses FEC coding. FEC is a technology that adds redundant information at the sending end to improve data transmission reliability, allowing the receiving end to recover the original data even if part of the data packet is lost, thereby reducing the impact of network packet loss on network transmission quality. The core idea of FEC is to provide sufficient error correction capability through redundant information, so that even if the data packet is lost or damaged during transmission, the receiving end can correctly recover the original data. In WebRTC (Real-time Communication Technology), two FEC implementation methods are mainly used: ULPFEC and FlexFEC. ULPFEC protects the integrity of individual data packets by adding redundant information to the data packets, which is suitable for low packet loss network environment; FlexFEC is an extension of ULPFEC, which can correct multiple data packets. By combining multiple data packets into a coding block and adding redundant information, the error correction capability in high packet loss rate or unstable network conditions is improved. There are also schemes that add an intermediate layer between the RDMA transmission layer and the UDP / IP layer, increase redundant data packets through XOR calculation, and perform redundancy coding on all long-distance transmission data packets.

[0047] Although the existing packet protection and recovery scheme based on redundancy coding has certain advantages in data transmission delay, it needs to encode all long-distance transmission data packets, that is, whether the link state is good or bad, all data packets in the transmission data stream will be protected by redundancy, which occupies extra bandwidth resources, and in the same redundancy coding block, only the loss of one data packet can be recovered. If multiple data packets are lost in succession, the lost data packets cannot be recovered. This scheme must make a choice between bandwidth loss and packet protection level, and no matter how the coding ratio is adjusted, bandwidth loss will be caused, thereby wasting valuable long-distance transmission bandwidth resources.

[0048] Based on this, in order to solve the problem of random packet loss in the lossy long-distance transmission of the RDMA network, the embodiment of the present application proposes a RDMA network transmission packet loss recovery method and system which can adapt to the lossy long-distance transmission environment of the RDMA network. The target is to combine the low overhead advantage of selective repeat and the fast recovery advantage of redundant coding of lost packets, and to adapt to the lossy long-distance transmission network environment of RDMA, to ensure the transmission reliability of data against packet loss, while as much as possible to reduce the bandwidth overhead and shorten the stream transmission time and the corresponding service completion time. In the process of realizing low overhead, adaptive packet loss recovery in the lossy long-distance transmission of the RDMA network, the present application needs to overcome two challenges. Among them, challenge one is how to organically combine selective repeat and redundant coding and how to design the switching mechanism of the two, so as to give full play to the advantages of the two packet loss recovery schemes; challenge two is how to design an adaptive redundant coding algorithm, so as to adaptively adjust the coding ratio according to the state of different transmission links. In order to cope with the above two challenges, the present application deeply analyzes the shortcomings of the two packet loss recovery schemes of selective repeat and redundant coding and the root causes of the shortcomings, that is, the tail BDP (Bandwidth-Delay Product, bandwidth-delay product) part of the selective repeat data stream will be extremely affected by the packet loss in the transmission process. The stream transmission time and the corresponding service completion time, all data packets in the redundant coding data stream will waste the transmission link bandwidth resources. In view of this, the RDMA network transmission packet loss recovery method and system designed and proposed by the present application contains the tail BDP-based packet loss recovery mechanism based on the organic combination of the two packet loss recovery schemes of selective repeat and redundant coding and the adaptive coding mechanism, which can cope with the above two challenges and overcome the above shortcomings, so as to guarantee the reliable transmission of data in the lossy long-distance transmission environment of the RDMA network at the lowest cost.

[0049] Figure 1 and Figure 3 respectively, the flowchart and the specific flowchart of the RDMA network transmission packet loss recovery method in an embodiment of the present application. As shown in Figure 1 and Figure 3 , the execution subject of the method is the sending end, which includes the following steps:

[0050] Step S110, the sending end determines whether the to-be-transmitted data stream is a larger to-be-transmitted data stream or a smaller to-be-transmitted data stream based on the multiple of the bandwidth-delay product of the RDMA network, and the data stream includes a plurality of data packets.

[0051] The bandwidth-delay product of the RDMA network is an index for measuring the performance of the RDMA network, representing the product of the propagation delay and the bandwidth of the link, and the size of the delay bandwidth product of different RDMA networks may be different. In the implementation process, when the sending end issues a working request of a cross-domain transmission service data stream, the size of the BDP is calculated according to the maximum bandwidth allowed to be used by a single data stream in the RDMA network. First, the sending end compares the size of the to-be-transmitted data stream of the RDMA network with the multiple of the delay bandwidth product of the RDMA network to determine whether the to-be-transmitted data stream is a larger data stream or a smaller data stream. If the size of the to-be-transmitted data stream of the RDMA network is greater than the multiple of the delay bandwidth product of the RDMA network, it is determined that the to-be-transmitted data stream is a larger data stream (which can be referred to as a large stream). If the size of the to-be-transmitted data stream of the RDMA network is less than or equal to the multiple of the delay bandwidth product of the RDMA network, it is determined that the to-be-transmitted data stream is a smaller data stream (which can be referred to as a small stream).

[0052] In step S120, the smaller to-be-transmitted data stream is redundantly encoded and encoded marked to generate a first data stream, and the first data stream is transmitted to the receiving end. The first data stream is a data stream carrying encoded marks and redundant data packets.

[0053] Since the smaller data stream often has a higher requirement for transmission delay, the smaller data stream is transmitted in a redundant encoding manner and the lost data packets in the transmission process are recovered, so that the service of the smaller data stream can be completed as soon as possible to meet the delay requirement of the smaller data stream. In the implementation process, a part of data packets of all data packets in the smaller to-be-transmitted data stream are combined into an encoded data block, thereby obtaining a plurality of encoded data blocks. Then, the redundant data packets can be obtained by performing an exclusive OR operation on each encoded data block, so as to obtain a plurality of redundant data packets. The plurality of redundant data packets are added as redundant information in the original to-be-transmitted data stream, and the encoded mark information is added and encapsulated in the packet header of the data packet in the to-be-transmitted data stream to mark the to-be-transmitted data stream, so as to finally obtain a data stream carrying the encoded mark and the redundant data packet. The redundant information, the encoded mark and the original to-be-transmitted data stream are transmitted from the sending end to the receiving end. In the redundant encoding process of other embodiments, RS code or convolution code and other FEC encoding methods can be used instead of the exclusive OR operation to obtain the above-mentioned redundant information.

[0054] In step S130, the larger to-be-transmitted data stream is divided into a non-tail bandwidth-delay product part and a tail bandwidth-delay product part based on the multiple of the bandwidth-delay product, and the tail bandwidth-delay product part is redundantly encoded and encoded marked to generate a tail bandwidth-delay product part carrying encoded marks and redundant data packets.

[0055] Since the redundancy coding of all data packets in a large data stream will bring extra bandwidth overhead in both good and bad network link states, and the tail BDP part of the large data stream is recovered by the selective retransmission, which will greatly affect the flow transmission time and the corresponding service completion time, the method adopts the selective retransmission to recover the non-tail BDP part of the large data stream and the redundancy coding to quickly recover the tail BDP part of the large data stream, so as to reduce the bandwidth overhead and shorten the flow transmission time as much as possible. Specifically, the tail bandwidth delay product part (tail BDP part) of the large data stream to be transmitted refers to the part composed of multiple data packets with a multiple length of the last BDP at the tail of the large data stream to be transmitted, and the part composed of multiple data packets in front of the tail BDP part of the large data stream to be transmitted is the non-tail BDP part. The specific implementation process of the redundancy coding of the tail BDP part is similar to the way of the redundancy coding of the small data stream in step S120. Since the large data stream includes two blocks of the coding part (tail BDP part) and the non-coding part (non-tail BDP part), the tail BDP part and the non-tail BDP part of the large data stream need to be marked differently to distinguish the two blocks. Specifically, the coding identification information can be added and encapsulated in the packet header of the data packet in the tail BDP part, and then the tail BDP part carrying the coding identification / mark and the redundant data packet and the non-tail BDP part not carrying any identification / mark are obtained. The coding identification information can also be added and encapsulated in the packet header of the data packet in the tail BDP part, and the non-coding identification information can be added and encapsulated in the packet header of the data packet in the non-tail BDP part, and then the tail BDP part carrying the coding identification / mark and the redundant data packet and the non-tail BDP part carrying the non-coding identification / mark, i.e. the tail BDP part carrying the coding identification / mark and the redundant data packet and the non-tail BDP part not carrying the coding identification / mark, are obtained. The identification information is added and encapsulated by using the marking method agreed by the sending end and the receiving end, for example, the coding identification and the non-coding identification can be 1 and 0 respectively, that is, 1 represents redundancy coding and 0 represents selective retransmission; or only the coding identification is 1, that is, 1 represents redundancy coding and there is no identification representing selective retransmission.

[0056] In the method, by judging the data stream to be transmitted as a large stream or a small stream based on the multiple of BDP, and positioning the starting data packet of the tail BDP part in the large stream to divide the large stream into the tail BDP part and the non-tail BDP part, the problem that the delay jitter of the long-distance link in the RDMA network may cause the change of BDP and the application scenario can be better coped with. In the specific implementation process, the range of the multiple of BDP is set to 1-2 times (including the two endpoints 1 and 2).

[0057] Step S140, transmitting the second data stream formed by the non-tail bandwidth-delay product part and the tail bandwidth-delay product part carrying the encoding mark and the redundant data packet to the receiving end, so that the receiving end detects the received data stream for packet loss and encoding mark, if there is packet loss and there is an encoding mark, the lost data packet is recovered based on the carried redundant data packet, if there is packet loss and there is no encoding mark, the lost data packet is recovered by requesting selective repeat transmission from the sending end.

[0058] The sending end transmits the data stream formed by the tail BDP part carrying the encoding mark and the redundant data packet and the non-tail BDP part not carrying the encoding mark, i.e. the second data stream, to the receiving end. After the receiving end receives the smaller first data stream or the larger second data stream sent by the sending end, it first records the state of the received first or second data stream by updating the bitmap, and detects in real time whether there is packet loss in the received first or second data stream. When no packet loss is detected, i.e. no packet loss occurs, the receiving end directly outputs the received data stream or data packet sequence and performs subsequent corresponding operations on the received data stream according to specific requirements; when packet loss is detected, i.e. packet loss occurs, the receiving end further detects whether the lost data packet carries an encoding mark, if the lost data packet carries an encoding mark, the lost data packet is recovered using the received redundant data packet to ensure data integrity, if the lost data packet does not carry an encoding mark, a selective repeat packet loss recovery scheme is used to recover the lost data packet.

[0059] In some embodiments, the method further comprises the following steps:

[0060] Periodically counting the packet loss frequency, if the packet loss frequency is greater than a preset threshold, increasing the redundancy encoding ratio and reducing the size of the redundancy encoding block;

[0061] If the packet loss frequency is less than the preset threshold, the redundancy encoding ratio is reduced and the size of the redundancy encoding block is increased.

[0062] In actual transmission process, when the non-tail BDP part of the larger data stream loses packets, the receiving end feeds back SACK message to the sending end, and the sending end periodically counts the number of SACK messages received in each period, that is, the packet loss frequency. According to the random packet loss probability of the lossy long-distance network link, that is, the packet loss frequency, the proportion of redundant coding is dynamically and adaptively adjusted, and the size of the corresponding coded data block is calculated. If packet loss occurs frequently in a certain period (the packet loss frequency is greater than the preset threshold), it indicates that the network link transmission state is poor, and there are many interference factors, so more redundant packets are needed in the subsequent redundant coding of the tail BDP part, that is, the size of the coding data block in this period needs to be reduced, and the redundant coding proportion needs to be increased. Conversely, when packet loss is less in a certain period (the packet loss frequency is less than the preset threshold), it indicates that the network link state is good, so it is not necessary to have too many redundant packets, the redundant coding proportion can be reduced, the size of the redundant coding block can be increased, and the bandwidth resource overhead of data protection can be reduced. For example, when one data packet is lost for every 8 data packets transmitted, redundant coding is performed once for every 8 data packets, so the 8 data packets are taken as a coding data block, and the redundant coding proportion is 1 / 8; when 2-3 or even more data packets are lost for every 8 data packets transmitted, the size of the coding data block needs to be reduced, and the redundant coding proportion needs to be increased. The coding proportion can be set to 1 / 4. The adaptive coding mechanism can dynamically adjust the coding proportion according to the state of the network link, reduce the additional bandwidth resource overhead as much as possible, ensure accurate packet loss recovery, and improve the anti-packet loss capability under different network link states.

[0063] In some embodiments, before transmitting the first data stream to the receiving end, the method further comprises the following steps:

[0064] The first data stream is preferentially transmitted by increasing the transmission priority of the first data stream.

[0065] Another object of the method is to optimize the transmission efficiency of the smaller data stream and reduce the flow completion time of the small stream, thereby improving the overall network performance. Small streams are often more sensitive to latency, so special processing is needed in long-distance transmission environment to avoid additional latency overhead caused by redundant coding or flow control mechanism. To this end, first, the sending end marks the smaller data stream in real time, and divides it into high-priority streams, which are processed separately from large streams. For these high-priority small streams, the transmission of small streams is preferentially arranged in the data forwarding queue to ensure that small streams can preferentially obtain bandwidth resources and avoid delaying transmission due to excessive queuing time, thereby shortening the flow completion time of small streams. While ensuring the fast transmission of small streams, the adverse effects of redundant coding on the transmission performance of small streams are also minimized, providing more efficient support for latency-sensitive long-distance transmission services.

[0066] Figure 2 FIG. 2 is a flowchart illustrating a method for recovering lost packets in an RDMA network transmission according to another embodiment of the present application. As shown in FIG. 2, the method is executed by a receiving end and includes the following steps: Figure 2 and Figure 3 As shown in FIG. 2, the method is executed by a receiving end and includes the following steps:

[0067] In step S210, the receiving end receives a first data stream or a second data stream from a sending end. The first data stream is a data stream carrying encoding marks and redundant data packets obtained by performing redundancy encoding and encoding marking on a smaller to-be-transmitted data stream determined based on a multiple of a bandwidth-delay product of the RDMA network. The second data stream is formed by a non-tail bandwidth-delay product part and a tail bandwidth-delay product part carrying encoding marks and redundant data packets. The non-tail bandwidth-delay product part and the tail bandwidth-delay product part carrying encoding marks and redundant data packets are obtained by dividing a larger to-be-transmitted data stream determined based on the multiple of the bandwidth-delay product into the non-tail bandwidth-delay product part and the tail bandwidth-delay product part based on the multiple of the bandwidth-delay product and performing redundancy encoding and encoding marking on the tail bandwidth-delay product part.

[0068] In step S220, the received data stream is subjected to packet loss and encoding mark detection.

[0069] In step S230, if there is packet loss and encoding marks, the lost data packets in the received data stream are recovered based on the carried redundant data packets.

[0070] In step S240, if there is packet loss and no encoding marks, the lost data packets in the received data stream are recovered by requesting selective repeat transmission from the sending end.

[0071] In some embodiments, in the case of continuous packet loss during transmission of the first data stream, the lost data packets in the received data stream are recovered by requesting selective repeat transmission from the sending end.

[0072] In some embodiments, the lost data packets in the received data stream are recovered by requesting selective repeat transmission from the sending end, including the following steps:

[0073] The sending end is fed back a selective acknowledgement message for each lost data packet, so that the sending end retransmits the lost data packet to the receiving end based on the received selective acknowledgement message.

[0074] When consecutive packet loss occurs in the data stream, the redundant coding packet loss recovery scheme, i.e., redundant data packet, cannot recover the consecutive lost data packet, and the receiving end immediately adopts the selective retransmission packet loss recovery scheme to recover the consecutive lost data packet, sends SACK feedback to the sending end to request retransmission of the lost data packet, and realizes switching of the redundant coding and selective retransmission two packet loss recovery schemes. In addition, the decoding recovery result is also fed back to the sending end to optimize the coding strategy.

[0075] In some embodiments, in the step of feeding back the selective acknowledgement message to the sending end based on each lost data packet, the cumulative selective acknowledgement mode is adopted for feedback. That is, each time the feedback is fed back, the data packet sequence that should be received but is not actually received in the bitmap is fed back, which can avoid the situation that the corresponding lost data packet cannot be retransmitted due to loss of SACK in the feedback process, and further cannot be recovered.

[0076] In summary, the RDMA network transmission packet loss recovery method of the embodiment of the present application can provide an adaptive and low-overhead packet loss recovery scheme for the random packet loss problem in the RDMA network lossy long-distance transmission process, break the inherent situation that the original coding bandwidth and computing overhead linearly increase with the increase of the transmission data amount, organically combine the low-overhead advantage of selective retransmission and the fast packet loss recovery advantage of redundant coding, guarantee the reliability of data stream transmission while reducing the bandwidth resource overhead as much as possible, guarantee the long-distance reliable transmission with extremely low overhead, and meet the cross-domain transmission demand of high throughput, low latency and anti-packet loss. Moreover, without actively sending a probe to perceive the link state, the redundant coding rate is adaptively adjusted, and the long-distance transmission bandwidth resource is further saved.

[0077] Corresponding to the above method, the present application also provides an RDMA network transmission packet loss recovery system, which comprises a computer device, the computer device comprising a processor and a memory, the memory storing computer instructions, and the processor being configured to execute the computer instructions stored in the memory, so that the system implements the steps of the foregoing method when the computer instructions are executed by the processor.

[0078] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by the processor to implement the steps of the foregoing method. The computer readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0079] The embodiment of the present application further provides a computer program product comprising computer instructions which, when executed by a processor, implement the steps of the aforementioned method.

[0080] Those skilled in the art should understand that the exemplary components, systems and methods described in connection with the embodiments disclosed herein can be implemented in hardware, software or a combination thereof. The exact implementation depends on the specific application and design constraints imposed on the overall system. Skilled persons can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the present application are program or code segments used to perform the required tasks. The program or code segments can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave over a transmission medium or communication link.

[0081] It should be noted that the present application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of known methods are omitted. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order of the steps, after understanding the spirit of the present application.

[0082] In the present application, the features described and / or illustrated for one embodiment can be used in the same way or in a similar way in one or more other embodiments, and / or in combination with or instead of features of other embodiments.

[0083] The above description is only preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the embodiments of the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for recovering lost packets in an RDMA network transmission, the method comprising: The method comprises: determining, by the sending end, a data stream to be transmitted as a larger data stream to be transmitted or a smaller data stream to be transmitted based on a multiple of a bandwidth-delay product of an RDMA network, the data stream comprising a plurality of data packets; the larger data stream to be transmitted having a size greater than the multiple of the bandwidth-delay product, and the smaller data stream to be transmitted having a size less than or equal to the multiple of the bandwidth-delay product; performing redundancy encoding and encoding marking on the smaller data stream to be transmitted to generate a first data stream, and transmitting the first data stream to the receiving end, the first data stream being a data stream carrying encoding markers and redundancy data packets; dividing the larger data stream to be transmitted into a non-tail bandwidth-delay product part and a tail bandwidth-delay product part based on the multiple of the bandwidth-delay product, performing redundancy encoding and encoding marking on the tail bandwidth-delay product part to generate a tail bandwidth-delay product part carrying encoding markers and redundancy data packets; transmitting a second data stream formed by the non-tail bandwidth-delay product part and the tail bandwidth-delay product part carrying encoding markers and redundancy data packets to the receiving end, so that the receiving end performs packet loss and encoding marker detection on the received data stream, recovers lost data packets based on the carried redundancy data packets if there is packet loss and encoding markers, and requests selective repeat transmission from the sending end to recover lost data packets if there is packet loss and no encoding markers.

2. The method of claim 1, wherein, The method further comprises: periodically counting packet loss frequency, and increasing redundancy encoding proportion and reducing size of redundancy encoding block if the packet loss frequency is greater than a preset threshold; decreasing redundancy encoding proportion and increasing size of redundancy encoding block if the packet loss frequency is less than the preset threshold.

3. The method of claim 1, wherein, Before transmitting the first data stream to the receiving end, the method further comprises: transmitting the first data stream preferentially by increasing transmission priority of the first data stream; determining, by the sending end, a data stream to be transmitted as a larger data stream to be transmitted or a smaller data stream to be transmitted based on a multiple of a bandwidth-delay product of an RDMA network, comprising: comparing, by the sending end, size of the data stream to be transmitted with the multiple of the bandwidth-delay product of the RDMA network, and if the size of the data stream to be transmitted is greater than the multiple of the bandwidth-delay product, the data stream to be transmitted is the larger data stream to be transmitted; if the size of the data stream to be transmitted is less than or equal to the multiple of the bandwidth-delay product, the data stream to be transmitted is the smaller data stream to be transmitted.

4. A method for recovering lost packets in an RDMA network transmission, the method comprising: The method comprises: The first data stream or the second data stream is received from the sending end by the receiving end, the first data stream is a data stream carrying encoding marks and redundant data packets obtained by performing redundant encoding and encoding marking on a smaller to-be-transmitted data stream determined based on a multiple of a bandwidth-delay product of an RDMA network, and the second data stream is formed by a non-tail bandwidth-delay product part and a tail bandwidth-delay product part carrying encoding marks and redundant data packets, wherein the non-tail bandwidth-delay product part and the tail bandwidth-delay product part carrying encoding marks and redundant data packets are obtained by dividing a larger to-be-transmitted data stream determined based on the multiple of the bandwidth-delay product into the non-tail bandwidth-delay product part and the tail bandwidth-delay product part based on the multiple of the bandwidth-delay product and performing redundant encoding and encoding marking on the tail bandwidth-delay product part; the size of the larger to-be-transmitted data stream is greater than the multiple of the bandwidth-delay product, and the size of the smaller to-be-transmitted data stream is less than or equal to the multiple of the bandwidth-delay product; Packet loss and encoding mark detection are performed on the received data stream; If there is packet loss and encoding marks, the lost data packets in the received data stream are recovered based on the carried redundant data packets; If there is packet loss and no encoding marks, the lost data packets in the received data stream are recovered by requesting selective repeat transmission from the sending end.

5. The method of claim 4, wherein, In the case of continuous packet loss in the tail bandwidth-delay product part carrying encoding marks and redundant data packets in the second data stream and the first data stream transmission process, the lost data packets in the received data stream are recovered by requesting selective repeat transmission from the sending end.

6. The method according to claim 4 or 5, characterized in that, The lost data packets in the received data stream are recovered by requesting selective repeat transmission from the sending end, including: Selective acknowledgement messages are fed back to the sending end based on each lost data packet, so that the sending end retransmits the lost data packets to the receiving end based on the received selective acknowledgement messages.

7. The method of claim 6, wherein, In the step of feeding back selective acknowledgement messages to the sending end based on each lost data packet, the cumulative selective acknowledgement mode is used for feedback.

8. An RDMA network transmission packet loss recovery system comprising a processor, a memory, and computer instructions stored on the memory, wherein, The processor is configured to execute the computer instructions, and when the computer instructions are executed, the system implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor, and when the computer program is executed, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising computer instructions, characterized in that, The computer instructions are executed by the processor, and when the computer instructions are executed, the steps of the method according to any one of claims 1 to 7 are implemented.

Images