Wide area network data transmission enhancement method and system

Abstract

The invention belongs to the technical field of wide area network data transmission, and provides a wide area network data transmission enhancement method and system.The wide area network data transmission enhancement method comprises the steps that at a sending end, a coding controller conducts coding processing on an identified RDMA message, and then an original data packet and a repair packet are together delivered to a lower-layer UDP protocol stack to be packaged and transmitted; when ACK / NACK is received, the collection module extracts packet loss statistical information carried by the message and delivers the packet loss statistical information to the coding controller; at a receiving end, a transmitted data packet is redirected to a decoding buffer area, and after decoding is completed through a decoding controller, a message is transmitted to an upper-layer RDMA protocol stack; and meanwhile, the receiving end redirects the ACK / NACK information to a collection module, and the module adds packet loss statistical information and then delivers the ACK / NACK information to a lower-layer protocol stack for transmission. According to the invention, by introducing a feedback collection mechanism and a self-adaptive coding algorithm in which the receiving end and the transmitting end cooperate, the dynamic perception of the network packet loss state is realized, so that the self-adaptive adjustment of the coding level is realized, and the transmission capability in a lossy WAN network environment is improved.

Description

Technical Field

[0001] This invention belongs to the field of wide area network (WAN) data transmission technology, and specifically relates to a WAN data transmission enhancement method and system. Background Technology

[0002] Remote Direct Memory Access (RDMA) is a popular high-speed networking technology widely used in data center networks (DCNs). By applying kernel bypass technology to offload the network protocol stack to the network interface card (NIC), RDMA achieves high throughput, ultra-low latency, and minimized CPU overhead. RDMA over Converged Ethernet (RoCE) is now widely used in large data centers and has become the de facto standard for DCNs. Furthermore, deploying applications across geographically distributed data centers is becoming an emerging trend. Large companies such as Microsoft Azure, Google, Alibaba, and OpenAI have already deployed geographically distributed large-scale model training, cloud computing, and cloud storage services in multiple regional data centers. In summary, given the widespread application of RDMA in traditional DCNs and the trend of deploying applications across long-distance distributed data centers, extending RDMA to wide area networks (WANs) has become a new hot topic of discussion.

[0003] However, deploying RDMA technology directly in WAN environments faces fundamental challenges. The prevalent non-congestion packet loss and long RTT characteristics in WANs inherently conflict with the core design assumption of RDMA's "lossless underlying network." This contradiction stems from RDMA's pursuit of hardware offloading efficiency, employing a Go-Back-N (GBN) style retransmission mechanism—which triggers the sender to retransmit all subsequent data after the lost packet upon detecting packet loss. In long RTT WAN scenarios, this retransmission method results in a bandwidth waste of up to one bandwidth-time product for each packet loss, leading to significant degradation in protocol performance. Experiments show that in a 10Gbps link with a 40ms RTT test scenario, a single connection can saturate the link with an effective throughput of 9.21Gbps under lossless conditions, but the effective throughput drops sharply to 2.92Gbps at a packet loss rate of 0.001%, and almost decays to zero when the packet loss rate exceeds 0.1%. Therefore, designing an RDMA transmission enhancement mechanism that can adapt to lossy WAN environments has become a core challenge and key direction in the current research field of wide-area RDMA.

[0004] Currently, the industry has proposed three main mechanisms for enhancing RDMA transmission in lossy WAN environments: architectural design schemes, scheduling optimization schemes, and error correction coding schemes. While all three methods have achieved some success in theory and practice, they still have shortcomings in real-world dynamic WAN scenarios. 1. The proposed architecture design aims to redesign the RDMA transmission architecture to enable lossless transmission in a lossy WAN environment. Its core lies in introducing a Selective Retransmission (SR) mechanism. This mechanism requires efficient storage and querying of packet reception or loss status, with bitmap tracking being a common method. However, this approach suffers from three significant problems: First, high storage overhead. RNIC SRAM resources are limited, while context management requires substantial storage space, making direct bitmap tracking inefficient, and resource consumption far exceeds expectations in high-concurrency scenarios. Second, high query latency. Diverse packet loss scenarios in WANs result in non-sequential access to lost contexts, frequently triggering interactions between the RNIC and hosts, leading to variable retrieval costs that increase with application scale. Third, high deployment costs. While existing optimization solutions attempt to alleviate these issues, they typically require customized architectural adjustments, are incompatible with existing RDMA protocol stacks, rely on dedicated hardware, and significantly increase deployment difficulty.

[0005] 2. Scheduling optimization schemes address packet loss issues in long-haul RDMA from a congestion control perspective. Typical approaches include extending priority-based flow control (PFC) mechanisms and introducing long-haul traffic identification methods. These schemes effectively coordinate the processing strategies for cross-datacenter traffic and intra-datacenter traffic by specifically detecting and differentially scheduling cross-datacenter traffic, thereby improving transmission efficiency. However, these schemes have two limitations: first, they fail to specifically distinguish between lossy link packet loss and congestion-induced packet loss, limiting their performance ceiling in real-world lossy environments; second, recovery latency is relatively high. Due to the reliance on end-to-end feedback mechanisms, processing packet loss events requires at least one round-trip time (RTT), making rapid response difficult.

[0006] 3. Error correction coding schemes encode data packets at the sending end and generate redundant packets, enabling the receiving end to recover lost data packets using this redundant information. This protects RDMA messages from packet loss and avoids long delays introduced by retransmission. However, existing schemes also have significant shortcomings: the coding levels of existing methods are usually fixed and rely on external input for updates, making it impossible to dynamically adjust parameters according to real-time network packet loss conditions. This results in difficulties in efficiently utilizing scarce bandwidth resources in dynamically changing wide area network (WAN) environments.

[0007] In summary, existing technologies suffer from drawbacks such as high deployment costs, poor applicability, long feedback times, low efficiency, and lack of dynamic adjustment. To address these issues, this invention proposes a method and system for enhancing wide area network data transmission, aiming to achieve low-overhead, link-adaptive, and easy-to-deploy transmission enhancement. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention provides a method and system for enhancing wide area network data transmission, thereby resolving the issues in the prior art. The technical solution adopted by this invention is as follows: A method for enhancing wide area network (WAN) data transmission at the sending end includes: The encoding controller at the sending end identifies the raw data packet of the RDMA message and routes it to the encoding buffer. Based on the dynamic encoding level information, the encoding controller performs forward error correction encoding on the RDMA message in the encoding buffer to generate a repair packet. The raw data packet and the repair packet are transmitted together to the lower-layer protocol stack for encapsulation and transmission, and finally transmitted to the receiving end.

[0009] Furthermore, when the sending end receives an ACK / NACK response message from the receiving end, the sending end's ACK / NACK feedback module intercepts and parses it, extracts the packet loss statistics information in the ACK / NACK response message, and then periodically updates the packet loss rate estimate based on the round-trip delay window. After the update, it is transmitted to the encoding controller as dynamic encoding level information to adjust the encoding level for the next transmission.

[0010] Furthermore, after extracting packet loss statistics, the ACK / NACK feedback module transmits the original ACK / NACK response message to the upper-layer protocol stack.

[0011] A method for enhancing wide area network (WAN) data transmission at the receiving end includes: The decoding buffer at the receiving end receives data packets from the sending end; the decoding controller at the receiving end classifies the data packets into the corresponding decoding blocks based on the header information of the data packets. When the number of data packets in any decoding block reaches the decoding threshold, the decoding operation is immediately performed, and the recovered original data packets are then submitted to the upper-layer protocol stack. At the same time, the decoding controller performs packet loss counting statistics. The ACK / NACK collection module of the receiving end obtains the current packet loss statistics from the decoding controller and writes them into the ACK / NACK response message. The enhanced ACK / NACK response message is delivered to the lower-layer protocol stack, further encapsulated, and then sent to the sending end.

[0012] A method for enhancing wide area network data transmission includes a transmitter and a receiver, wherein: The sending end's encoding buffer receives the raw data packets of the RDMA message; the encoding controller performs forward error correction encoding on the RDMA message in the encoding buffer according to the dynamic encoding level information to generate a repair packet; the raw data packet and the repair packet are jointly transmitted to the lower-layer protocol stack for encapsulation and transmission, and finally transmitted to the receiving end; when the sending end receives an ACK / NACK response message from the receiving end, the sending end's ACK / NACK feedback module extracts the packet loss statistics information in the ACK / NACK response message, and then periodically updates the packet loss rate estimate based on the round-trip delay window, and transmits the updated packet loss rate estimate to the encoding controller as dynamic encoding level information to adjust the encoding level for the next transmission; The receiving end receives data packets from the sending end; the decoding controller at the receiving end classifies the data packets into the corresponding decoding blocks based on the header information; when the number of data packets in any decoding block reaches the decoding threshold, the decoding operation is immediately performed, and then the recovered original data packets are submitted to the upper-layer protocol stack; at the same time, the decoding controller performs packet loss counting statistics, the ACK / NACK collection module at the receiving end obtains the current packet loss statistics from the decoding controller and writes them into the ACK / NACK response message, and the enhanced ACK / NACK response message is delivered to the lower-layer protocol stack and sent to the ACK / NACK feedback module at the sending end.

[0013] A transmitter encoding algorithm, comprising: Step 1: Initialize the output data packet sequence P to be empty, and initialize the block-based data packet set Blocks to be empty; set the initial index value i=0; Step 2, repeat the following operations until index i has traversed the original data packet set M (i < |M|): Starting from index i, take min(R,|M|-i) data packets (R is the number of original data packets required to encode a single coding block) to form a data block; Add the data block to the block set Blocks; Increment index i by R, preparing for the next block division; Step 3, iterate through each data block in the block set Blocks: Calculate the number of data packets k in the current data block; If k=R: Encode all data packets in this data block, generate C repair packets and add encoded headers; If k≠R: Encode the k data packets in the data block, generate a repair packet and add an encoded header; Add the original data packet and the generated repair packet together to the output sequence P; Finally, the output sequence P is returned.

[0014] A packet loss rate statistics update algorithm for the sender includes: Step 1, when an ACK message is received: The number of lost packets carried by the ACK, LostPkts, is added to the total number of packets lost in the period, LastPeriodLostPkts. Step 2, determine if the current ACK sequence number AckSeq is greater than the last updated sequence number LastSeq: If AckSeq is greater than LastSeq, and the total number of data packets sent in the previous period LastPeriodSentPkts is not equal to 0, meaning this period is not the first period, then proceed as follows: The periodic packet loss rate is calculated as follows: PeriodLossRate = Total number of packets lost in the period LastPeriodLostPkts / Total number of packets sent in the previous period LastPeriodSentPkts. Update the total packet loss rate using an exponentially weighted moving average: LossRate = LossRate * (1 - EWMARATE) + Period LossRate * EWMARATE; The coding level will be updated based on the updated total packet loss rate. The total number of packets lost during the update period is 0 (LastPeriodLostPkts). If AckSeq is greater than LastSeq, update the total number of data packets sent in the previous period LastPeriodSentPkts to the number of packets sent in the current period CurPeriodSentPkts, set the number of packets sent in the current period to 0, and update the last update sequence number LastSeq to the current sending sequence number SNDNXT. Step 3, when a NACK message is received: First, filter out duplicate NACKs; Upgrade the coding level; Update packet loss rate based on new coding level; Set the total number of data packets sent in the previous period (LastPeriodSentPkts), the number of packets sent in the current period (CurPeriodSentPkts), and the total number of packets lost in the period (LastPeriodLostPkts) to 0. Update the last update sequence number (LastSeq) to the current transmission sequence number (SNDNXT) and update the NACK deduplication flag.

[0015] A wide area network data transmission enhancement system includes a transmitter, a receiver, an upper-layer protocol stack, and a lower-layer protocol stack, wherein: the transmitter includes an encoding buffer, an encoding controller, and an ACK / NACK feedback module; the receiver includes a decoding buffer, a decoding controller, and an ACK / NACK collection module; The encoding controller is used to identify the raw data packets of the RDMA message and route them to the encoding buffer. The encoding controller is used to perform forward error correction encoding on the RDMA message in the encoding buffer according to the dynamic encoding level information, generate a repair packet, and transmit the raw data packet and the repair packet together to the lower-layer protocol stack for encapsulation and transmission, and finally transmit them to the receiving end. The ACK / NACK feedback module is used to receive ACK / NACK response messages, extract packet loss statistics from the ACK / NACK response messages, and then periodically update the packet loss rate estimate based on the round-trip delay window. After the update, it is transmitted to the encoding controller as dynamic encoding level information to adjust the encoding level for the next transmission. The decoding controller at the receiving end is used to receive data packets from the lower-layer protocol stack and classify them into the corresponding decoding blocks according to the header information of the data packets. When the number of data packets in any decoding block reaches the decoding threshold, the decoding operation is immediately performed, and then the decoded and repaired original data packets are submitted to the upper-layer protocol stack. At the same time, the decoding controller performs packet loss counting statistics. The ACK / NACK collection module is used to obtain the current statistics from the decoding controller and write them into the ACK / NACK response message. The ACK / NACK response message is delivered to the lower layer protocol stack and sent to the ACK / NACK feedback module of the sending end.

[0016] A data transmission system includes two terminal devices that interact via a wide area network. One of the terminal devices is configured as the transmitting end as described in any one of claims 1-8; Another terminal device is configured as the receiving end as described in any one of claims 1-8.

[0017] A data center employs the aforementioned wide area network data transmission enhancement system.

[0018] The present invention has the following beneficial effects: (1) The present invention realizes adaptive adjustment of coding level: In view of the problem that the coding level of the existing forward error correction coding scheme is fixed and cannot respond to network changes in real time, it is easy to lead to bandwidth waste in low packet loss scenarios or error correction failure in high packet loss scenarios. The present invention introduces a closed-loop feedback collection mechanism and adaptive coding algorithm in collaboration between the transmitting and receiving ends to realize dynamic perception of network packet loss status, thereby autonomously adjusting the coding strategy according to the actual link conditions, taking into account both transmission reliability and bandwidth utilization.

[0019] (2) This invention maintains architectural compatibility and deployment flexibility: Existing architectural enhancement schemes often require customized modifications to the RDMA protocol stack, making it difficult to be compatible with standard devices. This invention implements the encoding and decoding enhancement function in the form of an independent middleware. Without modifying the semantics of the core RDMA protocol, the closed-loop feedback mechanism is embedded in the encoding and decoding additional layer, making it run as an independent module, thereby ensuring compatibility with existing infrastructure based on the RoCEv2 protocol and supporting seamless deployment and smooth upgrades.

[0020] (3) The present invention reduces system resource overhead and is easy to expand and deploy on a large scale: Through a lightweight packet loss statistics method and a low-complexity packet loss rate update strategy, the present invention only needs to maintain extremely simple flow state information, avoids complex optimization problems, significantly reduces the occupation of computing and storage resources, has good scalability, and is suitable for large-scale network environments.

[0021] (4) This invention improves system performance: Existing architecture designs and scheduling optimization schemes that rely on end-to-end feedback require at least one round-trip time (RTT) to complete the state response in a long-latency wide area network environment, which seriously affects system efficiency and is particularly detrimental to short-stream transmission performance. This invention generates redundant packets through forward error correction coding, achieving zero round-trip time recovery, effectively avoiding the constraint of long RTT on transmission efficiency, and significantly improving network throughput. Attached Figure Description

[0022] Figure 1 This is an overall framework diagram of the present invention; Figure 2 Example diagram of adaptive forward error correction coding transmission enhancement mechanism; Figure 3 This is the pseudocode of the transmitting end encoding algorithm of this invention; Figure 4 This is the pseudocode for the packet loss rate statistics update algorithm at the sending end of this invention. Detailed Implementation

[0023] The following will be described in conjunction with embodiments of the present invention. Figures 1-4 The technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Unless otherwise specified, the technical means used in the embodiments are conventional means well known to those skilled in the art.

[0024] This invention proposes a method and system for enhancing wide area network (WAN) data transmission. At the sending end, the encoding controller identifies RDMA messages and routes them to the encoding buffer for FEC encoding. The encoding level is determined by the controller. After encoding, the original data packet and the repair packet are delivered to the lower-layer UDP protocol stack for encapsulation and transmission. When an ACK / NACK is received, the collection module extracts the packet loss statistics carried in the message and delivers them to the encoding controller. The controller updates the encoding level based on the RTT window. At the receiving end, the arriving data packets are redirected to the decoding buffer. After decoding by the decoding controller, the message is transmitted to the upper-layer RDMA protocol stack for lossless transmission. Simultaneously, the receiving end redirects the ACK / NACK information in the RDMA message queue to the collection module. This module appends packet loss statistics and delivers them to the lower-layer protocol stack for transmission. The packet loss statistics are generated by a lightweight packet loss statistics algorithm. This invention, by introducing a collaborative feedback collection mechanism between the sending and receiving ends and an adaptive encoding algorithm, achieves dynamic perception of network packet loss status, thereby enabling adaptive adjustment of the encoding level and improving transmission capabilities in lossy WAN environments. This invention is applicable to wide area network (WAN) communication scenarios with high requirements for transmission reliability, link utilization, and deployment flexibility under dynamic packet loss conditions, including but not limited to: cross-regional data center interconnection, distributed cloud computing platforms, high-performance computing cluster interconnection, and financial-grade cross-regional transaction systems. The network scale can cover heterogeneous topology environments ranging from a few nodes to thousands of nodes; link packet loss modes include random independent packet loss and bursty concentrated packet loss; transmission distance can be extended to intercontinental long-distance networks, and it has adaptive capabilities to dynamic changes in packet loss rate.

[0025] Specifically, this invention proposes a method for enhancing wide area network data transmission, comprising a transmitter and a receiver, wherein: The sending end's encoding buffer receives the raw data packets of the RDMA message; the encoding controller performs forward error correction encoding on the RDMA message in the encoding buffer according to the dynamic encoding level information to generate a repair packet; the raw data packet and the repair packet are jointly transmitted to the lower-layer protocol stack for encapsulation and transmission, and finally transmitted to the receiving end; when the sending end receives an ACK / NACK response message from the receiving end, the sending end's ACK / NACK feedback module extracts the packet loss statistics information in the ACK / NACK response message, and then periodically updates the packet loss rate estimate based on the round-trip delay window, and transmits the updated packet loss rate estimate to the encoding controller as dynamic encoding level information to adjust the encoding level for the next transmission.

[0026] The encoding controller at the transmitting end of this invention is responsible for identifying the raw data packets of RDMA messages from the upper-layer transport protocol stack and routing them to the encoding buffer. Based on its maintained dynamic encoding level information, the controller performs forward error correction (FEC) encoding on the messages in the buffer. After encoding, the system delivers the raw data packets and the generated repair packets to the lower-layer UDP protocol stack for encapsulation and transmission. When an ACK / NACK response message is received from the receiving end, the ACK / NACK feedback module intercepts and parses it. This module extracts the packet loss statistics carried in the message and periodically updates the packet loss rate estimate based on the round-trip time (RTT) window. The updated result is passed to the encoding controller as dynamic encoding level information to adjust the encoding level for the next transmission. After information extraction, the original ACK / NACK message is submitted to the upper-layer protocol stack for routine processing.

[0027] The receiving end receives data packets from the sending end; the decoding controller at the receiving end classifies the data packets into the corresponding decoding blocks based on the header information; when the number of data packets in any decoding block reaches the decoding threshold, the decoding operation is immediately performed, and then the recovered original data packets are submitted to the upper-layer protocol stack; at the same time, the decoding controller performs packet loss counting statistics, the ACK / NACK collection module at the receiving end obtains the current packet loss statistics from the decoding controller and writes them into the ACK / NACK response message, the enhanced ACK / NACK response message is delivered to the lower-layer protocol stack, and sent to the ACK / NACK feedback module at the sending end.

[0028] In this invention, data packets provided by the lower-layer protocol stack at the receiving end are redirected to the decoding buffer. The decoding controller categorizes the data packets into the corresponding decoding blocks based on the packet header information. When the number of data packets in any decoding block reaches the decoding threshold, a decoding operation is immediately performed to recover the original data packets, which are then submitted to the upper-layer protocol stack. Simultaneously, the decoding controller dynamically maintains a lightweight packet loss count statistic to track packet reception for each encoding block. When an ACK / NACK response message is required, the ACK / NACK collection module obtains the current statistics from the decoding controller and fills them into the response message. Finally, this enhanced response message is delivered to the lower-layer protocol stack and sent back to the sending end.

[0029] Based on the wide area network data transmission enhancement method of the present invention, the present invention also relates to a transmitter encoding algorithm, the pseudocode of which is as follows: Figure 3 ,include: Step 1: Initialize the output data packet sequence P to be empty, and initialize the block-based data packet set Blocks to be empty; set the initial index value i=0; Step 2, repeat the following operations until index i has traversed the original data packet set M (i < |M|): Starting from index i, take min(R,|M|-i) data packets (R is the number of original data packets required to encode a single coding block) to form a data block; Add the data block to the block set Blocks; Increment index i by R, preparing for the next block division; Step 3, iterate through each data block in the block set Blocks: Calculate the number of data packets k in the current data block; If k=R: Encode all data packets in this data block, generate C repair packets and add encoded headers; If k≠R: Encode the k data packets in the data block, generate a repair packet and add an encoded header; Add the original data packet and the generated repair packet together to the output sequence P; Finally, the output sequence P is returned.

[0030] 7. A packet loss rate statistical update algorithm for the sending end, characterized in that it is based on the method for enhancing wide area network data transmission at the sending end according to any one of claims 1-3, comprising: Step 1, when an ACK message is received: The number of lost packets carried by the ACK, LostPkts, is added to the total number of packets lost in the period, LastPeriodLostPkts. Step 2, determine if the current ACK sequence number AckSeq is greater than the last updated sequence number LastSeq: If AckSeq is greater than LastSeq, and the total number of data packets sent in the previous period LastPeriodSentPkts is not equal to 0, meaning this period is not the first period, then proceed as follows: The periodic packet loss rate is calculated as follows: PeriodLossRate = Total number of packets lost in the period LastPeriodLostPkts / Total number of packets sent in the previous period LastPeriodSentPkts. Update the total packet loss rate using an exponentially weighted moving average: LossRate = LossRate * (1 - EWMARATE) + Period LossRate * EWMARATE; The coding level will be updated based on the updated total packet loss rate. The total number of packets lost during the update period is 0 (LastPeriodLostPkts). If AckSeq is greater than LastSeq, update the total number of data packets sent in the previous period LastPeriodSentPkts to the number of packets sent in the current period CurPeriodSentPkts, set the number of packets sent in the current period to 0, and update the last update sequence number LastSeq to the current sending sequence number SNDNXT. Step 3, when a NACK message is received: First, filter out duplicate NACKs; Upgrade the coding level; Update packet loss rate based on new coding level; Set the total number of data packets sent in the previous period (LastPeriodSentPkts), the number of packets sent in the current period (CurPeriodSentPkts), and the total number of packets lost in the period (LastPeriodLostPkts) to 0. Update the last update sequence number (LastSeq) to the current transmission sequence number (SNDNXT) and update the NACK deduplication flag.

[0031] like Figure 1 The present invention also relates to a wide area network data transmission enhancement system, comprising a transmitter, a receiver, an upper-layer protocol stack, and a lower-layer protocol stack, wherein: the transmitter includes an encoding buffer, an encoding controller, and an ACK / NACK feedback module; the receiver includes a decoding buffer, a decoding controller, and an ACK / NACK collection module; The encoding controller is used to identify the raw data packets of the RDMA message and route them to the encoding buffer. The encoding controller is used to perform forward error correction encoding on the RDMA message in the encoding buffer according to the dynamic encoding level information, generate a repair packet, and transmit the raw data packet and the repair packet together to the lower-layer protocol stack for encapsulation and transmission, and finally transmit them to the receiving end. The ACK / NACK feedback module is used to receive ACK / NACK response messages, extract packet loss statistics from the ACK / NACK response messages, and then periodically update the packet loss rate estimate based on the round-trip delay window. After the update, it is transmitted to the encoding controller as dynamic encoding level information to adjust the encoding level for the next transmission. The decoding controller at the receiving end is used to receive data packets from the lower-layer protocol stack and classify them into the corresponding decoding blocks according to the header information of the data packets. When the number of data packets in any decoding block reaches the decoding threshold, the decoding operation is immediately performed, and then the decoded and repaired original data packets are submitted to the upper-layer protocol stack. At the same time, the decoding controller performs packet loss counting statistics. The ACK / NACK collection module is used to obtain the current statistics from the decoding controller and write them into the ACK / NACK response message. The ACK / NACK response message is delivered to the lower layer protocol stack and sent to the ACK / NACK feedback module of the sending end.

[0032] Specifically: Encoding buffer: At the sending end, RDMA messages (a message may contain one or more raw data packets, which are generated by the upper transport layer) are routed to this buffer to wait for encoding.

[0033] Encoding Controller: At the sending end, this controller maintains the encoding level information for the next transmission. When transmission is required, it performs encoding operations based on the number of original data packets in the buffer to generate a repair packet and add an encoding header (used to identify the encoding block and provide the sequence number within the block). After the operation is completed, it actively delivers the original packet and the repair packet to the lower-layer UDP protocol stack for further encapsulation and transmission.

[0034] ACK / NACK Feedback Module: At the sending end, ACK / NACK packets are processed by this module. For ACK packets, the packet loss statistics carried in the packet are extracted and updated based on the RTT window interval. During the update, an exponentially weighted moving average is used to smooth instantaneous fluctuations. The updated packet loss rate is then passed to the coding controller to update the next coding level. For NACK packets, since they represent a sudden high packet loss event, to quickly respond to degraded network conditions, the feedback module will directly notify the coding controller to raise the next coding level. After processing, the ACK / NACK packets are delivered to the upper-layer protocol stack for further processing.

[0035] Decoding buffer: At the receiving end, packets collected from the lower-layer UDP layer are redirected to the decoding buffer, which contains both raw packets and redundant packets.

[0036] Decoding Controller: At the receiving end, the decoding controller determines which decoding block each arriving data packet belongs to based on its encoding header. Decoding is performed when the number of data packets in each decoding block reaches the decoding requirement. After decoding, the data packet is submitted to the upper-layer protocol stack, achieving lossless transparent awareness in lossy networks. Furthermore, the decoding controller needs to track packet loss. This method only requires maintaining a packet loss count. Specifically, when a decoding block is successfully decoded, the number of redundant packets corresponding to that block is added to the packet loss count. For subsequent data packets arriving from that decoding block (even though they are no longer involved in decoding), the packet loss count is decremented by one for each packet identified. Ultimately, in scenarios without packet loss, the packet loss count will revert to its initial value (all data packets have arrived), while in scenarios with packet loss, the packet loss count represents the actual number of lost data packets. The algorithm has a time and space complexity of O(1), effectively reducing the additional overhead of introducing adaptive encoding.

[0037] ACK / NACK Collection Module: At the receiving end, packet loss statistics are transmitted via a multiplexed standard ACK / NACK channel. When returning an ACK response packet (which must occur after decoding a coding block), the receiving end includes the packet loss count statistics stored in the current decoding controller, and resets it to zero after transmission to begin counting packet loss for the next coding block. Furthermore, to avoid missing statistics due to a single ACK loss, the receiving end should embed this statistical information across multiple ACKs corresponding to the coding block, rather than centrally carrying it in a single frame ACK. No special processing is performed on NACKs.

[0038] The design concept of this invention stems from the core requirements of "decoupling" and "ease of deployment" when enhancing the RDMA protocol stack. By setting the functional entity as an additional layer below the RDMA transport layer, this invention can transparently process data as middleware, thereby providing enhanced functionality without modifying the core RDMA protocol. The system architecture of this invention is logically divided into two parts: a transmitting end and a receiving end, connected by an underlying network link. The overall operation is based on closed-loop control achieved through enhanced processing of two types of data packets: RDMA message data packets are dynamically encoded at the sending end and injected with forward error correction redundancy; they are decoded and recovered at the receiving end, achieving lossless transmission under lossy networks. ACK / NACK response packets: Lightweight link state information is embedded at the receiving end, and parsed at the sending end to drive adaptive adjustments to the coding strategy. Through the above bidirectional data processing flow, the system ultimately achieves adaptive forward error correction based on real-time network conditions, significantly improving the transmission reliability of RDMA in wide area network environments.

[0039] In summary, this invention constructs a complete lightweight state feedback closed loop and implements a dual-mode adaptive decision-making mechanism within this loop. This system-level closed loop consists of encoding and feedback processing at the transmitting end, and decoding and state statistics at the receiving end, transmitting optimized link state information through a standard acknowledgment / negative acknowledgment channel. Based on this closed loop, the sending end's decision-making mechanism is specifically manifested as follows: In normal update mode, the round-trip delay of the forward transmission is used as a statistical window to periodically aggregate feedback information, and an exponentially weighted moving average algorithm is used for smoothing to drive gradual adjustments to the encoding strategy; in event-triggered mode, when a negative acknowledgment message indicating sudden packet loss is received, the normal statistical cycle is immediately interrupted, and the encoding level is directly increased to achieve rapid response. This "dual-mode decision-making within a closed-loop framework" mechanism ensures the feasibility of adaptive capabilities from a system architecture perspective, and ensures both stability and agility in tracking network dynamics from an algorithmic perspective, thus efficiently solving the problem of balancing reliability and bandwidth efficiency in dynamic wide area networks.

[0040] This invention triggers incremental updates to packet loss statistics through successful decoding events and performs incremental corrections based on redundant packet arrival events, forming a lightweight "estimation-correction" statistical model. The statistical process employs state maintenance rules based on the lifecycle of the encoded block: a single counter is updated only at the decoding time and upon the arrival of subsequent packets, avoiding full-state bitmap tracking; simultaneously, the feedback mechanism requires that statistical information for the same encoded block be distributed across multiple acknowledgment messages. This mechanism not only reduces the overhead of statistics and feedback to a constant level (O(1)), making it suitable for resource-constrained hardware environments, but also ensures reliable transmission of state information through redundant feedback, thus providing a solid foundation for accurate adaptive decision-making at the sending end.

[0041] Furthermore, this invention can be used in a data transmission system comprising two terminal devices that interact via a wide area network. One terminal device is configured as the transmitting end of this invention; the other terminal device is configured as the receiving end of this invention. The terminal devices can be computers, edge devices, wireless devices, etc.

[0042] This invention can also be used in data centers, employing the aforementioned wide area network data transmission enhancement system. This data center can be used for cross-regional data center interconnection, distributed cloud computing platforms, high-performance computing cluster interconnection, and financial-grade cross-regional transaction systems, etc.

[0043] like Figure 2 This invention presents an example schematic diagram of the adaptive forward error correction coding transmission enhancement mechanism. The diagram illustrates the complete process of how the forward error correction code coding level changes with the network packet loss rate: 1. Normal Transmission: The sending end performs FEC encoding on the 6 original data packets, generates 2 additional repair packets, and sends them together. During WAN transmission, 1 original packet (number 4) is lost. The receiving end successfully receives 7 data packets, reaching the minimum number required for decoding (6), and then completes the decoding and submits the message to the upper layer. At the same time, the receiving end returns an ACK message carrying packet loss statistics (counted as 1 packet loss); 2. Increased Packet Loss Rate: After the previous transmission, the encoding level remained unchanged. The sender re-encoded the 6 original packets and generated 2 repair packets for transmission. Two original packets (numbered 4 and 5) were lost in this transmission. The receiver received 6 packets, which exactly met the decoding threshold, successfully recovered the data, and submitted the message. The returned ACK packet recorded the packet loss count as 2, indicating that the packet loss rate of the link had increased. 3. Encoding Level Upgrade: Based on the reported increase in packet loss rate, the system proactively upgrades the encoding level. The sending end encodes the 6 original packets and generates 3 repair packets for transmission. Two original packets (numbered 4 and 6) are still lost during transmission, but the receiving end successfully decodes them using the remaining 6 packets. The packet loss count in the ACK remains 2. This upgrade in encoding level increases redundancy and prevents decoding failures that might occur due to a continued increase in packet loss rate.

[0044] 4. Packet loss rate decreased: The previous feedback showed that the packet loss rate was stabilizing, and the encoding level remained unchanged. The sender continued to use the "6+3" encoding method. This time, only one original packet (number 4) was lost, and the receiver received and successfully decoded 7 packets. The number of lost packets in the returned ACK decreased to 1, indicating that the link condition has improved. 5. Encoding Level Reduction: Based on the decreasing trend of packet loss rate, the system appropriately reduces the encoding level. The sending end reverts to generating two repair packets. One original packet (number 4) is still lost during transmission, but the receiving end still receives seven packets, satisfying the decoding conditions. Compared to the previous round, this round reduces the transmission of one redundant packet, improving bandwidth utilization while ensuring reliability, demonstrating the efficiency advantages of adaptive coding.

[0045] This example visually demonstrates the three technical contributions of this invention: Efficient transmission recovery: By generating forward error correction redundancy packets at the sending end, the system can achieve zero round-trip time (0-RTT) data recovery at the receiving end after data packet loss, which significantly reduces the probability of end-to-end retransmission triggered by packet loss and effectively avoids the impact of long round-trip time (RTT) on transmission efficiency in wide area networks, thereby improving overall throughput. Link-adaptive coding strategy: The system can dynamically adjust coding redundancy based on real-time feedback of network packet loss. During periods of high packet loss, it automatically increases the coding level to enhance error correction capabilities and avoid protocol layer retransmissions due to insufficient redundancy; during periods of low packet loss, it decreases the coding level to reduce bandwidth overhead introduced by redundant packets, achieving an adaptive balance between reliability assurance and bandwidth utilization. Non-intrusive deployability: This invention is implemented as a forward error correction codec add-on layer, which is independent of the existing RDMA protocol stack and requires no modification to upper-layer applications or transport protocols. This decoupled design ensures the system's plug-and-play capability, significantly reducing the complexity and cost of deploying enhancements in existing RoCEv2 networks.

[0046] The above embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Any modifications, alterations, alterations, or substitutions made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for enhancing wide area network (WAN) data transmission at the sending end, characterized in that, include: The encoding controller at the sending end identifies the raw data packets of the RDMA message and routes them to the encoding buffer; Based on the dynamic coding level information, the coding controller performs forward error correction coding on the RDMA messages in the coding buffer to generate a repair packet; the original data packet and the repair packet are transmitted together to the lower-level protocol stack for encapsulation and transmission, and finally transmitted to the receiving end.

2. The method for enhancing wide area network data transmission at the sending end according to claim 1, characterized in that, When the sending end receives an ACK / NACK response message from the receiving end, the sending end's ACK / NACK feedback module intercepts and parses it, extracts the packet loss statistics from the ACK / NACK response message, and then periodically updates the packet loss rate estimate based on the round-trip delay window. After the update, it is transmitted to the encoding controller as dynamic encoding level information to adjust the encoding level for the next transmission.

3. The method for enhancing wide area network data transmission at the sending end according to claim 2, characterized in that, After extracting packet loss statistics, the ACK / NACK feedback module transmits the original ACK / NACK response message to the upper-layer protocol stack.

4. A method for enhancing wide area network data transmission at the receiving end, characterized in that, include: The decoding buffer at the receiving end receives data packets from the sending end; The decoding controller at the receiving end classifies data packets into the corresponding decoding blocks based on the header information. When the number of data packets in any decoding block reaches the decoding threshold, the decoding operation is immediately performed, and the recovered original data packets are then submitted to the upper-layer protocol stack. At the same time, the decoding controller performs packet loss counting statistics. The ACK / NACK collection module of the receiving end obtains the current packet loss statistics from the decoding controller and writes them into the ACK / NACK response message. The enhanced ACK / NACK response message is delivered to the lower-layer protocol stack, further encapsulated, and then sent to the sending end.

5. A method for enhancing data transmission over a wide area network, characterized in that, Includes the sender and receiver, where: The sending end's encoding buffer receives the raw data packets of the RDMA message; the encoding controller performs forward error correction encoding on the RDMA message in the encoding buffer according to the dynamic encoding level information to generate a repair packet; the raw data packet and the repair packet are jointly transmitted to the lower-layer protocol stack for encapsulation and transmission, and finally transmitted to the receiving end; when the sending end receives an ACK / NACK response message from the receiving end, the sending end's ACK / NACK feedback module extracts the packet loss statistics information in the ACK / NACK response message, and then periodically updates the packet loss rate estimate based on the round-trip delay window, and transmits the updated packet loss rate estimate to the encoding controller as dynamic encoding level information to adjust the encoding level for the next transmission; The receiving end receives data packets from the sending end; the decoding controller at the receiving end classifies the data packets into the corresponding decoding blocks based on the header information; when the number of data packets in any decoding block reaches the decoding threshold, the decoding operation is immediately performed, and then the recovered original data packets are submitted to the upper-layer protocol stack; at the same time, the decoding controller performs packet loss counting statistics, the ACK / NACK collection module at the receiving end obtains the current packet loss statistics from the decoding controller and writes them into the ACK / NACK response message, and the enhanced ACK / NACK response message is delivered to the lower-layer protocol stack and sent to the ACK / NACK feedback module at the sending end.

6. A transmitting end encoding algorithm, characterized in that, A method for enhancing wide area network data transmission at the sending end according to any one of claims 1-3, comprising: Step 1: Initialize the output data packet sequence P to be empty, and initialize the block-based data packet set Blocks to be empty; set the initial index value i=0; Step 2, repeat the following operations until index i has traversed the original data packet set M (i < |M|): Starting from index i, take min(R,|M|-i) data packets (R is the number of original data packets required to encode a single coding block) to form a data block; Add the data block to the block set Blocks; Increment index i by R, preparing for the next block division; Step 3, iterate through each data block in the block set Blocks: Calculate the number of data packets k in the current data block; If k=R: Encode all data packets in this data block, generate C repair packets and add encoded headers; If k≠R: Encode the k data packets in the data block, generate a repair packet and add an encoded header; Add the original data packet and the generated repair packet together to the output sequence P; Finally, the output sequence P is returned.

7. A packet loss rate statistics update algorithm for the sending end, characterized in that, A method for enhancing wide area network data transmission at the sending end according to any one of claims 1-3, comprising: Step 1, when an ACK message is received: The number of lost packets carried by the ACK, LostPkts, is added to the total number of packets lost in the period, LastPeriodLostPkts. Step 2, determine if the current ACK sequence number AckSeq is greater than the last updated sequence number LastSeq: If AckSeq is greater than LastSeq, and the total number of data packets sent in the previous period LastPeriodSentPkts is not equal to 0, meaning this period is not the first period, then proceed as follows: The periodic packet loss rate is calculated as follows: PeriodLossRate = Total number of packets lost in the period LastPeriodLostPkts / Total number of packets sent in the previous period LastPeriodSentPkts. Update the total packet loss rate using an exponentially weighted moving average: LossRate = LossRate * (1 - EWMARATE) + Period LossRate * EWMARATE; The coding level will be updated based on the updated total packet loss rate. The total number of packets lost during the update period is 0 (LastPeriodLostPkts). If AckSeq is greater than LastSeq, update the total number of data packets sent in the previous period LastPeriodSentPkts to the number of packets sent in the current period CurPeriodSentPkts, set the number of packets sent in the current period to 0, and update the last update sequence number LastSeq to the current sending sequence number SNDNXT. Step 3, when a NACK message is received: First, filter out duplicate NACKs; Upgrade the coding level; Update packet loss rate based on new coding level; Set the total number of data packets sent in the previous period (LastPeriodSentPkts), the number of packets sent in the current period (CurPeriodSentPkts), and the total number of packets lost in the period (LastPeriodLostPkts) to 0. Update the last update sequence number (LastSeq) to the current transmission sequence number (SNDNXT) and update the NACK deduplication flag.

8. A wide area network data transmission enhancement system, characterized in that, It includes a transmitter, a receiver, an upper-layer protocol stack, and a lower-layer protocol stack. The transmitter includes an encoding buffer, an encoding controller, and an ACK / NACK feedback module. The receiver includes a decoding buffer, a decoding controller, and an ACK / NACK collection module. The encoding controller is used to identify the raw data packets of the RDMA message and route them to the encoding buffer. The encoding controller is used to perform forward error correction encoding on the RDMA message in the encoding buffer according to the dynamic encoding level information, generate a repair packet, and transmit the raw data packet and the repair packet together to the lower-layer protocol stack for encapsulation and transmission, and finally transmit them to the receiving end. The ACK / NACK feedback module is used to receive ACK / NACK response messages, extract packet loss statistics from the ACK / NACK response messages, and then periodically update the packet loss rate estimate based on the round-trip delay window. After the update, it is transmitted to the encoding controller as dynamic encoding level information to adjust the encoding level for the next transmission. The decoding controller at the receiving end is used to receive data packets from the lower-layer protocol stack and classify them into the corresponding decoding blocks according to the header information of the data packets. When the number of data packets in any decoding block reaches the decoding threshold, the decoding operation is immediately performed, and then the decoded and repaired original data packets are submitted to the upper-layer protocol stack. At the same time, the decoding controller performs packet loss counting statistics. The ACK / NACK collection module is used to obtain the current statistics from the decoding controller and write them into the ACK / NACK response message. The ACK / NACK response message is delivered to the lower layer protocol stack and sent to the ACK / NACK feedback module of the sending end.

9. A data transmission system comprising two terminal devices that interact via a wide area network, characterized in that: One of the terminal devices is configured as the transmitting end as described in any one of claims 1-8; Another terminal device is configured as the receiving end as described in any one of claims 1-8.

10. A data center, characterized in that, The wide area network data transmission enhancement system described in claim 8 is adopted.

Images