Systems and methods for inter-data center network bridging
Patent Information
- Application Number
- PCT/CN2023/097732
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-06-01
- Publication Date
- 2024-12-05
Smart Images

Figure CN2023097732_05122024_PF_FP_ABST
Abstract
Description
SYSTEMS AND METHODS FOR INTER-DATA CENTER NETWORK BRIDGINGTECHNICAL FIELD
[0001] The present disclosure pertains to the field of data center networks (DCNs) , and in particular to methods and systems for inter-data center network bridging.BACKGROUND
[0002] Modern Internet services and applications distribute data and compute on different geographically distributed data center networks to improve performance and reliability. Data center networks provide the computing power and storage necessary to support these services and applications. As the amount of data generated and processed by data center networks continues to grow, there is a growing need for efficient and effective data center networks (DCNs) that can handle the increased traffic. One of the key challenges facing DCN is congestion control, which refers to the management of network traffic to prevent bottlenecks and to provide smooth data flow. With increasing data flowing through DCNs, congestion can quickly become a major problem, leading to slow data transfer speeds, lost packets, and other issues that can undermine the performance and reliability of the network.
[0003] Current solutions for congestion control in DCNs often fall short in handling the demands of modern data center network traffic. Despite efforts to develop congestion control strategies and protocols, many existing solutions are prone to issues such as packet loss, network instability, and poor scalability. As such, improved approaches are needed to manage data traffic effectively and ensure reliable operation of these critical systems.
[0004] Therefore, there is a need for systems and methods for inter-data center network bridging that obviates or mitigates one or more limitations of the prior art.
[0005] This background information is provided to reveal information believed by the applicant to be of possible relevance to the present invention. No admission is necessarily intended, nor should be construed, that any of the preceding information constitutes prior art against the present invention.SUMMARY
[0006] Methods and systems for inter-data center bridging may be provided. According to an aspect, a method for recovering lost packets may be provided. The method includes marking for drop, by a first network device, a set of packets within a range. The range may be based on a transmission rate, R, of the first network device. The range may further be based on a buffer size, B, of the first network device. The range may further be based on a link or a data path latency, τ, for a link or a data path between the first network device and a second network device. The method may further include cloning, by the first network device, the set of packets. The method may further include storing, by the first network device, the cloned set of packets at a stalled queue according to a transmission order. The method may further include transmitting, by the first network device to a second network device, the set of packets via the link according to the transmission order. The method may further include receiving, by the first network device from the second network device, an acknowledgement (ACK) corresponding to a packet of the set of packets. The method may further include handling, by the first network device, the cloned set of packets at the stalled queue based on the ACK.
[0007] Handling, by the first network device, the cloned set of packets at the stalled queue may include dropping, by the first network device from the stalled queue, a packet of the cloned set of packets at a head of the stalled queue if the ACK corresponds to the packet of the cloned set of packets.
[0008] Handling, by the first network device, the cloned set of packets at the stalled queue may include transmitting, by the first network device to the second network device, one or more packets of the cloned set of packets beginning from a head of the stalled queue until a packet of the cloned set of packets corresponds to the ACK, the one or more packets of the cloned set of packets excluding the packet of the cloned set of packets that corresponds to the ACK.
[0009] The method may further include dropping, by the first network device from the stalled queue, the packet of the cloned set of packets that corresponds to the ACK. The marking may be distributed uniformly with time. The range may be one of: [R-r, R] , and [ (R-r / 2) , (R+r / 2) ] , where r = B / 2τ. The transmission rate, R, may be one of: an average transmission rate, and a maximum transmission rate. The stalled queue may be a first-in- first-out (FIFO) queue. Each of the first network device and the second network device may be one of: an edge router, border router and a switch.
[0010] According to another aspect, another method for recovering lost packets may be provided. The method includes sending, by a first network device to a second network device, an S number of packets. The S number of packets may be based on a number of packets that can experience a G number of consecutive lost packets. The method may further include sending, by the first network device to the second network device, G number of XORed packets generated based on one or more XOR operations on the S number of packets. As would be readily understood, an XOR operation defines that a true output results if one, and only one, of the input of the XOR operation is true. Hence, it allows regenerating one missing packet if the resulted XORed packet is known.
[0011] The method may further include receiving, by the first network device from the second network device, a request for the G number of XORed packets. The G number of XORed packets may be sent in response to the request.
[0012] The method may further include receiving, by the first network device from the second network device, a request for the G number of consecutive lost packets. The method may further include generating, by the first network device, the G number of consecutive lost packets using the G number of XORed packets. The method may further include sending, by the first network device to the second network device, the G number of consecutive lost packets
[0013] Sending, by a first network device to a second network device, an S number of packets may include sending, by the first network device to the second network device, according to a transmission order, the S number of packets in G number of sets of packets.
[0014] Each of the G number of XORed packets may correspond to a respective set of packets of the G number of sets of packets. Each of the G number of XORed packets may be sent or transmitted after said respective set of packets.
[0015] Each XORed packet of the G number of XORed packets may be generated based on an XOR operation on an Sg number of packets. The Sg number of packets may be determined according to: wherein operation S / G is rounded down to an integer value. The G number of XORed packets may include one or more XORed packets. A first XORed packet of the G number of XORed packets may be generated based on the first Sg number of packets of the S number of packets determined according to the transmission order. The Sg number of packets for each XORed packet after the first XORed packet may include: one or more packets of the corresponding respective set of packets of the G number of set of packets. The Sg number of packets for each XORed packet after the first XORed packet may further include at least one packet of a set of packets previous to the corresponding respective set of packets of the G number of sets of packets, as determined according to the transmission order.
[0016] According to another aspect, a method for recovering lost packets may be provided. The method includes receiving, by a first network device from a second network device, an M number of packets. The method may further include sending, by the first network device to the second network device, an additional of G number of XORed packets generated based on one or more XOR operations on an S number of packets including the M number of packets. The G number of packets may be based on a maximum number of lost packets that can be experienced while transmitting S number of packets. The method may further include generating, by the first network device, the G of XORed consecutive lost packets based on one or more XOR operations on the M number of packets and the G number of XORed packets.
[0017] The method may further include sending, by the first network device to the second network device, a request for the G number of XORed packets, wherein the corresponding G number of XORed packet is received in response to the request.
[0018] Receiving, by a first network device from a second network device, the M number of packets may include receiving, by the first network device from the second network device, G number of sets of packets including the M number of packets.
[0019] Each of the G number of XORed packets may correspond to a respective set of G number of sets of packets. Each of the G number of XORed packets may be generated based on an XOR operation on an Sg number of packets. The Sg number of packets may be determining according to: wherein operation S / G is rounded down to an integer value.
[0020] Generating, by the first network device, the G number of consecutive lost packets based on one or more XOR operations on the M number of packets and the G number of XORed packets may include generating, by the first network device, the G number of consecutive lost packets based on one or more XOR operations on: one or more XORed packets of the G number of XORed packets and one or more sets of packets of the G number of sets of packets.
[0021] According to another aspect, an apparatus is provided. The apparatus includes modules configured to perform one or more of the methods described herein.
[0022] According to one aspect, an apparatus is provided, where the apparatus includes: a memory, configured to store a program; a processor, configured to execute the program stored in the memory, and when the program stored in the memory is executed, the processor is configured to perform one or more of the methods described herein.
[0023] According to another aspect, a computer readable medium is provided, where the computer readable medium stores program code executed by a device and the program code is used to perform one or more of the methods described herein.
[0024] According to one aspect, a chip is provided, where the chip includes a processor and a data interface, and the processor reads, by using the data interface, an instruction stored in a memory, to perform one or more of the methods described herein.
[0025] Other aspects of the disclosure provide for apparatus, and systems configured to implement the methods according to the first aspect disclosed herein. For example, wireless stations and access points can be configured with machine readable memory containing instructions, which when executed by the processors of these devices, configures the device to perform one or more of the methods and systems described herein.
[0026] Embodiments have been described above in conjunction with aspects of the present invention upon which they can be implemented. Those skilled in the art will appreciate that embodiments may be implemented in conjunction with the aspect with which they are described but may also be implemented with other embodiments of that aspect. When embodiments are mutually exclusive, or are incompatible with each other, it will be apparent to those skilled in the art. Some embodiments may be described in relation to one aspect, but may also be applicable to other aspects, as will be apparent to those of skill in the art.BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Further features and advantages of the present invention will become apparent from the following detailed description, taken in combination with the appended drawings, in which:
[0028] FIG. 1 illustrates an architecture of inter-data center connectivity, according to an aspect.
[0029] FIG. 2 illustrates a DCI-bridging work mechanism, according to an aspect.
[0030] FIG. 3 illustrates an architecture of queue disciplines required at two edge routers, according to an aspect.
[0031] FIG. 4A and FIG. 4B illustrate working methods for queue discipline, according to an aspect.
[0032] FIG. 5 illustrates results of a simulation according to an aspect.
[0033] FIG. 6A and 6B illustrate results of another simulation, according to an aspect.
[0034] FIG. 7 illustrates results of another simulation, according to an aspect.
[0035] FIG. 8 illustrates DC-Bridging requirements to avoid packet re-ordering, according to an aspect.
[0036] FIG. 9 illustrates a comparison of DCI-bridging and transmission control protocol (TCP) connections at edge routers, according to an aspect.
[0037] FIG. 10 illustrates inter-data center bridging with network coding, according to an aspect.
[0038] FIG. 11A and 11B illustrate inter-data center bridging with network coding based on a sliding window grouping, according to an aspect.
[0039] FIG. 12 illustrates a method for recovering lost packets, according to an aspect.
[0040] FIG. 13 illustrates another method of recovering lost packets, according to an aspect.
[0041] FIG. 14 illustrates another method for recovering lost packets, according to an aspect.
[0042] FIG. 15 illustrates an apparatus that may perform any or all of operations of the above methods and features explicitly or implicitly described herein, according to different aspects of the present disclosure.
[0043] It will be noted that throughout the appended drawings, like features are identified by like reference numerals.DETAILED DESCRIPTION
[0044] Methods and systems for inter-data center bridging may be provided. According to an aspect, a method 1200 for recovering lost packets may be provided. The method 1200 includes marking 1201 for drop, by a first network device, a set of packets within a range (r) . The range (r) may be based on a transmission rate, R, of the first network device. The range may further be based on a buffer size, B, of the first network device. The range may further be based on a link / datapath latency, τ, for a link / datapath between the first network device and a second network device. The method may further include cloning 1202, by the first network device, the set of packets. The method may further include storing 1203, by the first network device, the cloned set of packets at a stalled queue according to a transmission order. The method may further include transmitting 1204, by the first network device to a second network device, the set of packets via the link according to the transmission order. The method may further include receiving 1205, by the first network device from the second network device, an acknowledgement (ACK) corresponding to a packet of the set of packets. The method may further include handling 1206, by the first network device, the cloned set of packets at the stalled queue based on the ACK.
[0045] According to an aspect, another method 1300 for recovering lost packets may be provided. The method 1300 includes sending 1301, by a first network device to a second network device, an S number of packets. The S number of packets may be based on a number of packets that can experience a maximum G number of consecutive lost packets. The method may further include sending 1302, by the first network device to the second network device, G number of XORed packets generated based on one or more XOR operations on the S number of packets.
[0046] According to an aspect, another method 1400 for recovering lost packets may be provided. The method 1400 includes receiving 1401, by a first network device from a second network device, an M number of packets. The method may further include receiving 1402, by the first network device from the second network device, G number of XORed packets generated based on one or more XOR operations on an S number of packets including the M number of packets. The S number of packets may be based on a number of packets that can experience a maximum G number of consecutive lost packets. The method may further include generating 1403, by the first network device, the G number of consecutive lost packets based on one or more XOR operations on the M number of packets and the G number of XORed packets.
[0047] Modern Internet services and applications distribute data and compute on different geographically distributed datacenters to improve performance and reliability. For example, cloud providers build storage datacenters close to their customers while build compute datacenters in rural areas to reduce the capital cost. Moreover, content providers distribute their datacenters among different regional areas and periodically replicate the content between all datacenters. In addition, database related services need to be deployed close to the customers while allowing the datacenters to synchronize data among all datacenter periodically.
[0048] Data center interconnect (DCI) technology connects two or more of these datacenters together over short, medium or long distances using high-speed connectivity. The connections between datacenters carry high volumes of continuous network traffic, comprising bulk of traffic. To support such a large volume of traffic, inter-DC network operators need to invest in expensive high capacity wide area network (WAN) backbones to connect their datacenters globally. Such connections impose significant challenges in transport design because of the distinct characteristics among routers and switches associated with these connections.
[0049] DCI connection characteristics may be described as follows: no explicit feedback signal may be available; lack of administrative control at the link / datapath (s) ; a mix of deep and shallow buffers; different round trip times (RTTs) . Service level agreements (SLA) may provide for stochastic guarantees regarding packet loss and delays for DCI traffic. Further, packet loss can also happen due to noisy transmission channel (e.g., satellite channel) , packet distortion or hardware bugs. Therefore, a vast amount of work is carried out to fully utilize these connections while maintaining low latency.
[0050] Inter-DC bulk transfers typically require transfer of traffic size ranging from tens of terabytes to petabytes, and a key performance requirement of these transfers is timely delivery. Hence, maximizing utilization is a key metric to fully utilize available bandwidth. In addition, delaying transmission completion can significantly degrade service quality and may even violate SLAs agreements.
[0051] To handle congestion control in a data center network (DCN) , some existing solutions leverage explicit congestion notification (ECN) as the congestion signal, e.g., Data Center TCP (DCTCP) and Data Center QCN (DCQCN) . Some existing solutions leverage delay as the congestion signal, e.g., TIMELYTM and SWIFTTM. Such methods provide compelling performance in terms of high-throughput and low-latency for traffic inside DCNs. Unfortunately, due to the discrepancies between DCN and DCI, none of existing solutions designed for DCN or WAN work well for a cross-DC networks.
[0052] ECN signals cannot handle the heterogenous nature of DCI or WAN. For example, configuring prober thresholds for ECN becomes challenging when flows of different RTTs coexist (may vary by up to 1000x between DCN and DCI traffic) , since small RTT traffic require lower ECN while larger RTT traffic require a larger threshold. Moreover, delay signals may vary drastically with the variation of RTT. Thus, using existing solutions may not allow for a reliable signal for DCI traffic.
[0053] A few congestion control algorithms have been proposed with the consideration of DCI / WAN characteristics, such as BBR which uses a probe-based approach to utilize available bandwidth in the network. Other solutions are proposed to use ECN for internet; e.g., L4S (low latency, low loss, scalable throughput RFC 9330. However, because of the long control loop in WAN, such approaches still suffer from slow reaction time.
[0054] TCP CUBIC is a de-facto standard congestion control algorithm used in the internet. CUBIC uses a loss-based scheme to detect congestion. However, routers and switches do not provide explicit signals for packet loss. Moreover, packet loss may also occur as a result of a noisy channel or a hardware bug. Therefore, TCP needs to use heuristic approaches to trigger the reaction to packet loss and perform retransmission and transmission rate reduction. These approaches may cause unjustified rate reduction and spurious retransmissions. Hence, recovering, by the edge routers, most of the packets that are lost may allow TCP connections to finish faster and maintain a high rate.
[0055] TCP may react to packet loss very aggressively, and at the same time, probe for available bandwidth very carefully. This reaction slows down data transmission as it needs to reduce the transmission rate by half when packet loss occurs. Routers and switches both inside DCN and WAN do not provide an explicit signal for packet loss. Moreover, as previously discussed packet loss may also occur as a result of a noisy channel or a hardware bug. Therefore, TCP needs to use heuristic approaches to trigger the reaction to packet loss and perform retransmission and transmission rate reduction.
[0056] According to an aspect, a method is provided that may allow edge routers to recover the majority of packets lost. In doing so, the rate reduction and retransmission at the host side may be reduced.
[0057] According to an aspect, an intermediate function may be introduced at the egress of the data center (inside the controlled domain) . This intermediate function may react to packet loss in WAN without triggering a host reaction. In some aspects, in case of severe congestion, the intermediate function may interact with the source hosts (e.g., using Priority-based Flow Control (PFC) IEEE standard (802.1Qbb) , Quantized Congestion Notification (QCN) IEEE standard (802.1Qau) or other ECN-based approach) to reduce their transmission rates. Hence, recovery for packet loss for DCN and DCI connections may be improved, and the control loop may be kept short, for example between the intermediate function and the source host. Such a flow control mechanism can aim to resolve transient congestion and however these is an assumption that the network load is not high (e.g., network load < 90%) .
[0058] FIG. 1 illustrates an architecture of inter-data center connectivity, according to an aspect. Architecture 100 illustrates DCNA 102 connected with DCNB 104 via a DCI connection 106 connecting edge router A 108 and edge router B 110. It is noted that the RTT of DCI link 106 (RTTDCI) can increase as high as 100x -1000x the intra-DCN RTT (RTTDCN) . This RTT of DCI link can represent a long control loop that may result in TCP slowly recovering available bandwidth and delaying a reaction to congestion and packet loss. Channel noise and transient congestion in the WAN connection can cause packet loss which can further trigger transmission rate reduction and spurious retransmission at the host. With both a long control loop and misclassified packet loss, TCP performance can drop drastically and may not recover sufficiently, thereby causing low link utilization and long completion time for transmission.
[0059] In some embodiments, a router (e.g., an edge router) may be configured to perform one more methods according to different aspects described herein. In some embodiments, one or more routers and switches for data center network may be configured to perform one or more methods, according to different aspects described herein. In some embodiments, vulnerable data paths may be defined for enabling one or more aspects, e.g., DCI-bridging (DCIB) , on these data paths. In some embodiments, edge routers may implement DCI-bridging, involving retransmission, which may allow for a shorter retransmission time than would be achieved by hardware RTT.
[0060] According to an aspect, traffic shaping may be applied at one or more edge routers using a service level agreement (SLA) transmission rate, R, or using the last used transmission rate that did not cause packet loss. FIG. 2 illustrates a DCI-bridging work mechanism, according to an aspect. In an aspect, router A 202 may inject a set of packets, e.g., packets 211, 212, 213, and 214, after marking them for drop (e.g., ECN-disabled) , with a rate higher than the rate R 206. Hence, packets 211, 212, 213, and 214 acts as probes and intermediate network devices (e.g., switches / routers) drop them in case of congestion. Router A 202 may further clone these packets and save them in a buffer. In some embodiments, router A 202 may clone the packets at the same time as injecting the packets. Moreover, a backward signal might be triggered requesting the hosts to reduce their transmission rate when router A 202 buffer encounters a high backlog.
[0061] In some embodiments, this buffer may be a low priority queue buffer capable of applying a pause action that could be provided by imitating a priority flow control (PFC) pause frame, which can be configured as further defined in IEEE 802.1Qbb. The traffic rate illustrated in graph 230, may be based on a probing range 232 which involves transmitting packets at rates above and below the selected rate R 206 within the probing range.
[0062] For each of the packets received by router B, router B 202 sends an ACK back to router A. In some embodiments, ACK may be a custom acknowledge packets defined to work solely between Router B and Router A. Router A 202 may delete or drop packets identified by the received ACK, and retransmit packets for which an ACK was not received. As illustrated in FIG. 2, when router A 202 receives an ACK 223 for packet 213 but no ACK for packet 211 and 212, the ACK 223 may indicate that the packets before packet 213 (e.g., packets 211 and 212) were lost and need to be retransmitted. Based on the ACK 223, router A 202 may resume transmission on the paused queue, by referring to the buffer storing the cloned set of packets, until the queue reaches packet 213. Router A 202 may then discard packet 213 and pause the transmission of any packets in the queue (e.g. associated with the buffer) again until router A 202 receives another ACK.
[0063] In some embodiments, the behavior of transmission, dropping, and pausing of the queue can be carried out using PFC pause frames. The fast recovery of lost packets may allow the host (e.g. the transmitting node or router) to maintain a high transmission rate and prevent TCP from reducing the congestion window (CWND) which is triggered by packet loss or duplicate acknowledgments (DupAcks) . These actions may also avoid the need for 3 DupAcks or waiting for the retransmission timeout (RTO) to expire in order to trigger retransmission thereby alleviating a potential packet reordering issue.
[0064] In some embodiments, router A can increase the traffic shaping R by an amount equivalent to the received ACK.
[0065] In some embodiments, the probing range may be a range below the selected transmission rate, R, which may be the assigned rate to the link in the SLA. For example, graph 330 (as shown in FIG. 3) illustrates such a case in which the probing range 332 is below the selected transmission rate R 331, which is the maximum rate at which packets are transmitted. In some embodiments, the selected transmission rate R may be an average transmission rate.
[0066] According to an aspect, an adaptive marking method is used. For adaptive marking probability, router A 202 may estimate the link latency (τ ) towards the other edge router, router B 204. Hence, router A 202 uses its buffer size (B ) assigned for such operations and calculates the marking rate (r ) to be r = B / 2τ. In some embodiments, the marking rate (r ) can be defined as the peak rate from the SLA agreement. Accordingly, router A 202 may mark all of the packets for drop that are within the range [R-r / 2, R+r / 2] , wherein R is the transmission rate and r is the marking rate. In some embodiments, router A 202 may probe the network within the range [R-r, R] in case the transmission rate (R ) is selected to be the maximum transmission rate allowed by the application and the SLA agreement (i.e., R=min (Rateall flows, RateSLA) . In some embodiments, the packets that are marked for dropping may be distributed uniformly with time. In some embodiments, where only one DCI link exists between router A 202 and router B 204, a first-in-first-out (FIFO) queue may be used. In some embodiments, where multiple data paths exist, router A 202 can use one queue per path assuming the number of data paths are known.
[0067] FIG. 3 illustrates an architecture of queue disciplines required at two edge routers, according to an aspect. It is to be understood that the use of “queue discipline” is used to define the features associated with a set of rules under which a router processes incoming data packets. In an aspect, two queue disciplines may be used, namely, a first queue discipline associated with the transmitting or transmitter queue (Q1 311 at router A 302 and Q1 312 at router B 304) and a second queue discipline can be associated with the receiver or receiving queue (Q2c 315 and Q2d 316 at router A and Q2a 313 and Q2b 314 at router B 304) . In an embodiment, routers in WAN may drop ECN-disabled packets in case of congestion (which can be defined in the SLA agreement) . The transmitting queue Q1 311, 312 and the receiving queue Q2 at each router 302 and 304 may operate, via their respective router, according to the associated queue discipline methods illustrated, respectively in FIG. 4A and FIG. 4B.
[0068] FIG. 4A illustrates working methods for queue discipline, according to an aspect. According to an aspect, the transmitting or sender queue, Q1, may operate, via its respective router, according to method 400. The method 400 includes receiving 401 packets to be transmitted to a remote router. The method 400 may further include calculating 402 the marking rate (r) if not preconfigured. The method 400 may further include enabling 403 ECN with rate = (R-r) , which indicates enabling ECN for all packets at rate R except than the number of packets to be marked for potential dropping (as determined via the calculated marking rate r) . Enabling ECN may include modifying an ECN field in an IP header of the packets. The method 400 may further include determining 404 whether a packet is marked for potential dropping. If a packet is marked for dropping, then the method 400 may further include cloning 405 the packet marked for dropping and storing the cloned packet in a stalled queue (alow priority queue) . After cloning 405 of the packet, the packet may be transmitted 406 to the remote router. In some embodiments, if the packet is not marked for potential dropping, then the packet is transmitted 406 to the remote router, without cloning of this unmarked packet. The method 400 may then end 407.
[0069] According to an aspect, the receiving queue, Q2, may operate, via its respective router, according to method 420. Method 420 includes receiving 421 a packet from a remote router. The packet that is received may the packet that was transmitted 406 according to method 400. The method 420 may further include determining 422 whether the packet is marked for potential dropping. If the packet is not marked for dropping, then the method 420 may further include transmitting 425 the packet and the method 420 may end 426. In some embodiments, if the packet is marked for potential dropping, then the method 420 may further include generating 423 an acknowledgement (ACK) corresponding to that marked packet. The method 420 may further include sending 424 the generated ACK to the edge router that had sent the packet. The method 420 may further include transmitting 425 the packet to the subsequent destination. The method may then end 426.
[0070] According to an aspect, the transmitting queue, Q1, may further operate, via its router, according to method 410. The method 410 may be triggered when the respective transmitting router receives 411 the ACK from the remote router. The ACK may refer to the ACK sent 424 according to method 420. The method 410 may further include checking 412 the head of the stalled queue. The method 410 may further include determining 413 whether the ACK matches the head of the stalled queue. As described, with further reference to method 400, cloned packets may be stored 405 at the stalled queue. If the ACK matches the packet at the head of the stalled queue, then the method 410 may further include dropping 415 the packet at the head of the stalled queue, and method 410 may then end 416. If the received ACK does not match the packet at the head of the stalled queue, then one or more packets at the head of the stalled queue may be transmitted 414 until a packet within the stalled queue is reached that matches the received ACK. The method 410 may then end 416.
[0071] As an example, a maximum drop rate of a DCI link may be calculated. For a DCI link of 100Gbps (maximum link capacity being defined as C) , a latency of 10ms (which may equate to a 2,000 Km distance separating the transmitting router and the receiving router) and a buffer size of 2 megabyte (MB) , the maximum drop rate that a DCI-bridging mechanism can support can be determined as follow:
[0072] Accordingly, the drop rate that can be supported for the DCI link is 1.6%. Another way to calculate the same requirements can be defined. In instances where prior knowledge of the average packet size (which can be defined as L) the requirements relating to the drop rate may be determined as follows:
[0073] As illustrated above, both calculation method lead to the same result, namely that a 2-MB buffer in a 100-Gbps link with a latency of 10 ms is sufficient to support up to a packet loss probability of approximately 1.6%, wherein it can be understood that the loss probability may be defined as the drop rate.
[0074] FIG. 5 illustrates the results of simulation, which has been characterized using a method according to an aspect. The simulation parameters 502 include: a network load of 60%; 400 number of flows; 10 seconds of simulated time; a link capacity of 1Gbps; a DCI link latency of 3ms; a marking rate of packets which may be recovered (ECN disabled) of 0.1×R (which may be defined as 10%of packets are ECN disabled and 90%of the packets are ECN enabled) ; and a WAN packet drop probability of 5%. As illustrated in graph 504, using DCI-bridging, the flow completion time (FCT) is reduced, when compared to not using DCI-bridging. The flow completion rate has changed from 21.2257 to 7.065541, which can be defined such that the rate of flow completion increasing by 66.7123%.
[0075] FIG. 6A and 6B illustrate results of another simulation, which has been characterized using a method according to an aspect. The simulation parameters 602 include: 100 number of flows; 5 sec simulated time; link capacity of 1Gbps; a DCI link latency of 3ms; a marking rate of packets which may be recovered (ECN disabled) of 0.1×R (which may be defined as 10%of packets are ECN disabled and 90%of the packets are ECN enabled) ; and a WAN packet drop probability of 5%of the recoverable packets. Using DCI-bridging, a better performance is observed with an increased number of flows. As the time between packets increases, the reaction to DupAck is slower. As illustrated in graphs 604, 606 and 608, CDI bridging allows for 8x improvement in transmission rate (Mbit / sec) when compared to the case without using CDI bridging.
[0076] FIG. 7 illustrates results of another simulation, which has been characterized using a method according to an aspect. The simulation parameters 702 include: a network load of 60%; 400 number of lows; 10 seconds of simulated time; a link capacity of 1Gbps; a DCI link latency of 3ms; a marking rate packet for recovery (ECN disabled) of 0.1×R (which may be defined as 10%of packets are ECN disabled and 90%of the packets are ECN enabled) ; and a WAN drop probability of 1%of all packets (regardless of the marking) . As illustrated in graph 704, using DCI-bridging, flow completion time (FCT) is reduced when compared to not using DCI-bridging. This reduction in flow completion time reduces from 9.2274 to 8.473, which can be defined as the rate of flow completion increasing by 8.17%.
[0077] Accordingly, using DCI-bridging may allow for increased link utilization, e.g., by a factor of 8 times, as illustrated in FIG. 6A and 6B. Further, DCI-bridging may allow for a reduction in flow completion time, e.g., which can result in about a 60%increase in the speed at which flow completion occurs, as illustrated in FIG. 5. When packet loss occurs without considering the marked packets, DCI-bridging can further allow for an increased bandwidth of the link, e.g., increases the bandwidth by 40%and reduces FCT by 8%as illustrated in reference to FIG. 7.
[0078] According to an aspect, DCI-bridging may operate per virtual link, thus DCI-bridging can operate per group of flows. In some embodiments, if the number of flows is larger than the bandwidth-delay product (BDP) of a particular link, DCI-bridging may allow packet loss recovery before the first DupAck, as illustrated in FIG. 8. FIG. 8 illustrates DC-Bridging requirements to avoid packet re-ordering, according to an aspect. Router A 802 may transmit data to router B 804 via link 806. For example, router A 802 may transmit packet f1pi 811 and packet f2pi 812 to router B 804. However, packet f1pi 811 is dropped, along link 806, and thus router B 804 does not receive packet f1pi 811. Router B 804 may send a request 807 to router A 802 for packet f1pi 811. Router A 802 may receive the request 807 for packet f1pi 811 before router B receives packet f1pi+1 813. Accordingly, router A may retransmit the lost packet f1pi 811 to router B 804 before the first DupAck, since packet f1pi+1 813 would be the first packet to generate a DupAck.
[0079] Accordingly, router A 802 may recover a lost packet before the first DupAck. If the number of flows is larger than 1 / 2 BDP (only one direction) , router A 802 may recover lost packets before the second DupAck. For a 1000-Km link, there can be a 5 ms latency, and 10 G capacity, thus BDP ≈ 8.2K packets (in the two directions) . Hence, for such a situation, 4K flows may be the minimum number of flows that allow DCI-bridging to work without triggering a TCP recovery phase at the host (namely to originally transmitting device) because of packet loss. Accordingly, DCI-bridging may maintain high throughput and smooth operation at the host.
[0080] FIG. 9 illustrates a comparison of DCI-bridging and performance-enhancing proxy (PEP) solution at edge routers, according to an aspect. According to an aspect, PEP may allow edge routers 902 and 904 to terminate TCP connections 910, 911, and 912 to alleviate the issue of DCI interconnectivity. When TCP connections are terminated, hosts may not need to wait for the end-to-end ACK to control transmission, however, each edge router 902 and 904 needs to store all inflight packets and maintain knowledge of the state for each connection from the router to the hosts to the other edge router. In addition, managing these connections can encounter more delay in processing the whole TCP stack, since it is being processed two extra times (namely inbound and outbound at the two edge routers) . Further, retransmission and buffer management may be more complex.
[0081] It is to be understood that DCI-bridging does not need to keep details relating to any per-connection state and may have a lower buffer requirement. The buffer requirement may be dependent on the maximum loss probability of the communication channel, and not the number of connections. As would be readily understood, the loss probability can be related to the drop rate. Based on DCI-bridging, router A 902 may terminate the TCP connections associated therewith and open new connections towards router B 904.
[0082] According to an aspect, network coding may be used to reduce buffer requirements to one packet and perform an XOR operation on transmitted marked-for-drop packets. As would be readily understood, an XOR operation defines that a true output results if one, and only one, of the input of the XOR operation is true. Such an approach may be useful when a service provider cannot guarantee that packet loss takes place on marked packets only. In an aspect, the XOR operation can be applied in a probabilistic manner for a number of packets for example (S) packets. FIG. 10 illustrates inter-data center bridging with network coding, according to an aspect. Router A 1002 may transmit the XORed packet after transmitting a group of S packets 1012, wherein the last packet of the group of S packets has been identified as packet S. In some embodiments, router A 1002 may transmit the XORed packet upon request from router B 1004, after transmitting the S packets 1012. Router B 1004 may intercept the XORed packet and generate the missing packet 1013 based on a local buffer of the one packet. By applying the XOR operation on all received packets, router B 1004 may generate the lost packet or zeroes if no loss occurred. Router B 1004 may then reset the XOR packet buffer and start the process again after each operation.
[0083] In some embodiments, the determination of a suitable value for S can be determined as the inverse of the loss probability in the channel (S≤ 1 / LossProb) . As may be appreciated, intermediate routers do not need to support inter-data center bridging with a network coding mechanism.
[0084] For example, referring to FIG. 10, the S packets transmitted by router A 1002 may include packet 3. However, the received packets 1015 at router B 1004 may not include packet 3. Accordingly, router B 1004 may perform one or more XOR operations 1016 on the received packets 1015 to generate the lost packet 1013 (the lost packet being packet 3) .
[0085] Accordingly, the XORed packet 1’ 1011 can be transmitted upon request from the terminator or end router. In some embodiments, the XORed packet 1’ 1011 may be transmitted after the S packets in order to guarantee packet delivery without extra delay caused by waiting for a request and a reply. Hence, lost packets may be generated before any negative acknowledgements reaching the source.
[0086] In some aspects, the inter-data center bridging with network coding can be tuned to support packet loss of approximately G packets by engaging multiple XOR operations in a sliding window configuration as shown in FIGs. 11A and 11B. FIGs. 11A and 11B illustrate inter-data center bridging with network coding based on a sliding window grouping, according to an aspect. In an embodiment, the mechanism may be applied to all packets to be transmitted, or the operation may be divided into G groups 1110 of packets, at router A 1102. The number of groups of packets, G, may be defined as the maximum number of back-to-back or consecutive packets that can be lost, wherein this packet loss can depend on the congestion state of the network.
[0087] Referring to FIG. 10, router A 1102 may transmit an XORed packet upon request from router B or at the end of transmission of all packets within a group. The XORed packet may be generated based on an XOR operation performed on the packets within the group. Router B 1104 may intercept the XORed packet and generate the missing packet based on a its locally calculated XORed packet. Hence, the edge routers need to keep one packet per group. The same method to recover lost packets may be applied in this case.
[0088] In the sliding window grouping mechanism, e.g., as illustrated in FIGs. 11A and 11B, S may refer to the number of packets that can experience a maximum of G consecutive packets being lost, and Sg may refer to the number of packets per group which is defined as In an aspect, the group size may be determined by rounding down the division of the total S packets by the number of groups; in addition, at least one packet is kept shared among two consecutive groups. It may be appreciated that such an approach can be self-tuned, wherein parameters S or G can be measured from the network.
[0089] As illustrated, router A 1102 may apply XOR operation on packets forwarded toward the terminator or end router B 1104 in a sliding window manner. For example, router A 1102 may apply XOR operations on packets 1 to Sg. In a first window 1112, router A 1102 may apply an XOR operation on packets 1111 to generate a first XORed packet (1’ ) . In a second window 1114, router A 1102 may apply an XOR operation on packets 1115 and 1113 to generate a second XORed packet (2’ ) . Router A 1102 may continue to perform similar operations on the packets until a last window 1116.
[0090] In case losing packet i in group g, terminator or end router B 1104 may generate the missing packet (g×Sg+i) 1123 by performing one or more XOR operations 1126 on the received packets 1125.
[0091] As an example, referring to FIG. 11B, for a link between router A 1102 and router B 1104, parameters S and G, where S is the number of packets that can experience a maximum of G packet loss, may respectively be 7 and 3. Hence, the number of packets per group and XOR may slide with stride of 2 which is calculated as follow: stride=Sg-1 to force sharing only one packet between two consecutive groups.
[0092] In an embodiment, packets 1131, 1132, 1133, 1134, 1135, 1136, and 1137 are to be transmitted by router A 1102 to router B 1104. The XOR packet may be generated based on each group of packets, S being 3 packets.
[0093] Accordingly, a first XOR packet group of packets may include packets 1131, 1132 and 1133. Router A may generate a first XOR packet 1151 based on an XOR operation on the first group of packets. Router A 1102 may transmit to router B 1104 the first group of packets and thereafter transmit the first XOR packet 1151. As mentioned, XOR may slide with stride of 2. Thus, the second group of packets may include packets 1133, 1134 and 1135. Router A 1102 may generate a second XOR packet 1152 based on an XOR operation on the second group of packets. Router A 1102 may then transmit to router B 1104 packets 1134 and 1135. Thereafter, router A 1102 may transmit the second XOR packet 1152. Similarly, the third group of packets, based on sliding stride of 2, may include packets 1135, 1136 and 1137, as illustrated. Router A 1102 may then generate a third XOR packet 1153 based on an XOR operation on the third group of packets. Router A 1102 may then transmit to router B 1104 packets 1136 and 1137. Thereafter, router A 1102 may transmit the third XOR packet 1153.
[0094] Of packets 1131 to 1137, two packets 1134 and 1135 may be lost in transmission. Accordingly, router B 1104 may receive packets 1131, 1132, 1133, the first XOR packet 1151, the second XOR packet 1152, packets 1136, 1137 and the third XOR packet 1153. To generate the lost packets 1134 and 1135, router B 1104 may generate the lost packet 1135 based on an XOR operation 1140 on packets 1136 and 1137 and the third XOR packet 1153 (since the third XOR packet 1153 was generated based on an XOR operation of the third group of packets, including 1135, 1136 and 1137) . Having generated packet 1135, router B 1104 may then generate the lost packet 1134 based on an XOR operation 1141 on the second XOR packet 1152 and packets 1135 and 1133 (since the second XOR packet 1152 was generated based on an XOR operation of the second group of packets, including 1133, 1134 and 1135) .
[0095] According to an aspect, edge routers use part of their deep memory to store certain packets that are marked for drop according to queue disciplines described with reference to FIG. 4A and 4B. A receiving queue may, via its respective router, notify a transmitting queue about receiving each marked packet. Accordingly, the transmitting queue, via its respective router, may remove received packets or transmit lost packets. Such an approach may allow the recovery of lost packet between data centers without the need for the hosts to reduce their transmission rates, where the packet loss rate is less than (reserved buffer / (2 x propagation delay) ) .
[0096] According to an aspect, a switch may be configured to provide both congestion control and fast packet-loss recovery. This approach may provide for improved network utilization, particularly with the long reaction time for DCI traffic (milliseconds granularity) .
[0097] According to an aspect, lost packets may be recovered using networking coding as described in reference to FIG. 10, 11A and 11B. Using networking coding may allow for packet loss recovery regardless of the marking. This approach may further allow for fast packet recovery and maintaining high throughput.
[0098] Although one or more aspects are described in reference to routers, a person skilled in the art may appreciate that a variety of different appropriate network devices (e.g., a switch) may be used to perform one or more operations described herein with reference to the routers. Thus, a variety of appropriate network devices may be configured to perform one or more methods, according to one or more aspects, described herein.
[0099] FIG. 12 illustrates a method for recovering lost packets, according to an aspect. The method 1200 includes marking 1201 for drop, by a first network device, a set of packets within a range. The range may be based on a transmission rate, R, of the first network device. The range may further be based on a buffer size, B, of the first network device. The range may further be based on a link latency, τ, for a link or a data path between the first network device and a second network device. The method may further include cloning 1202, by the first network device, the set of packets. The method may further include storing 1203, by the first network device, the cloned set of packets at a stalled queue according to a transmission order. The method may further include transmitting 1204, by the first network device to a second network device, the set of packets via the link according to the transmission order. The method may further include receiving 1205, by the first network device from the second network device, an acknowledgement (ACK) corresponding to a packet of the set of packets. The method may further include handling 1206, by the first network device, the cloned set of packets at the stalled queue based on the ACK.
[0100] Handling, by the first network device, the cloned set of packets at the stalled queue may include dropping, by the first network device from the stalled queue, a packet of the cloned set of packets at a head of the stalled queue if the ACK corresponds to the packet of the cloned set of packets.
[0101] Handling, by the first network device, the cloned set of packets at the stalled queue may include transmitting, by the first network device to the second network device, one or more packets of the cloned set of packets beginning from a head of the stalled queue until a packet of the cloned set of packets corresponds to the ACK, the one or more packets of the cloned set of packets excluding the packet of the cloned set of packets that corresponds to the ACK.
[0102] The method may further include dropping, by the first network device from the stalled queue, the packet of the cloned set of packets that corresponds to the ACK. The marking may be distributed uniformly with time. The range may be one of: [ (R-r) , R] , and [ (R-r / 2) , (R+r / 2) ] , where r = B / 2τ. The transmission rate, R, may be one of: an average transmission rate, and a maximum transmission rate. The stalled queue may be a first-in-first-out (FIFO) queue. Each of the first network device and the second network device may be one of: an edge router, border router and a switch.
[0103] FIG. 13 illustrates another method of recovering lost packets, according to an aspect. The method 1300 includes sending 1301, by a first network device (e.g., router A 1002, 1102, a switch, or a transmitting device) to a second network device (e.g., router B 1004, 1104, a switch or a receiving device) , an S number of packets. The S number of packets may be based on a number of packets that can experience a G number of consecutive lost packets. In some embodiments, the G number of consecutive lost packets may be a maximum number of lost packets based on the S number of transmitted packets. The method may further include sending 1302, by the first network device to the second network device, G number of XORed packets generated based on one or more XOR operations on the S number of packets.
[0104] The method may further include receiving, by the first network device from the second network device, a request for the G number of XORed packets. The G number of XORed packets may be sent in response to the request.
[0105] The method may further include receiving, by the first network device from the second network device, a request for the G number of consecutive lost packets. The method may further include generating, by the first network device, the G number of consecutive lost packets using the G number of XORed packets. The method may further include sending, by the first network device to the second network device, the G number of consecutive lost packets.
[0106] Sending, by a first network device to a second network device, an S number of packets may include sending, by the first network device to the second network device, according to a transmission order, the S number of packets in G number of sets of packets.
[0107] Each of the G number of XORed packets may correspond to a respective set of packets of the G number of sets of packets. Each of the G number of XORed packets may be sent or transmitted after said respective set of packets.
[0108] Each XORed packet of the G number of XORed packets may be generated based on an XOR operation on an Sg number of packets. The Sg number of packets may be determined according to: wherein operation S / G is rounded down to an integer value.
[0109] The G number of XORed packets may include one or more XORed packets. A first XORed packet of the G number of XORed packets may be generated based on the first Sg number of packets of the S number of packets determined according to the transmission order. The Sg number of packets for each XORed packet after the first XORed packet may include: one or more packets of the corresponding respective set of packets of the G number of sets of packets. The Sg number of packets for each XORed packet after the first XORed packet may further include at least one packet of a set of packets previous to the corresponding respective set of packets of the G number of sets of packets, as determined according to the transmission order.
[0110] FIG. 14 illustrates another method for recovering lost packets, according to an aspect. The method 1400 includes receiving 1401, by a first network device (e.g., router B 1004, 1104, a switch, or a receiving device) from a second network device (e.g., router A 1002, 1102, a switch, or a transmitting device) , an M number of packets. The method may further include receiving 1402, by the first network device from the second network device, G number of XORed packets generated based on one or more XOR operations on an S number of packets including the M number of packets. The S number of packets may be based on a number of packets that can experience G number of consecutive lost packets. The method may further include generating 1403, by the first network device, the G number of consecutive lost packets based on one or more XOR operations on the M number of packets and the G number of XORed packets.
[0111] The method may further include sending, by the first network device to the second network device, a request for the G number of XORed packets. The G number of XORed packet may be received in response to the request.
[0112] Receiving, by a first network device from a second network device, the M number of packets may include receiving, by the first network device from the second network device, G number of sets of packets including the M number of packets.
[0113] Each of the G number of XORed packets may correspond to a respective set of G number of sets of packets. Each of the G number of XORed packets may be generated based on an XOR operation on an Sg number of packets. The Sg number of packets may be determining according to: wherein operation S / G is rounded down to an integer value.
[0114] Generating, by the first network device, the G number of consecutive lost packets based on one or more XOR operations on the M number of packets and the G number of XORed packets may include generating, by the first network device, the G number of consecutive lost packets based on one or more XOR operations on: one or more XORed packets of the G number of XORed packets and one or more sets of packets of the G number of sets of packets.
[0115] FIG. 15 illustrates an apparatus 1500 that may perform any or all of operations of the above methods and features explicitly or implicitly described herein, according to different aspects of the present disclosure. For example, a computer equipped with network function may be configured as the apparatus 1500. In some aspect, apparatus 1500 can be a device that connects to the network infrastructure over a radio interface, such as a mobile phone, smart phone or other such device that may be classified as user equipment (UE) . In some aspects, the apparatus 1500 may be a machine type communications (MTC) device (also referred to as a machine-to-machine (m2m) device) , or another such device that may be categorized as a UE despite not providing a direct service to a user. In some aspects, apparatus 1500 may be used to implement one or more aspects described herein. For example, the apparatus 1500 may be a network device (arouter, an edge or border router, or a switch or other suitable device) configured to implement or perform one or more operations described herein.
[0116] As shown, the apparatus 1500 may include a processor 1510, such as a central processing unit (CPU) or specialized processors such as a graphics processing unit (GPU) or other such processor unit, memory 1520, non-transitory mass storage 1530, input-output interface 1540, network interface 1550, and a transceiver 1560, all of which are communicatively coupled via bi-directional bus 1570. According to certain aspects, any or all of the depicted elements may be utilized, or only a subset of the elements. Further, apparatus 1500 may contain multiple instances of certain elements, such as multiple processors, memories, or transceivers. Also, elements of the hardware device may be directly coupled to other elements without the bi-directional bus. Additionally, or alternatively to a processor and memory, other electronics, such as integrated circuits, may be employed for performing the required logical operations.
[0117] The memory 1520 may include any type of non-transitory memory such as static random-access memory (SRAM) , dynamic random-access memory (DRAM) , synchronous DRAM (SDRAM) , read-only memory (ROM) , any combination of such, or the like. The mass storage element 1530 may include any type of non-transitory storage device, such as a solid-state drive, hard disk drive, a magnetic disk drive, an optical disk drive, USB drive, or any computer program product configured to store data and machine executable program code. According to certain aspects, the memory 1520 or mass storage 1530 may have recorded thereon statements and instructions executable by the processor 1510 for performing any of the aforementioned method operations described above.
[0118] Aspects of the present disclosure can be implemented using electronics hardware, software, or a combination thereof. In some aspects, this may be is implemented by one or multiple computer processors executing program instructions stored in memory. In some aspects, the invention is implemented partially or fully in hardware, for example using one or more field programmable gate arrays (FPGAs) or application specific integrated circuits (ASICs) to rapidly perform processing operations.
[0119] It will be appreciated that, although specific aspects of the technology have been described herein for purposes of illustration, various modifications may be made without departing from the scope of the technology. The specification and drawings are, accordingly, to be regarded simply as an illustration of the invention as defined by the appended claims, and are contemplated to cover any and all modifications, variations, combinations or equivalents that fall within the scope of the present invention. In particular, it is within the scope of the technology to provide a computer program product or program element, or a program storage or memory device such as a magnetic or optical wire, tape or disc, or the like, for storing signals readable by a machine, for controlling the operation of a computer according to the method of the technology and / or to structure some or all of its components in accordance with the system of the technology.
[0120] Acts associated with the method described herein can be implemented as coded instructions in a computer program product. In other words, the computer program product is a computer-readable medium upon which software code is recorded to execute the method when the computer program product is loaded into memory and executed on the microprocessor of the wireless communication device.
[0121] Further, each operation of the method may be executed on any computing device, such as a personal computer, server, PDA, or the like and pursuant to one or more, or a part of one or more, program elements, modules or objects generated from any programming language, such as C++, Java, or the like. In addition, each operation, or a file or object or the like implementing each said operation, may be executed by special purpose hardware or a circuit module designed for that purpose.
[0122] Through the descriptions of the preceding aspects, the present invention may be implemented by using hardware only or by using software and a necessary universal hardware platform. Based on such understandings, the technical solution of the present invention may be embodied in the form of a software product. The software product may be stored in a non-volatile or non-transitory storage medium, which can be a compact disc read-only memory (CD-ROM) , USB flash disk, or a removable hard disk. The software product includes a number of instructions that enable a computer device (personal computer, server, or network device) to execute the methods provided in the aspects of the present invention. For example, such an execution may correspond to a simulation of the logical operations as described herein. The software product may additionally or alternatively include a number of instructions that enable a computer device to execute operations for configuring or programming a digital logic apparatus in accordance with aspects of the present invention.
[0123] Although the present invention has been described with reference to specific features and aspects thereof, it is evident that various modifications and combinations can be made thereto without departing from the invention. The specification and drawings are, accordingly, to be regarded simply as an illustration of the invention as defined by the appended claims, and are contemplated to cover any and all modifications, variations, combinations or equivalents that fall within the scope of the present invention.
Claims
1.A method comprising:marking for drop, by a first network device, a set of packets within a range, where the range is based on one or more of:a transmission rate, R, of the first network device;a buffer size, B, of the first network device; anda link latency, τ, for a link between the first network device and a second network device;cloning, by the first network device, the set of packets;storing, by the first network device, the cloned set of packets at a stalled queue according to a transmission order;transmitting, by the first network device to a second network device, the set of packets via the link according to the transmission order;receiving, by the first network device from the second network device, an acknowledgement (ACK) corresponding to a packet of the set of packets; andhandling, by the first network device, the cloned set of packets at the stalled queue based on the ACK.2.The method of claim 1, wherein handling, by the first network device, the cloned set of packets at the stalled queue comprises:dropping, by the first network device from the stalled queue, a packet of the cloned set of packets at a head of the stalled queue if the ACK corresponds to the packet of the cloned set of packets.3.The method of claim 1, wherein handling, by the first network device, the cloned set of packets at the stalled queue comprises:transmitting, by the first network device to the second network device, one or more packets of the cloned set of packets beginning from a head of the stalled queue until a packet of the cloned set of packets corresponds to the ACK, the one or more packets of the cloned set of packets excluding the packet of the cloned set of packets that corresponds to the ACK.4.The method of claim 3 further comprising:dropping, by the first network device from the stalled queue, the packet of the cloned set of packets that corresponds to the ACK.5.The method of claim 1 or 4, wherein the range is one of: [R-r, R] , and [ (R-r / 2) , (R+r / 2) ] , where r = B / 2τ.6.The method of any one of claims 1 to 5, wherein R is one of: an average transmission rate and a maximum transmission rate.7.The method of any one of claims 1 to 6, wherein the stalled queue is a first-in-first-out (FIFO) queue.8.The method of any one of claims 1 to 7 wherein:each of the first network device and the second network device is one of: an edge or border router and a switch.9.A method comprising:sending, by a first network device to a second network device, an S number of packets, wherein: S is a number of packets that can experience G number of consecutive lost packets; andsending, by the first network device to the second network device, G number of XORed packets generated based on one or more XOR operations on the S number of packets.10.The method of claim 9 further comprising:receiving, by the first network device from the second network device, a request for the G number of XORed packets, wherein the G number of XORed packet is sent in response to the request.11.The method of claim 9 further comprising:receiving, by the first network device from the second network device, a request for the G number of consecutive lost packets;generating, by the first network device, the G number of consecutive lost packets using the G number of XORed packets; andsending, by the first network device to the second network device, the G number of consecutive lost packets.12.The method of any one of claims 9 to 11 wherein sending, by a first network device to a second network device, an S number of packets comprises:sending, by the first network device to the second network device, according to a transmission order, the S number of packets in G number of sets of packets.13.The method of claim 12 wherein each of the G number of XORed packets:corresponds to a respective set of packets of the G number of sets of packets; andis sent after said respective set of packets.14.The method of claim 13, wherein each XORed packet of the G number of XORed packets is generated based on an XOR operation on an Sg number of packets determined according to: wherein operation S / G is rounded down to an integer value.15.The method of claim 14 wherein:a first XORed packet of the G number of XORed packets is generated based on the first Sg number of packets of the S number of packets determined according to the transmission order; andthe Sg number of packets for each XORed packet after the first XORed packet comprises:one or more packets of the corresponding respective set of packets of the G number of sets of packets; andat least one packet of a set of packets previous to the corresponding respective set of packets of the G number of sets of packets, as determined according to the transmission order.16.A method comprising:receiving, by a first network device from a second network device, an M number of packets;receiving, by the first network device from the second network device, G number of XORed packets generated based on one or more XOR operations on an S number of packets comprising the M number of packets, wherein S is a number of packets that can experience G number of consecutive lost packets; andgenerating, by the first network device, the G number of consecutive lost packets based on one or more XOR operations on the M number of packets and the G number of XORed packets.17.The method of claim 16 further comprising:sending, by the first network device to the second network device, a request for the G number of XORed packets, wherein the G number of XORed packets is received in response to the request.18.The method of claim 16 or 17 wherein receiving, by a first network device from a second network device, the M number of packets comprises:receiving, by a first network device from a second network device, G number of sets of packets comprising the M number of packets.19.The method of claim 18 wherein each of the G number of XORed packets:corresponds to a respective set of the G number of sets of packets; andis generated based on an XOR operation on an Sg number of packets, wherein Sg is determined according to: wherein operation S / G is rounded down to an integer value.20.An apparatus comprising:at least one processor; andat least one machine-readable medium storing executable instructions which when executed by the at least one processor configure the apparatus to perform any one of method claims 1 to 19.21.A computer device comprising a non-transitory computer readable medium having instructions stored thereon which, when executed by a computer processor, causes the computer to perform the method as described in any one of claims 1 to 19.
Citation Information
Patent Citations
Cache sharing method and device
CN106850457A
Data transmission method, apparatus and system
CN108965308A
Live broadcast data transmission method and system and readable storage medium
CN114124910A
Data processing method and device, program product, computer equipment and medium
CN114978427A
Adaptive acknowledgement control
US20190222528A1
