Zero-waiting multi-round active transmission protocol
A sender-driven control loop with rate controller Mild and selective packet loss mechanism addresses bandwidth and latency issues in data center networks by dynamically adjusting transmission rates and minimizing packet loss, enhancing network efficiency and reducing latency.
Patent Information
- Application Number
- CN202411125666.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-12-17
AI Technical Summary
Existing active transmission protocols in high-speed data center networks face challenges in optimizing bandwidth utilization and minimizing latency due to limited global information at individual receivers, leading to underutilization, congestion, and packet loss, while multi-round matching introduces significant latency overhead.
A sender-driven control loop with a rate controller (Mild), selective packet loss mechanism, and packet loss recovery mechanism to dynamically adjust transmission rates and discard opportunistic packets when bandwidth is near saturation, using in-network telemetry for accurate load detection and minimizing packet loss.
The solution effectively utilizes idle bandwidth without causing queue accumulation, reduces packet loss, and maintains low latency across varying network loads, achieving superior performance compared to existing protocols.
Smart Images
Figure CN119155247A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of congestion control, and in particular relates to a zero-wait multi-round active transmission protocol. Background Art
[0002] To meet the growing performance requirements of high-speed data center networks (DCNs), a recent line of work allocates network bandwidth in a "request and assign" manner, which is called active transmission (e.g., pHost, Homa). Under these schemes, the receiver explicitly allocates the bottleneck bandwidth to the active sender in the form of signaling. The sender then sends packets at the optimal rate based on the received signaling.
[0003] Although active transmission has great potential, it still faces challenges to achieve optimal performance. The main problem is that each individual receiver only has a small part of the global information, which makes it difficult to make optimal decisions on signaling allocation. If the receiver only allows the highest priority traffic to be sent (for example, pHost), a lot of bandwidth will be underutilized. Although allowing multiple flows to be sent at the same time (for example, Homa) can improve bandwidth utilization, it may also introduce link congestion or even packet loss.
[0004] Some recent solutions (e.g., dcPIM) enable the receiver to obtain as much global information as possible through multiple rounds of matching, thereby elegantly utilizing the link bandwidth without causing congestion. However, multiple rounds of matching introduce significant latency overhead, because during the matching phase at the beginning of a new flow, the sender and receiver only exchange control packets to negotiate the matching, wasting a lot of bandwidth that could be used to transmit packets. While dcPIM allows flows smaller than a given threshold to be transmitted without participating in matching, flows exceeding this threshold still suffer from expensive matching overhead. Summary of the invention
[0005] The purpose of the present invention is to introduce a sender-driven control loop in the multi-round matching stage, so as to elegantly utilize the idle bandwidth of the network while avoiding additional overhead.
[0006] The object of the present invention can be achieved by the following technical scheme: a zero-wait multi-round active transmission protocol, comprising: a sender-driven rate controller Mild, a selective packet loss mechanism and a packet loss recovery mechanism;
[0007] Sender-driven rate controller Mild: used to obtain accurate network load and fill an adaptive number of opportunistic packets accordingly;
[0008] Selective packet loss mechanism: used to selectively drop opportunistic packets when bandwidth is close to being exhausted;
[0009] Packet loss recovery mechanism: used to recover from selectively discarded opportunistic data packets.
[0010] Preferably, the specific content of the rate controller Mild driven by the sender is as follows:
[0011] The rate controller Mild uses the advanced in-network telemetry signal INT to precisely detect the network load, obtains the INT information to estimate the number of bytes in transit in the network, and maintains the number of bytes in transit (that is, the amount of data that has been sent but not yet confirmed by the sender) close to the BDP (bandwidth-delay product), maximizing link utilization without causing queue accumulation in the switch.
[0012] Preferably, the INT information includes: queue length qlen, number of bytes transmitted on the port txBytes, timestamp ts and link bandwidth capacity B;
[0013] When the receiving server receives the data packet, it copies the recorded metadata into the ACK packet that will be sent back to the server that sent the data later. Then, the server that sent the data adjusts its sending window according to the network load information carried in the received ACK.
[0014] Preferably, the number of bytes transmitted on the port txBytes includes data packets in the queue and the pipeline:
[0015] For each link j, we calculate its in-transit bytes I based on the queue length qlen and its output rate txRate j , as shown in the following formula:
[0016] I j =qlen+txRate*T
[0017] In this formula, qlen is obtained through in-network telemetry technology (INT), T is the basic round-trip time, which is a constant. Now, only txRate needs to be estimated. This solution uses txBytes and ts to calculate txRate:
[0018]
[0019] ts represents the timestamp, ack1 and ack0 are two consecutive acknowledgment packets, txRate*T is the estimate of the number of bytes in the pipeline, given the estimate of the number of bytes in transit, an exponentially weighted moving average is taken to maintain the number of bytes in transit on each link I j Continue to approach BDP.
[0020] Preferably, the selective packet loss mechanism is specifically as follows: when an opportunistic packet arrives, if the buffer occupancy reaches a saturation threshold, the switch selectively discards the opportunistic packet, and only discards the main loop data packet when a queue overflow occurs. In this way, it is ensured that the data packets that fill the loop do not affect the main loop.
[0021] Preferably, the saturation threshold is 60% of the buffer capacity.
[0022] Preferably, the packet loss recovery mechanism specifically includes: locating lost opportunistic data packets, and resending the discarded opportunistic data packets using a well-protected primary loop. In this way, the rate controller Mild ensures that almost all opportunistic packets are retransmitted at most once.
[0023] Preferably, the method of locating lost opportunistic data packets is as follows: the rate controller Mild uses the sequence number carried in the ACK to detect packet loss in the middle of the first matching phase, and then, at the end of the first matching phase, the rate controller Mild sends a probe packet containing the sequence number of the last opportunistic data packet in the matching phase to locate the tail loss of the opportunistic data packet. If the server sending the data receives the probe ACK, the rate controller Mild immediately identifies all lost opportunistic data packets and retransmits them.
[0024] The beneficial effects of the present invention are as follows:
[0025] The rate controller Mild of the present invention uses a control loop driven by the sender to send sufficient opportunistic data packets to elegantly utilize the spare bandwidth left by the main loop. In addition, the rate controller Mild introduces a sophisticated selective packet loss mechanism to ensure that opportunistic data packets do not affect the transmission of RCP. Compared with the latest transmission protocols, the rate controller Mild can achieve extremely superior performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] The present invention will be further described below in conjunction with the accompanying drawings;
[0027] Figure 1 It is a schematic diagram of the system architecture of this method;
[0028] Figure 2 A flow chart of the method is given;
[0029] Figure 3 It is the latency optimization result under CacheFo ll ower workload;
[0030] Figure 4 This is the latency optimization result under WebSearch workload;
[0031] Figure 5It is the latency optimization result under DataMining workload. DETAILED DESCRIPTION
[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0033] Example:
[0034] See also Figures 1 to 5 As shown, the present invention is a zero-wait multi-round active transmission protocol, including: a sender-driven rate controller Mild, a selective packet loss mechanism and a packet loss recovery mechanism;
[0035] Rate control logic: The key challenge of rate controller design is to obtain accurate network load and fill the appropriate number of opportunistic packets accordingly. Injecting too few opportunistic packets will waste network capacity, while injecting too many will endanger normal main loop packets. The rate controller Mild uses advanced in-network telemetry signal I NT to finely detect network load. There are two reasons for choosing to adjust the rate based on INT. First, modern data center switches are becoming more and more open and flexible on the data plane. In particular, almost all switch vendors (such as Tofino, Tomahawk3) have enabled I NT functions in their new products. Second, compared with previous congestion signals (such as ECN, delay), I NT can provide more accurate link information (such as queue length, transmission rate), which is convenient for the sender to make more accurate rate adjustments.
[0036] Selective packet loss mechanism: Although the sender adopts a sophisticated rate control mechanism, the available bandwidth in the network still varies over time and space. Therefore, queue accumulation and even packet loss may still occur at some times (for example, when there is a short traffic burst). In order to minimize the impact of opportunistic packets (packets that fill the loop) on the main loop packets, this solution introduces a selective packet loss mechanism in the switch, which can selectively discard opportunistic packets when the bandwidth is close to being exhausted.
[0037] In order to protect the main loop data packets in the network, our initial idea is to use priority scheduling. However, the introduction of priority scheduling will bring some problems. First, it may cause packet disorder, because the main loop data packets with later sequence numbers in the high priority queue may arrive earlier than the opportunistic packets in the low priority queue. In addition, low-priority opportunistic packets may occupy a large number of switch buffers, increasing the risk of main loop data packets being discarded. In order to alleviate the problems caused by priority scheduling, we introduced a selective packet loss mechanism. Specifically, when an opportunistic packet arrives, if the buffer occupancy rate reaches a certain threshold, the saturation threshold of this scheme is 60% of the buffer capacity, the switch will selectively discard the opportunistic packet, and only discard the main loop data packet when the queue overflow occurs, which allows the opportunistic packet to make full use of the spare bandwidth when the network load decreases, and will not affect the transmission of the main loop packet when the network is congested. It is worth noting that in this compromise, a higher threshold allows more opportunistic packets to be transmitted on the idle bandwidth, while a lower threshold is more conducive to protecting the main loop data packets.
[0038] Packet loss recovery mechanism: Packet loss recovery is required for the selectively discarded opportunistic packets. Since the rate controller Mild has ensured the security of the primary loop packets, a fairly simple and effective packet loss recovery can be designed on the terminal host. Specifically, our idea is to use the well-protected primary loop to resend the discarded opportunistic packets. In this way, the rate controller Mild guarantees that almost all opportunistic packets are retransmitted at most once.
[0039] The specific content of the rate controller Mild driven by the sender is as follows:
[0040] The INT information obtained is used to estimate the number of bytes in transit in the network and maintain the number of bytes in transit (that is, the amount of data that has been sent but not yet confirmed by the sender) close to the BDP (bandwidth-delay product), maximizing link utilization without causing queue accumulation in the switch.
[0041] During the transmission of a data packet from the sender to the receiver, each switch on the path can insert metadata into the packet header to report the network load information of the packet's egress port. This information includes the queue length q len, the number of bytes transmitted on the port txBytes, the timestamp ts, and the link bandwidth capacity B. When the receiving server receives the data packet, the receiving server copies the recorded metadata to the acknowledgment packet ACK that will be sent back to the server that sent the data later. Then, the server that sent the data adjusts its sending window according to the network load information carried in the received ACK.
[0042] The number of bytes transmitted on the port, txBytes, includes packets in queues and pipelines:
[0043] The key design goal of the rate controller Mild is to keep the bytes in transit close to the BDP. The first challenge to achieve this goal is how to estimate the number of bytes in transit. The bytes in transit consist of packets in queues and pipelines. Similar to previous work, for each link j, we calculate its bytes in transit I based on the queue length qlen and its output rate txRate j , as shown in the following formula:
[0044] I j =qlen+txRate*T
[0045] In this formula, qlen is obtained through in-network telemetry technology (INT), T is the basic round-trip time, which is a constant. Now, only txRate needs to be estimated. This solution uses txBytes and ts to calculate txRate:
[0046]
[0047] ts represents the timestamp, ack1 and ack0 are two consecutive acknowledgment packets, txRate*T is the estimate of the number of bytes in the pipeline, given the estimate of the number of bytes in transit, an exponentially weighted moving average is taken to maintain the number of bytes in transit on each link I j Continue to approach BDP.
[0048] The selective packet loss mechanism is specifically as follows: when an opportunistic packet arrives, if the buffer occupancy reaches the saturation threshold, the switch will selectively discard the opportunistic packet, and only discard the main loop data packet when the queue overflows. By this approach, it is ensured that the data packets filling the loop do not affect the main loop.
[0049] The packet loss recovery mechanism specifically considers: locating lost opportunistic data packets and resending the discarded opportunistic data packets using a well-protected primary loop. In this way, the rate controller Mild ensures that almost all opportunistic packets are retransmitted at most once.
[0050] The method of locating lost opportunistic data packets is as follows: the rate controller Mild uses the sequence number carried in the ACK to detect packet loss in the middle of the first matching phase, and then, at the end of the first matching phase, the rate controller Mild sends a probe packet containing the sequence number of the last opportunistic data packet in the matching phase to locate the tail loss of the opportunistic data packet. If the server sending the data receives the probe ACK, the rate controller Mild immediately identifies all lost opportunistic data packets and retransmits them.
[0051] Specifically, we compare the rate controller Mild with dcPIM, Aeol us, and HPCC, which represent three design choices for data center transport, using a 100Gbps leaf-spine topology in our evaluation. For the switch, we use a 500KB buffer per port or a 16MB switch buffer. In addition, the switch enables packet spraying for load balancing. For all protocols, we set the parameters according to the parameters recommended in their respective papers.
[0052] Figure 2 , 3 Figures 4 and 5 show the average delay of different flow sizes and the tail delay of short flows when the network load changes from 0.2 to 0.8. It can be seen that the existing protocols cannot achieve low delay on all flow sizes at the same time. Mild can achieve near-optimal delay for all flow sizes at the same time. For example, under the Cache Fol lower workload, Mild reduces the average delay of short flows / medium flows / large flows by up to 8% / 57% / 46%, 69% / 1% / 24% and 52% / 91% / 86% respectively compared with dcPIM, Aeolus and HPCC. The above formulas are obtained by collecting a large amount of data for software simulation and selecting a formula close to the actual value. The coefficients in the formula are set by those skilled in the art according to the actual situation. The above are only preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field within the technical scope disclosed by the present invention shall make equivalent replacements or changes according to the technical solution and inventive concept of the present invention, which shall be covered within the protection scope of the present invention.
Claims
1. A zero-wait multi-round active transmission protocol, characterized in that: include: Sender-driven rate controller Mild, selective packet loss mechanism and packet loss recovery mechanism; Sender-driven rate controller Mild: used to obtain accurate network load and fill an adaptive number of opportunistic packets accordingly; Selective packet loss mechanism: used to selectively drop opportunistic packets when bandwidth is close to being exhausted; Packet loss recovery mechanism: used to recover from selectively discarded opportunistic data packets.
2. A zero-wait multi-round active transmission protocol according to claim 1, characterized in that: The specific content of the rate controller Mild driven by the sender is as follows: The rate controller Mild uses the advanced in-network telemetry signal INT to finely detect the network load, obtains the INT information to estimate the number of bytes in transit in the network, and maintains the number of bytes in transit to maximize link utilization without causing queue accumulation in the switch.
3. A zero-wait multi-round active transmission protocol according to claim 2, characterized in that: The INT information includes: queue length qlen, number of bytes transmitted on the port txBytes, timestamp ts and link bandwidth capacity B; When the receiving server receives the data packet, it copies the recorded metadata into an acknowledgment packet ACK that will be sent back to the server that sent the data later. Then, the server that sent the data adjusts its sending window according to the network load information carried in the received ACK.
4. A zero-wait multi-round active transmission protocol according to claim 3, characterized in that: The number of bytes transmitted on the port, txBytes, includes packets in queues and pipelines: For each link j, we calculate its in-transit bytes I based on the queue length qlen and its output rate txRate j , as shown in the following formula: I j =qlen+txRate*T In this formula, qlen is obtained through the in-network telemetry technology INT, and T is the basic round-trip time, which is a constant and is calculated using txBytes and ts: ts represents the timestamp, ack1 and ack0 are two consecutive acknowledgment packets, txRate*T is the estimate of the number of bytes in the pipeline, given the estimate of the number of bytes in transit, an exponentially weighted moving average is taken to maintain the number of bytes in transit on each link I j Continue to approach BDP.
5. A zero-wait multi-round active transmission protocol according to claim 1, characterized in that: The selective packet loss mechanism is specifically as follows: When opportunistic packets arrive, the switch selectively discards the opportunistic packets if the buffer occupancy reaches the saturation threshold, and only discards the primary loop packets when a queue overflow occurs.
6. A zero-wait multi-round active transmission protocol according to claim 5, characterized in that: The saturation threshold is 60% of the buffer capacity.
7. A zero-wait multi-round active transmission protocol according to claim 1, characterized in that: The packet loss recovery mechanism specifically includes: locating lost opportunistic data packets and resending the discarded opportunistic data packets using a well-protected primary circuit.
8. A zero-wait multi-round active transmission protocol according to claim 7, characterized in that: The method of locating lost opportunistic data packets is as follows: the rate controller Mild uses the sequence number carried in the ACK to detect packet loss in the middle of the first matching phase, and then, at the end of the first matching phase, the rate controller Mild sends a probe packet containing the sequence number of the last opportunistic data packet in the matching phase to locate the tail loss of the opportunistic data packet. If the server sending the data receives the probe ACK, the rate controller Mild immediately identifies all lost opportunistic data packets and retransmits them.