RoCE network accurate congestion control method and system based on congestion source differentiation

Abstract

The application discloses a congestion control method and system for RoCE network based on congestion source differentiation, which comprises the following steps: a sending end marks a signaling flag according to the residual sending state of a data flow to which a data packet belongs, so as to differentiate the data packet into a signaling packet and a non-signaling packet; a switch takes the growth amount of the length of a port queue of the switch within a round trip time as a congestion indication, generates a first type of token and a second type of token when detecting the growth of the queue length, and appends the tokens to an acknowledgement packet, which carries and consumes the tokens; a receiving end copies a custom packet header carrying token information in a received data packet to a corresponding acknowledgement packet, and returns along a reverse path; the sending end pauses sending data according to the number of the first type of token when receiving the acknowledgement packet carrying the first type of token; and the sending end reduces the sending rate when receiving an acknowledgement packet indicating that the second type of token has been consumed. The application can realize higher-precision congestion control and reduce the degree of overreaction in congestion processing.

Description

Technical Field

[0001] This invention relates to the field of congestion control technology for data center networks, and specifically to a precise congestion control method and system for RoCE networks based on congestion source differentiation. Background Technology

[0002] As applications such as cloud computing, distributed storage, and machine learning place increasingly stringent demands on latency and throughput, data center link speeds are evolving towards 800Gbps and even higher. In Remote Direct Memory Access (RDMA) networks, new data flows are typically initiated at line rate to maximize link utilization. While this line-rate initiation can improve application performance in some scenarios, it exacerbates congestion in large-scale networks: when a new flow enters a saturated link, its impact on the bottleneck queue depends on the flow size—flows larger than the bandwidth-delay product (BDP) will increase the bottleneck queue length by approximately one BDP within a round-trip time (RTT), while flows smaller than the BDP will increase the queue length by a length equivalent to its own size. As line rates double from 100Gbps to 200Gbps, the queue growth caused by sudden surges in flow also increases exponentially.

[0003] Meanwhile, most modern data center switches are shallow-buffered switches. A rapid influx of burst traffic can easily exhaust the limited buffers, leading to buffer overflows. Although lossless Ethernet uses Priority Flow Control (PFC) to prevent packet loss due to buffer overflows, PFC introduces well-known problems such as head-of-line blocking (HoL blocking) and unfairness. Furthermore, many-to-one incast traffic, a common traffic pattern in data centers, is also highly susceptible to triggering buffer overflows and activating PFC.

[0004] To address sudden congestion in high-speed networks, an effective congestion control scheme should meet three key requirements: (1) maintain a very small queue length, i.e., keep the queue size close to zero in steady state to reserve buffer margin, thereby absorbing sudden traffic and reducing the PFC trigger frequency when congestion occurs; (2) achieve timely congestion response, i.e. shorten the feedback delay to speed up congestion resolution, while traditional congestion control schemes require at least one RTT of feedback delay; (3) adopt a precise control strategy, i.e. accurately calculate the rate reduction within the range required to eliminate congestion, avoiding long-term link underutilization caused by over-reaction. The third requirement is the most critical: when facing severe sudden congestion, blindly and drastically reducing the rate may alleviate the suddenness, but it often leads to severe link underutilization.

[0005] Existing congestion control schemes generally fail to distinguish congestion sources and can be broadly categorized into fine-grained and coarse-grained approaches. Fine-grained schemes, such as the High-Precision Congestion Control (HPCC) algorithm, utilize intra-network telemetry (INT) to obtain precise link load information for accurate rate adjustment, but their feedback delay remains one RTT, and congestion quantification is performed at the endpoint. Coarse-grained schemes, such as the Data Center Congestion Control (DCQCN) and Timely algorithms based on round-trip time (RTT), lack precise link load information and can only adjust rates tentatively, often requiring multiple iterations to stabilize, resulting in poor dynamic performance. More importantly, congestion control has two decoupled core objectives: clearing queued packets accumulated in the network and ensuring that the summed sending window (SWND) on the link equals the bandwidth-delay product (BDP). Traditional schemes first lower SWND below BDP to clear the queue, then rely on a rate-up mechanism to restore SWND to BDP. Therefore, during the rate recovery phase after congestion, SWND remains below BDP, resulting in wasted link utilization.

[0006] In summary, existing congestion control technologies have at least the following drawbacks:

[0007] (1) No distinction between congestion sources. Existing solutions treat all congestion equally. Even HPCC, which is known for its high accuracy, only controls based on link utilization without distinguishing the source of congestion. This can lead to overshooting during the deceleration step, resulting in underutilization of the link, and overshooting during the acceleration step, resulting in excessive queuing.

[0008] (2) Large feedback delay. Traditional solutions require at least one RTT for feedback, making it difficult to respond to sudden congestion in a timely manner within a sub-RTT time scale; some existing sub-RTT solutions are either not designed for RDMA or are too complex to be practically deployed.

[0009] (3) Coupling of speed reduction and queue emptying. Because the two objectives of reducing the speed and emptying the queue are coupled together, the link remains underutilized during the recovery phase after congestion, wasting bandwidth.

[0010] (4) Overreacting to mouse flow congestion. For congestion caused by mouse flow with a lifespan of less than one RTT, it does not essentially change the steady-state SWND. The optimal response should be a precisely timed brief pause rather than a reduction in speed. Existing solutions generally reduce speed in this case, resulting in significant overreaction and bandwidth waste.

[0011] Therefore, there is an urgent need for a precise congestion control method for RoCE (RDMA over Converged Ethernet) networks based on congestion source differentiation to solve the above-mentioned shortcomings. Summary of the Invention

[0012] The purpose of this invention is to address the shortcomings of existing technologies by providing a precise congestion control method and system for RoCE networks based on congestion source differentiation. This invention distinguishes different congestion sources and applies targeted control to them, decoupling the deceleration operation from the queue clearing operation. This achieves high-precision congestion control by minimizing overreaction while maintaining minimal queue length and sub-RTT timely response.

[0013] The objective of this invention is achieved through the following technical solution: The first aspect of this invention provides a precise congestion control method for RoCE networks based on congestion source differentiation, applied to a RoCE network including a transmitter, a switch, and a receiver. The method specifically includes: When sending a data packet, the sending end marks the data packet with a signaling flag according to the remaining transmission status of the data stream to which the data packet belongs, so as to distinguish the data packet from the signaling packet and the non-signaling packet. The switch uses the increase in its port queue length over a round-trip time as a congestion indicator. When it detects an increase in the queue length, it generates a first type of token and a second type of token on the port. The first type of token is used to instruct the sender to pause transmission, and the second type of token is used to instruct the sender to reduce the transmission rate. The switch attaches the first type of token and the second type of token to the acknowledgment packet that returns along the reverse path. The acknowledgment packet carries and consumes the acknowledgment packet. After receiving the data packet, the receiving end generates a corresponding acknowledgment packet, copies the token information carried in the data packet into the custom header of the acknowledgment packet, and returns the acknowledgment packet to the sending end along the reverse path; When the sending end receives an acknowledgment packet carrying a first-class token, it pauses data transmission according to the number of first-class tokens carried to clear the backlog queue; when the sending end receives an acknowledgment packet indicating that a second-class token has been consumed, it reduces the transmission rate. The pause operation to clear the backlog queue is decoupled from the speed-down operation to reduce the sending rate.

[0014] Furthermore, marking the data packet with a signaling flag based on the remaining transmission status of the data stream to which the data packet belongs specifically includes: If the number of remaining bytes to be sent in the transmission buffer of the data stream to which the data packet belongs is not less than the product of the current transmission rate and the round-trip delay of the data stream, the data packet is marked as a signaling packet; otherwise, it is marked as a non-signaling packet.

[0015] Furthermore, the number of both the first and second type of tokens generated by the switch is equal to the growth rate; The duration for which the sending end pauses transmission is the ratio of the number of first-type tokens carried in the acknowledgment packet to the current transmission rate; The step size by which the transmitting end reduces the transmission rate is the ratio of a maximum transmission unit to the round-trip delay; When the first type of token and the second type of token are carried and consumed by the acknowledgment packet, the switch will give priority to the second type of token for consumption by non-signaling packets, so that non-signaling packets can consume the second type of token on both the forward and reverse paths. Each data packet and its corresponding acknowledgment packet consumes no more than the size of its own bytes of first and second type tokens.

[0016] Furthermore, the custom header includes the following fields: The acknowledgment packet contains a size field for transmitting the corresponding data packet size, a token number field for recording the number of Type 1 tokens generated, a Type 1 token field, an acceleration field for indicating the acceleration level, a deceleration field for indicating that Type 2 tokens have been consumed and a deceleration notification has been issued, a user data packet identifier field, an acknowledgment packet identifier field, a signaling flag field for distinguishing between signaling packets and non-signaling packets, an access flag field for obtaining data on the forward path, and a reserved field for byte alignment.

[0017] Furthermore, it also includes a task verification mechanism: Set up a capacity register for each port of the switch, and set up a remaining register for the transmitting end; When the switch generates the first type of token, it increments the value of the capacity register by the corresponding number of tokens. When the sending end pauses transmission, it adds the number of received Type I tokens to the remaining register; after the sending end resumes transmission, it fills the Type I tokens stored in the remaining register back into the data packets sent in subsequent transmissions. Each data packet is filled with Type I tokens no larger than its own byte size and the value of the remaining register is reduced accordingly. When a switch receives a data packet carrying a Type 1 token, it subtracts the number of Type 1 tokens carried in the data packet from the value of the capacity register and compares the current queue length with the value of the capacity register. If the queue length still exceeds the value of the capacity register, it continues to generate new Type 1 tokens based on the difference.

[0018] Furthermore, the header of the data packet is provided with a token number field for recording the number of first-type tokens generated. When the data packet passes through multiple switches, the downstream switch first increments the value of the token number field in its capacity register after receiving the data packet, and then executes the subsequent task verification mechanism.

[0019] Furthermore, the switch counts the number of data packets sent for each port since the last time the capacity register was cleared. When the number reaches a preset capacity register reset threshold and the capacity register is still not cleared, the capacity register is reset to zero.

[0020] Furthermore, it also includes the step of detecting available bandwidth to increase the transmission rate: The switch measures the actual dequeue time interval between adjacent data packets at the port where the queue exists and compares it with a reference interval; Based on the comparison results, the corresponding acceleration level is marked in the acceleration field of the data packet header, and only the smaller acceleration level indicating higher link utilization is retained when the data packet passes through each port; When the sending end receives an acknowledgment packet carrying an acceleration field, it increases the sending rate according to the acceleration level. Wherein, the actual dequeue time interval is the difference between the arrival timestamp of the current data packet and the arrival timestamp of the previous data packet, and the reference interval is the ratio of the maximum transmission unit to the link bandwidth.

[0021] Furthermore, marking the corresponding acceleration level in the acceleration field of the data packet header based on the comparison result specifically includes: Calculate the ratio of the actual outbound time interval to the baseline interval, map the ratio to several preset intervals, and then mark the corresponding acceleration level in the acceleration field of the data packet header according to the interval in which the ratio falls.

[0022] A second aspect of the present invention provides a system for implementing the above-described RoCE network precise congestion control method based on congestion source differentiation, comprising a transmitter, a switch, and a receiver; The sending end is used to: when sending a data packet, mark the data packet with a signaling flag according to the remaining transmission status of the data stream to which the data packet belongs, so as to distinguish the data packet into a signaling packet and a non-signaling packet; when receiving an acknowledgment packet carrying a first type of token, suspend data transmission according to the number of first type of tokens carried, so as to clear the backlog queue; and reduce the transmission rate when receiving an acknowledgment packet indicating that a second type of token has been consumed. The switch is used to: use the increase in port queue length within a round-trip time as a congestion indication, and when a queue length increase is detected, generate independent first and second type tokens on the port, and attach the first and second type tokens to the acknowledgment packet returned along the reverse path, which is carried and consumed by the acknowledgment packet; The receiving end is used to: copy the custom header carrying token information in the received data packet into the corresponding confirmation packet and return it along the reverse path; The pause operation to clear the backlog queue and the deceleration operation to reduce the transmission rate are decoupled from each other.

[0023] Compared with the prior art, the beneficial effects of the present invention are:

[0024] (1) Congestion source perception congestion handling: This invention applies different strategies to rat flow congestion and elephant flow congestion, especially to resolve congestion caused by rat flow with negligible overreaction.

[0025] (2) Sub-RTT control and minimal queue: This invention mainly alleviates congestion by pausing transmission, can respond within the sub-RTT time scale, and keeps the queue growth at a minimal level during traffic bursts.

[0026] (3) Fast bandwidth recovery: When underutilization occurs, the present invention can quickly and accurately increase the transmission rate to recover available bandwidth, so that SWND and BDP are tightly coupled, avoiding the problem of continuous underutilization during the recovery phase of the traditional scheme.

[0027] (4) Implementation of fair bandwidth share: This invention decouples the speed reduction and queue clearing while taking fairness into account, so that each flow eventually converges to its fair bandwidth share. Experiments show that, compared with the representative high-precision scheme, this invention can significantly improve the tail flow completion time and greatly reduce the tail queue length.

[0028] (5) Strong applicability: This invention can be applied to the sub-RTT level precise control of sudden congestion in high-speed, low-latency data center networks such as RDMA. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the overall architecture of the RoCE network precise congestion control system based on congestion source differentiation of the present invention; Figure 2 This is a flowchart illustrating the determination of signaling packets and non-signaling packets according to the present invention. Figure 3 This is a flowchart of the switch-side congestion elimination mechanism and task verification mechanism of the present invention; Figure 4 This is a flowchart of the present invention for detecting available bandwidth to accurately improve the transmission rate. Detailed Implementation

[0030] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0031] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0032] It should be understood that although the terms first, second, third, etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of this invention, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0033] The present invention will now be described in detail with reference to the accompanying drawings. Unless otherwise specified, the features of the following embodiments and implementations can be combined with each other.

[0034] This invention addresses the problem that traditional congestion control algorithms fail to distinguish between different congestion sources and overreact when data center line speed increases lead to sudden and aggravated congestion. This results in a greater reduction in link utilization after congestion handling and more serious bandwidth waste. The invention provides a RoCE network precise congestion control method and system based on congestion source differentiation.

[0035] Example 1: A precise congestion control method for RoCE networks based on congestion source differentiation

[0036] The RoCE network precision congestion control method based on congestion source differentiation of the present invention can be applied to high-speed network environments such as data centers, and is especially suitable for networks that have deployed the RDMA protocol. In a typical implementation scenario, the network system consists of multiple transmitters, multiple receivers, and several switches that support programmable data planes. The transmitters and receivers communicate with each other through RoCE or other similar protocols.

[0037] like Figure 1 As shown, the congestion control method specifically includes the following steps:

[0038] (1) Sending end data packet marking stage: When sending a data packet, the sending end marks the data packet with a signaling flag according to the remaining sending status of the data stream to which the data packet belongs, so as to distinguish the data packet into signaling packets and non-signaling packets. The purpose is to accurately identify which data packets correspond to acknowledgment packets that can effectively carry congestion control instructions.

[0039] In this embodiment, it is necessary to first classify and analyze the congestion sources. Specifically, congestion sources can be fundamentally divided into two categories: the arrival of new flows and the acceleration of existing flows. Among the arrival of new flows, they are further distinguished into mouse flows (characterized by small flow, strong bursts, and short duration) and elephant flows (characterized by large flow and long duration) based on the size of the flow; while the acceleration of existing flows can only come from elephant flows. Thus, three types of congestion sources are obtained: arrival of mouse flows, arrival of elephant flows, and acceleration of elephant flows. After analysis, these three types of congestion sources ultimately boil down to two basic effects: either keeping SWND unchanged (corresponding to non-signaling packets) or increasing SWND (corresponding to signaling packets). Among them, non-signaling packets are data packets sent within the last RTT of the data flow. When their corresponding acknowledgment packets return, the sender has no subsequent data to send, so applying congestion marking to them is meaningless for congestion control; signaling packets are data packets sent before the last RTT. When their corresponding acknowledgment packets return, the sender can still adjust the sending of subsequent data packets, thereby achieving effective congestion control. Therefore, data packets need to be classified into signaling packets and non-signaling packets based on whether they can report congestion.

[0040] Furthermore, based on the remaining transmission status of the data stream to which the data packet belongs, a signaling flag is marked on the data packet. Specifically, in RDMA, based on the comparison between the number of remaining bytes to be transmitted in the transmission buffer of the data stream to which the data packet belongs and the product of the current transmission rate and round-trip time of the data stream, it is directly determined whether the data packet belongs to the last RTT of the stream, and the data packet is marked with a signaling flag to distinguish the data packet from a signaling packet and a non-signaling packet. Specifically, when the number of remaining bytes to be transmitted in the transmission buffer of the data stream to which the data packet belongs is not less than the product of the current transmission rate and the round-trip time of the data stream, the data packet is marked as a signaling packet; otherwise, it is marked as a non-signaling packet.

[0041] (2) Switch congestion detection and token generation and distribution phase: The increase in the length of the switch's port queue within one round-trip time. As a congestion indicator, upon detecting an increase in queue length, the switch generates independent Type I tokens (tasks) and Type II tokens (dbytes) on the port. The Type I token instructs the sender to pause transmission to clear the backlog queue, while the Type II token instructs the sender to reduce the transmission rate so that SWND returns to BDP. The switch appends both Type I and Type II tokens to acknowledgment packets returning along the reverse path, which are then carried and consumed by the acknowledgment packets.

[0042] It should be understood that a switch has multiple ports, and the port queue length for packets waiting to be sent is in bytes. When the queue length is detected to be increasing, a first-type token and a second-type token are generated; conversely, if the queue length does not increase, packets can be forwarded normally without congestion control. The consumption of first-type and second-type tokens allows for the recording of their quantities. When a packet passes through the switch, the number of first-type or second-type tokens in the switch register decreases, and the corresponding value is written to the packet header to indicate that the corresponding first-type or second-type token has been consumed.

[0043] In this embodiment, based on the classification analysis results of congestion sources, congestion can be handled in two scenarios: For congestion caused by the arrival of "mouse flows," since these are all non-signaling packets with a lifetime shorter than one RTT, their arrival only temporarily causes SWND to exceed BDP and creates a backlog within that RTT. This increase automatically cancels out after the flow ends. Therefore, the optimal response is not to reduce speed, but rather a precisely timed pause—pausing for a sufficient duration to clear the backlog, and then immediately resuming transmission at the original sending rate. For congestion caused by the arrival of "elephant flows," the continuous existence of new flows causes SWND to remain greater than BDP. Therefore, it is necessary to both reduce speed to bring SWND back to BDP and pause to quickly clear the queue. For the acceleration of elephant flows, it depends on whether the acceleration occurs in the last RTT. If it occurs in the last RTT, it means the flow is about to leave the network, and its impact on congestion is equivalent to the arrival of "mouse flows." Conversely, if the acceleration of elephant flows does not occur in the last RTT, it will continuously cause an increase in SWND, and its impact on congestion is equivalent to the arrival of elephant flows.

[0044] The aforementioned targeted strategies are unified into a single mechanism based on switch-side tokens, and the quantization operation is performed at the switch rather than the endpoint, to meet the needs of congestion source differentiation and sub-RTT control granularity. Specifically, the switch maintains the state for each port, and the increment is based on the port queue length observed by the switch over one RTT. As a congestion indicator, when detected This indicates that within that RTT, the SWND of all transmitters on the link exceeded the BDP, resulting in... A backlog of bytes. To handle this backlog, the exchange generates two types of tokens simultaneously:

[0045] ① First type of token: its quantity is equal to The first type of token is used to drive the sender to pause transmission to clear the backlog queue. Specifically, the first type of token can only be consumed by the acknowledgment packet corresponding to the signaling packet. When the sender receives an acknowledgment packet carrying tasks from the receiver, it will pause transmission for a period of time equal to the number of first type tokens carried divided by the current transmission rate. The purpose of this pause operation is to quickly clear the backlog of data packets in the network without changing the transmission rate.

[0046] ②Second type of tokens: Their quantity is also equal to The second type of token is used to drive the sender to reduce its transmission rate so that SWND returns to BDP. When the sender receives an acknowledgment packet indicating that the second type of token has been consumed—that is, for every maximum transmission unit (MTU) of second type token consumed—it reduces its transmission rate. Each reduction step is one MTU divided by the RTT, decreasing the transmission rate by one step to ensure that the aggregated transmission window is reduced. This is to restore the match between SWND and BDP.

[0047] Furthermore, the number of Type I and Type II tokens consumed by each data packet and its corresponding acknowledgment packet does not exceed its own byte size.

[0048] Furthermore, both Type 1 and Type 2 tokens are carried and consumed by acknowledgment packets returning along the reverse path. When Type 1 and Type 2 tokens are carried and consumed by acknowledgment packets, the switch prioritizes the allocation of Type 2 tokens to non-signaling packets, ensuring that non-signaling packets can consume Type 2 tokens on both the forward and reverse paths. This significantly reduces overreaction and improves link utilization. Signaling packets, on the other hand, must first acquire an access flag on the forward path before consuming Type 2 tokens on the reverse path. This ensures the accuracy of rate reduction while allowing the incremental transmission window that would otherwise be eliminated due to congestion to be consumed on non-signaling packets about to leave the network, thus suppressing overreaction. Type 1 tokens are only allowed to be consumed by the acknowledgment packets corresponding to the signaling packets. By prioritizing the allocation of Type 2 tokens to non-signaling packets (using the access flag to make the mouse flow consume as many Type 2 tokens it generates as possible, thus making them invisible), unified handling of the impact of both types of congestion is achieved.

[0049] It should be noted that when a new elephant stream joins and causes congestion, the acknowledgment packets corresponding to existing data streams cannot consume the second type of token because they do not carry the access flag on the forward path. Both new and existing streams must first obtain the access flag on the forward path, and then compete fairly to consume the second type of token on the reverse path, thereby ensuring the fairness of rate reduction among data streams.

[0050] The two operations described above (i.e., the slowdown operation and the pause operation) are decoupled from each other: the pause operation is driven independently by the first type of token and aims to quickly eliminate instantaneous backlog; the slowdown operation is driven independently by the second type of token and aims to balance network load in the long term. They execute in parallel and are independent of each other.

[0051] (3) Receiver confirmation packet return stage: After receiving the data packet, the receiver generates the corresponding confirmation packet, copies the token information carried in the data packet to the custom header of the confirmation packet, resets the tasks field in it, and returns the confirmation packet to the sender along the reverse path.

[0052] Furthermore, to support collaborative work between the sender, switch, and receiver, a custom header needs to be inserted into the RoCEv2 format packet header. The custom header is inserted between the User Datagram Protocol (UDP) header and the InfiniteBand (BTH) header. The custom header includes the following fields: a size field (udpsize) (16 bytes) for transmitting the corresponding data packet size in the acknowledgment packet, a token number field (dtask) (16 bytes) for recording the number of Type 1 tokens generated, a Type 1 token field (tasks) (24 bytes) for carrying Type 1 tokens, an acceleration (inc) field (2 bytes) for indicating the acceleration level, a deceleration (dec) field (1 byte) for indicating that Type 2 tokens have been consumed and a deceleration is notified, a user data packet (udp) identifier field (1 byte), an acknowledgment packet (ack) identifier field (1 byte), a signaling flag field (signal) for distinguishing signaling packets from non-signaling packets (1 byte), an access flag field (access) for obtaining data on the forward path (1 byte), and a reserved field (1 byte) for byte alignment. Among them, the dtask field can avoid duplicate responses across switches; the tasks field can instruct the sender to pause transmission; the inc field can notify the sender to increase transmission speed; the dec field can notify the sender to decrease transmission speed; the udp identifier field of 1 can identify the packet as a user packet; the ack identifier field of 1 can identify the packet as an acknowledgment packet; when the signal field is 1, it means that the packet is a signaling packet, and when the signal field is 0, it means that the packet is a non-signaling packet; the access field of 1 means that an access flag has been obtained. Signaling packets can only consume the second type of token in the reverse path after obtaining the access flag on the forward path.

[0053] (4) Sender congestion response phase: When the sender receives an acknowledgment packet carrying a first-type token, it suspends data transmission according to the number of first-type tokens carried to clear the backlog queue in the network; when the sender receives an acknowledgment packet indicating that a second-type token has been consumed, it reduces the transmission rate to bring the aggregated transmission window of all senders on the link (i.e., the sum of the transmission windows of all senders) back to the bandwidth-delay product. Among them, the pause operation to clear the backlog queue and the rate reduction operation to reduce the transmission rate are decoupled from each other (i.e., the two operations are independent of each other). The first-type token drives the transmission pause to clear the backlog queue, and the second-type token drives the rate reduction to bring the aggregated transmission window back to the bandwidth-delay product.

[0054] In some embodiments, to address dynamic situations in real networks such as new flows arriving during a pause and queues not being emptied as scheduled, a task verification mechanism is introduced: the capacity register cap of the switch port and the remaining register rest of the sender collaboratively track the completion status of paused tasks. When the sender resumes transmission, it fills the data packet with a first-type token, and the switch performs task verification accordingly. If the queue has not actually decreased... For the remaining difference, a new first-class token is generated; and the dtask field in the packet header is used to prevent repeated responses from multi-hop switches, and the capacity register is used to reset the threshold K to reset cap in case of abnormalities such as link failure to ensure reliability.

[0055] Furthermore, the task verification mechanism specifically includes: setting a capacity register `cap` for each port of the switch, recording the maximum allowed queue length before generating a new first-type token through the capacity register `cap`, and setting a remaining register `rest` for the sending end; when an increase in the port queue length is detected... When generating bytes When a switch generates a Type 1 token, it increments the value of the capacity register (cap) by the corresponding number of tokens. After receiving an acknowledgment packet carrying a Type 1 token, the sending end pauses transmission and increments the number of Type 1 tokens in the remaining register `rest`. Upon resuming transmission, the sending end fills subsequent data packets with the Type 1 tokens stored in the remaining register `rest`, with each packet having no more than its own byte size filled with Type 1 tokens and the value of the remaining register `rest` decreased accordingly. When the switch receives a data packet carrying a Type 1 token, it subtracts the number of Type 1 tokens carried in the data packet from the value of the capacity register `cap` and compares the current queue length with the value of the capacity register `cap`. If the queue length still exceeds the value of the capacity register `cap`, it continues to generate new Type 1 tokens based on the difference until the queue length is actually reduced by the increase amount. The backlog queue has been cleared.

[0056] Furthermore, the packet header includes a dtask field to record the number of first-type tokens generated. When a packet passes through multiple switches, after the upstream switch has generated a first-type token for the packet, the downstream switch, upon receiving the packet, first increments its capacity register cap by the value of the dtask field before executing the subsequent task verification mechanism. This effectively prevents multiple switches from responding repeatedly to the same congestion.

[0057] Furthermore, the switch counts the number of data packets sent for each port since the last time the capacity register cap was cleared. When this number reaches the preset capacity register reset threshold K and the capacity register cap is still not cleared, that is, the capacity register cap of the port is still not cleared after sending K data packets, the capacity register cap of the port is reset to zero. Here, K is a configurable capacity register reset threshold, which is used to ensure the reliability of the algorithm when there is a link failure or uneven token distribution on multiple equal-cost paths.

[0058] In other embodiments, to achieve precise rate-up, the control method of this invention further includes detecting available bandwidth to increase the transmission rate. Specifically, the switch measures the actual dequeue time interval of adjacent data packets, maps its ratio to a reference interval to several intervals and marks them with a rate-up field in the packet header, and retains only the smaller rate-up level indicating the bottleneck link utilization along the path, so that the sender can precisely increase the rate accordingly, thus tightly coupling SWND and BDP.

[0059] Furthermore, probing available bandwidth to improve transmission rates specifically includes: the switch measuring the actual dequeue time interval between adjacent data packets at ports where queues exist. and compare it with the reference interval. A comparison was made; among which, the actual departure time interval was... The base interval is the difference between the arrival timestamp of the current data packet and the arrival timestamp of the previous data packet. This is the ratio of MTU to link bandwidth. Based on the comparison result, the corresponding acceleration level is marked in the acceleration field of the packet header. As the packet passes through each port, only the smaller acceleration level indicating higher link utilization is retained, allowing the packet to accumulate along the entire path to obtain the acceleration level corresponding to the bottleneck link. When the sending end receives an acknowledgment packet carrying the acceleration field, it increases the transmission rate according to its acceleration level, keeping SWND and BDP tightly coupled, i.e., SWND equals BDP.

[0060] Furthermore, based on the comparison results, the corresponding acceleration level is marked in the acceleration field of the data packet header, specifically including: calculating the actual dequeue time interval. Relative Spacing The ratio is calculated and mapped to several preset intervals. Then, the corresponding acceleration level is marked in the acceleration field of the data packet header according to the interval into which the ratio falls.

[0061] It should be noted that when mapping ratios to preset intervals, the larger the ratio, the larger the corresponding interval, and the higher the speed-up level. The bottleneck link is the most congested link, and also the link with the highest link utilization (when there is no congestion), and its corresponding speed-up level is the lowest. When retaining the smaller speed-up level that indicates higher link utilization, the port calculates its local interval and converts it into the corresponding speed-up level, compares it with the speed-up level currently carried in the packet header, and only retains the smaller one.

[0062] Example 2: Distinguishing between signaling packets and non-signaling packets

[0063] A key aspect of this invention is the distinction between signaling packets and non-signaling packets within data packets. The purpose is to accurately identify which data packets correspond to acknowledgment packets that can effectively carry congestion control instructions.

[0064] like Figure 2 As shown, when sending each data packet, the sending end reads the remaining bytes (remaining_bytes) of the data stream to which that data packet belongs in the sending buffer. Simultaneously, it calculates the product of the current sending rate and the round-trip time (RTT) of the data stream. It then determines whether the data packet belongs to the last RTT of the data stream according to the following formula:

[0065] In the formula, Indicates the number of bytes remaining to be sent; This indicates the current data stream transmission rate, measured in bits per second. This indicates the round-trip time delay, measured in seconds.

[0066] If the number of remaining bytes to be sent is greater than or equal to the product, it means that after the sender sends this data packet, there are still subsequent data packets waiting to be sent within one RTT. When the acknowledgment packet for this data packet is returned, the sender still has the ability to adjust the sending behavior of subsequent data packets according to the instructions in the acknowledgment packet. Therefore, such data packets are marked as signaling packets and the signal flag field in the custom header is set to 1.

[0067] Conversely, if the remaining bytes to be sent are less than this product, it indicates that the data packet belongs to the last RTT (Round-Trip Time) of the data stream. When the acknowledgment packet corresponding to this data packet is returned, the sender has no subsequent data packets to send, making congestion control meaningless. Therefore, such data packets are marked as non-signaling packets, and their signal field is set to 0.

[0068] The significance of distinguishing between signaling packets and non-signaling packets lies in the fact that non-signaling packets are sent at the last RTT of a data stream, after which the data stream will leave the network and no longer affect SWND. Therefore, in congestion handling, non-signaling packets can be considered to have no effect on SWND, while signaling packets will increase SWND. By distinguishing between signaling and non-signaling packets, all data packets in data streams with a lifespan of less than one RTT (i.e., mouse streams) and data packets in the last RTT of long data streams (i.e., elephant streams) can be considered non-signaling packets, thus avoiding unnecessary slowdown responses to these soon-to-disappear traffic in subsequent congestion handling.

[0069] Example 3: Congestion Relief Mechanism – Decoupling of Speed ​​Reduction and Queue Clearing

[0070] The core of this invention lies in decomposing the traditional congestion response into two independent and parallel operations: slowing down and clearing the queue.

[0071] See Figure 3 The switch in steady state, i.e. In the case where an increase in port queue length is observed within one RTT, The number of bytes indicates that SWND exceeds BDP within that RTT. ,lead to Bytes of data are piling up on the network. This is handled in two scenarios, unified by a single token mechanism. The aggregated sending window refers to the total amount of data that all sending ends on the link can send within one RTT, and its unit is bytes. It is the bandwidth-delay product, which is equal to the link bandwidth and the reference round-trip time, and its unit is bytes. This represents the amount of queue length growth observed at a switch port within one round-trip time, expressed in bytes.

[0072] Scenario 1: Congestion caused by rat flows: If the queue growth originates from the arrival of rat flows, these flows can leave the network within the current RTT, and SWND remains unchanged at BDP. In this case, simply pause the sender for a precise period and then resume at the original sending rate. Considering the total backlog is... Bytes, pause duration Obtained using the following formula:

[0073] Pause duration At that time, the queue can be emptied precisely. To achieve sub-RTT control, the switch generates [something] for the corresponding port. The first type of token is consumed by the reverse confirmation packet; the sender pauses the process after receiving the confirmation packet. This reduces SWND to a lower level within the current RTT. This allows the queue to be emptied within that RTT. Only acknowledgment packets corresponding to signaling packets are allowed to consume Type 1 tokens, because allowing non-signaling packets to consume Type 1 tokens does not help reduce congestion and wastes resources.

[0074] Scenario 2: Congestion caused by elephant flow: If the congestion is caused by the arrival of elephant flow, then This state will persist until the stream leaves; simply pausing is insufficient to resolve the issue, and the sending rate also needs to be reduced to recover. Given a total backlog of Bytes, reduce SWND The queue can be emptied within one RTT, at which point the sending rate will be reduced accordingly. Therefore, the switch generates The second type of token is consumed by the reverse acknowledgment packet. To ensure balanced resource allocation, the first and second type tokens consumed by each data packet and its acknowledgment packet do not exceed their own byte size; the udpsize field in the packet header transmits the corresponding data packet size to the acknowledgment packet, and the dec field is used to indicate the consumption of the second type of token and notify the rate reduction. Each time the sender receives an acknowledgment packet carrying the dec flag, it reduces the sending rate according to the following formula:

[0075] In the formula, Indicates the amount of reduction in transmission rate; This represents the maximum transmission unit, which is measured in bytes.

[0076] To unify the two scenarios and achieve a unified control strategy and fairness, a second type of token is also introduced in scenario one. Since all rat flows are non-signaling packets, if the second type of token generated in scenario one can be completely consumed by the rat flow itself, its effect is invisible, thus achieving a unified control method. Therefore, this embodiment stipulates that non-signaling packets can consume the second type of token on both the forward and reverse paths; while signaling packets must first obtain an access flag on the forward path before consuming the second type of token on the reverse path. This access flag also ensures fairness: when an elephant flow joins, the acknowledgment packets of existing flows do not carry an access flag and therefore cannot consume the second type of token; both the new flow and the existing flow must first obtain an access flag on the forward path, and then compete fairly for the second type of token on the reverse path, avoiding excessive slowdown of existing flows due to their acknowledgment packets arriving first.

[0077] Example 4: Network Dynamics and Task Verification Mechanism

[0078] A simplistic design that uses queue growth solely as an indicator of congestion is unreliable in real-world networks: new flows may still arrive when the sender pauses, causing the queue length to stop growing or even decrease, thus increasing the queue length. Pause It may not be possible to truly empty the queue. Therefore, a task verification mechanism was introduced.

[0079] See Figure 3 When the switch generates When the token is of the first type, it means that the current task objective is to reduce the queue length. Upon receiving an acknowledgment packet carrying a Type 1 token, the sending end not only pauses transmission but also adds the received Type 1 tokens to the remaining register `rest`. After resuming transmission, the sending end fills subsequent UDP packets with Type 1 tokens (each packet is filled with Type 1 tokens no larger than its own size and the value of the remaining register `rest` is reduced accordingly) until `rest` is exhausted, indicating that the pause task has been completed. When the switch receives a UDP packet carrying a Type 1 token, it subtracts the number of Type 1 tokens from the value of the capacity register `cap` and checks whether the current queue length still exceeds the value of `cap`, i.e., it checks whether the task has truly been completed (whether the queue has actually decreased). If not completed, new first-class tokens are generated for the remaining difference; this process is called the task verification phase. For example, if the queue length has only decreased... Then it will be generated again. The first type of token. Among them, cap is the switch port capacity register, which represents the maximum queue length allowed before generating a new first type of token, and its unit is bytes; rest is the sender's remaining register, which is used to store the number of first type of tokens to be backfilled, and its unit is bytes.

[0080] To prevent data packets from being repeatedly responded to when passing through multiple switches, the packet header includes a dtask field, which records the number of first-type tokens generated for the data packet. When a downstream switch receives the data packet, it first increments the value of the dtask field by its cap, and then performs a subsequent task verification mechanism check to ensure that data packets for which tokens have already been generated upstream will not have tokens generated repeatedly downstream.

[0081] The processing logic of the congestion elimination mechanism and the task verification mechanism described above is as follows: Figure 3 As shown, when a packet arrives at the switch port, it is first determined whether it is a user data packet. If it is a user data packet, its capacity register `cap` is incremented by the value of the `dtask` field. Then, the subsequent task verification mechanism checks the data, subtracting the number of first-type tokens from the value of the capacity register `cap` based on the number of first-type tokens carried. Next, it checks whether the current queue length still exceeds the value of the capacity register `cap`. If the current queue length exceeds the value of the capacity register `cap`, new first-type tokens are generated for the remaining difference. Specifically, the increment is obtained by subtracting the value of the capacity register `cap` from the current queue length. ,generate One first-class token, simultaneously increasing the capacity register cap. And record the dtask field = Conversely, if the current queue length is less than or equal to the value of the capacity register cap, no token is added. Subsequently, the signal field carried in the data packet is used to determine whether it is a non-signaling packet. If it is a non-signaling packet, the second type of token is consumed directly, and the dec field is set to 1 to notify a reduction in the sending rate. If it is a signaling packet, an access flag is obtained on the forward path, and the second type of token is consumed on the reverse path based on the access flag.

[0082] If the acknowledgment packet is not a user data packet but an acknowledgment packet, the system first checks if the second type of token is sufficient and if the dec field is not set to 1. If the second type of token is sufficient and the dec field is not set to 1, and the acknowledgment packet is not a signaling packet or already has an access flag, then the second type of token is consumed, the dec field is set to 1, and the sender is notified to reduce the sending rate. If the second type of token is insufficient or the dec field is 0, then no rate reduction operation is performed. Next, the system checks if the number of first type of tokens is greater than 0 and if the acknowledgment packet is a signaling packet. If the number of first type of tokens is greater than 0 and the acknowledgment packet is a signaling packet, then the acknowledgment packet carries a tasks field, consumes the first type of token, and reduces the tasks field accordingly. The pause duration is determined by the ratio of the number of first type of tokens carried in the acknowledgment packet to the current sending rate, and the sender's data transmission is paused for the specified pause duration. If the number of first type of tokens is 0 or the acknowledgment packet is not a signaling packet, then the acknowledgment packet does not carry a tasks field, and no pause operation is performed.

[0083] Example 5: Equivalent Paths and Reliability

[0084] When multiple equivalent paths exist in a network, different load balancing methods need to be considered:

[0085] For data flow-based load balancing (such as Equal Cost Multipath (ECMP) load balancing), it is essential to ensure that acknowledgment packets return along the reverse path of their corresponding data packets. Otherwise, the congestion flags carried by the acknowledgment packets will not faithfully reflect the congestion status of the forward path, and the task verification process will be disrupted. To address this, symmetric hashing can be used to ensure that the source, destination, and port addresses exchanged result in the same hash value, thereby forcing both directions to map to the same path.

[0086] For packet-based load balancing (such as random packet spraying RPS), packets from the same data stream are evenly distributed across multiple equivalent paths, with each path having approximately equal load. Therefore, the number of Type I tokens and Type II tokens on each path is also approximately equal, eliminating the need to force acknowledgment packets to follow the same path as data packets. In this case, slight inconsistencies may occur during the task verification phase because the number of tokens on each path is not strictly equal.

[0087] To ensure the robustness of the algorithm under abnormal conditions such as uneven token distribution or token loss due to link failure, this invention also includes a reset mechanism. Specifically, each switch port maintains a counter that counts the number of data packets sent since the last time the port's capacity register cap was cleared. When the counter reaches a preset capacity register reset threshold K (e.g., 1000), if cap is still not cleared, cap is forcibly reset to zero to ensure algorithm reliability. This operation prevents the problem of cap continuously accumulating due to the loss of individual tokens, thus failing to accurately reflect the congestion state.

[0088] Example 6: Detecting Available Bandwidth and Precise Acceleration

[0089] When congestion is relieved or the link is idle, a mechanism is needed to accurately detect available bandwidth and guide the sender to increase its transmission rate, so that the aggregated transmission window remains tightly coupled with BDP. Such a mechanism for detecting available bandwidth to increase the transmission rate could be... Figure 4 As shown.

[0090] See Figure 4 To achieve precise speed-up, link utilization must be measured. Speed-up is only reasonable when all links on the path are underutilized (i.e., no queuing along the path); as long as queuing exists, the links are fully utilized. When a port of a switch has a queue, data packets are dequeued according to the port bandwidth. The dequeueing time interval between adjacent signaling packets is fixed as the baseline interval. This is the ideal dequeue interval for a link when it is fully utilized:

[0091] In the formula, This represents the reference interval, and its unit is seconds. This indicates the link bandwidth, and its unit is bits per second.

[0092] When the link is not fully utilized, an increased dequeue time will be observed. This represents the ideal situation when the link is fully utilized. This dequeue interval reflects the link utilization rate; the ratio of the utilization rate on the bottleneck link (i.e., the actual dequeue interval) to the baseline interval is:

[0093] In the formula, V represents the ratio of the actual outbound time interval to the baseline interval. V≥1. The larger the value of V, the lower the link utilization (there are idle time slots). This represents the actual dequeue time interval between adjacent data packets, and its unit is seconds; This indicates the bottleneck link utilization. When V=2, the link utilization is 50%, meaning that processing one data packet takes twice the base interval—two data packets could have been served within that time period. Therefore, the sender should increase SWND by one MTU when it receives such an acknowledgment packet. Generally, when the dequeue interval is... At that time, the wasted throughput is:

[0094] In the formula, Wasted throughput refers to the number of bytes that could still be transmitted within that time period. Since directly calculating the precise utilization rate on a switch requires division operations, which is inconvenient, this embodiment maps the ratio V to four preset intervals (only two binary comparisons are needed to determine the interval). Each interval corresponds to a rate-up level, and the corresponding rate-up level is marked in the rate-up field of the packet header accordingly. An example of interval division is: when V < 1.05 (i.e., ... When the ratio V falls within interval I (the utilization rate is already high enough, and only a very small increase is needed to maintain network dynamics and fairness); when 1.05 ≤ V < 1.25 (i.e. When 1.25 ≤ V < 1.95, the ratio V falls into interval II; when 1.25 ≤ V < 1.95 (i.e. When V > 1.95, the ratio V falls into interval III; when V ≥ 1.95 (i.e. When the ratio V falls into interval IV, the ratio V is equal to 0.

[0095] Furthermore, since end-to-end path utilization is determined by the bottleneck link, when a data packet passes through each port, the port calculates its local interval and converts it into the corresponding speed-up level. This is compared with the speed-up level currently carried in the packet header, and only the smaller one (i.e., the one indicating lower utilization) is retained. In this way, the minimum speed-up level corresponding to the bottleneck link is accumulated along the entire path.

[0096] After receiving the acknowledgment packet carrying the rate-up field, the sending end increases the transmission rate according to its rate-up level, for example, by... , , Equal to different growth rates, among which When utilization is already high enough, use only very small increments (such as...). (This will be increased to maintain network dynamism and fairness.)

[0097] The processing logic of the above acceleration mechanism is as follows: Figure 4 As shown, when a user data packet is dequeued, the switch measures the actual dequeue time interval between adjacent data packets at the port where the queue exists. ,in, Indicates the arrival timestamp of the current data packet. Indicates the arrival timestamp of the previous data packet, and updates the arrival timestamp of the previous data packet. The baseline interval is calculated based on the maximum transmission unit and the link bandwidth. Calculate the ratio of the actual departure time interval to the baseline interval. The ratio is mapped to four preset intervals. Based on the interval the ratio falls into, the corresponding acceleration level is marked in the acceleration field of the packet header. Specifically, when mapping the ratio to the four preset intervals, V < 1.05 falls into interval I, 1.05 ≤ V < 1.25 falls into interval II, 1.25 ≤ V < 1.95 falls into interval III, and V ≥ 1.95 falls into interval IV. As the data packet passes through each port, the port calculates its local interval and converts it into the corresponding acceleration level. This is compared with the current acceleration level carried in the packet header, and only the smaller one is retained. In this way, the minimum acceleration level corresponding to the bottleneck link can be obtained by accumulating the data packet along the entire path. When the sending end receives an acknowledgment packet carrying the acceleration field, it increases the sending rate according to the acceleration level.

[0098] Example 7: RoCE Network Precise Congestion Control System Based on Congestion Source Differentiation

[0099] See Figure 1 The RoCE network precise congestion control system based on congestion source differentiation of this invention is composed of three parts working together: a transmitter, a switch, and a receiver. The transmitter is responsible for marking and sending data packets, and executing congestion control actions based on feedback. Specifically, the transmitter marks data packets as signaling / non-signaling packets and writes the packet size, sends the data packets according to the calculated rhythm via a rate shaper, and performs pause, deceleration, or acceleration upon receiving an acknowledgment packet carrying two types of tokens. The switch, as the core execution unit, is responsible for detecting congestion, generating tokens, performing task verification, and marking acceleration information. Specifically, the switch maintains the states of two types of tokens and capacity registers in several registers on each port in the programmable data plane, detects and quantifies congestion based on queue length growth, distinguishes congestion sources, generates and verifies tokens, and marks acceleration levels according to the dequeue interval. The receiver is responsible for copying the control information (i.e., the custom header) from the data packets into the acknowledgment packet and returning along the reverse path.

[0100] In this embodiment, the sending end is configured to: when sending a data packet, mark the data packet with a signaling flag according to the remaining transmission status of the data stream to which the data packet belongs, so as to distinguish the data packet into a signaling packet and a non-signaling packet; when receiving an acknowledgment packet carrying a first type of token, suspend data transmission according to the number of first type of tokens carried, so as to clear the backlog queue; and reduce the transmission rate when receiving an acknowledgment packet indicating that a second type of token has been consumed.

[0101] In this embodiment, the switch is used to: use the increase in port queue length within one round-trip time as a congestion indication, and when a queue length increase is detected, generate a first type of token and a second type of token that are independent of each other on the port, and attach the first type of token and the second type of token to the acknowledgment packet returned along the reverse path, which is carried and consumed by the acknowledgment packet.

[0102] In this embodiment, the receiving end is used to copy the custom header carrying token information in the received data packet into the corresponding confirmation packet and return along the reverse path.

[0103] The pause operation to clear the backlog queue and the deceleration operation to reduce the transmission rate are decoupled from each other.

[0104] Example 8: Host-side rate control

[0105] In one embodiment on the host side (sender / receiver), a custom header is placed after the UDP header in the network protocol stack (such as SoftRoCE): A buffer is reserved for this header in the request module, the remaining bytes in the buffer are obtained via the send work queue, signaling flags are marked for packets not in the last RTT, and the packet size is written to the udpsize field. To support pauses and acceleration / deceleration, the sender's host side has a rate shaper in the network protocol stack that precisely controls the packet sending rhythm according to a calculated pace. This rate shaper precisely controls the packet sending rhythm based on received tokens and deceleration / deceleration instructions, and maintains a remaining register rest for each data stream to store the first type of token to be backfilled. Since pauses are performed at the stream granularity, independent rate shaping and queuing are implemented for each stream. When the rest value in the stream context is greater than the size of the next pending UDP packet, the corresponding amount is written to the tasks field of the header, and rest is reduced accordingly. In the response module, the receiver copies the custom header to the acknowledgment packet and resets the tasks field therein.

[0106] The processing logic when the host receives an acknowledgment packet is as follows: if the acknowledgment packet carries a Type I token, then... Pause and increment the number of Type 1 tokens to rest; if a dec field is included, then proceed as follows. Decrease the speed; otherwise, increase the speed by the corresponding step size according to the speed increase level pointed to by the inc field.

[0107] Example 9: Switch-side control

[0108] In one embodiment on the switch side, the switch is a programmable data plane switch, and the method and system described in this invention are implemented on the data plane of the programmable data plane switch. Several registers are allocated to each port, including a register for storing a first type of token, a register for storing a second type of token, a capacity register (cap), a register for recording the arrival time of the previous data packet, and a counter register, which are used to store the first type of token, the second type of token, the capacity register (cap), the arrival time of the previous data packet, and a counter, respectively. Since the reverse acknowledgment packet needs to access and modify these registers, resource contention may occur. Therefore, read and write operations on the above registers are all enclosed in atomic operation blocks to ensure contention-free access and prevent subsequent data packets from reading old values ​​that have not yet been committed.

[0109] When a UDP packet arrives at the switch, the pipeline parses the custom header and updates the capacity register `cap` corresponding to its outgoing port based on the `dtask` and `tasks` fields in the header. If the queue length exceeds `cap`, the first type of token register, the capacity register `cap`, and the second type of token register are updated based on the difference. For acknowledgment packets at the outgoing port, the corresponding register is queried to obtain token resources. Simultaneously, for each UDP packet, the actual dequeue interval is calculated using the arrival timestamp of the current packet and the arrival timestamp of the previous packet. And based on its interval with the reference. The ratio is marked in the inc field of the packet header.

[0110] In the above embodiments, compared to schemes that do not distinguish congestion sources, the present invention can maintain the queue length at a level far lower than existing schemes during sudden congestion and quickly restore link utilization after congestion subsides. For congestion caused by mouse flows, the overreaction of the present invention is far less than that of high-precision schemes that only control link utilization. For congestion caused by elephant flows, the present invention can maintain a steady-state queue of about one packet depth and near-full link utilization. In terms of end-to-end performance, the present invention can significantly reduce the tail flow completion time of small and large flows and greatly reduce the tail queue length in large-scale incast scenarios, thereby verifying the effectiveness of sub-RTT high-precision congestion control based on congestion source differentiation, decoupling of deceleration and queue clearing.

[0111] In summary, this invention, by distinguishing the contribution of congestion sources to the sending window, employs a unique task mechanism to handle burst congestion within the sub-RTT timescale, and allocates switch port queue absorption as an independent task to data packets, significantly reducing overreaction in congestion handling. Furthermore, this invention achieves a balance between the sending window and the bandwidth-delay product by reducing the rate, and clears accumulated data packets in the network by pausing source-end transmission, thus decoupling rate reduction and queue clearing. This invention maintains a slight degree of overreaction, achieving higher precision congestion control. This invention demonstrates superior performance in reducing the flow completion time of various traffic types and shortening the average queue length of switches.

[0112] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A precise congestion control method for RoCE networks based on congestion source differentiation, characterized in that, Applied to RoCE networks including transmitters, switches, and receivers, the method specifically includes: When sending a data packet, the sending end marks the data packet with a signaling flag according to the remaining transmission status of the data stream to which the data packet belongs, so as to distinguish the data packet from the signaling packet and the non-signaling packet. The switch uses the increase in its port queue length over a round-trip time as a congestion indicator. When it detects an increase in the queue length, it generates a first type of token and a second type of token on the port. The first type of token is used to instruct the sender to pause transmission, and the second type of token is used to instruct the sender to reduce the transmission rate. The switch attaches the first type of token and the second type of token to the acknowledgment packet that returns along the reverse path. The acknowledgment packet carries and consumes the acknowledgment packet. After receiving the data packet, the receiving end generates a corresponding acknowledgment packet, copies the token information carried in the data packet into the custom header of the acknowledgment packet, and returns the acknowledgment packet to the sending end along the reverse path; When the sending end receives an acknowledgment packet carrying a first-class token, it pauses data transmission according to the number of first-class tokens carried to clear the backlog queue; when the sending end receives an acknowledgment packet indicating that a second-class token has been consumed, it reduces the transmission rate. The pause operation to clear the backlog queue is decoupled from the speed-down operation to reduce the sending rate.

2. The RoCE network precise congestion control method based on congestion source differentiation according to claim 1, characterized in that, The step of marking the data packet with a signaling flag based on the remaining transmission status of the data stream to which the data packet belongs specifically includes: If the number of remaining bytes to be sent in the transmission buffer of the data stream to which the data packet belongs is not less than the product of the current transmission rate and the round-trip delay of the data stream, the data packet is marked as a signaling packet; otherwise, it is marked as a non-signaling packet.

3. The RoCE network precise congestion control method based on congestion source differentiation according to claim 1, characterized in that, The number of both the first and second type tokens generated by the switch is equal to the growth rate; The duration for which the sending end pauses transmission is the ratio of the number of first-type tokens carried in the acknowledgment packet to the current transmission rate; The step size by which the transmitting end reduces the transmission rate is the ratio of a maximum transmission unit to the round-trip delay; When the first type of token and the second type of token are carried and consumed by the acknowledgment packet, the switch will give priority to the second type of token for consumption by non-signaling packets, so that non-signaling packets can consume the second type of token on both the forward and reverse paths. Each data packet and its corresponding acknowledgment packet consumes no more than the size of its own bytes of first and second type tokens.

4. The RoCE network precise congestion control method based on congestion source differentiation according to claim 1, characterized in that, The custom header includes the following fields: The acknowledgment packet contains a size field for transmitting the corresponding data packet size, a token number field for recording the number of Type 1 tokens generated, a Type 1 token field, an acceleration field for indicating the acceleration level, a deceleration field for indicating that Type 2 tokens have been consumed and a deceleration notification has been issued, a user data packet identifier field, an acknowledgment packet identifier field, a signaling flag field for distinguishing between signaling packets and non-signaling packets, an access flag field for obtaining data on the forward path, and a reserved field for byte alignment.

5. The RoCE network precise congestion control method based on congestion source differentiation according to claim 1, characterized in that, It also includes a task verification mechanism: Configure a capacity register for each port of the switch and a remaining register for the transmitting end; When the switch generates the first type of token, it increments the value of the capacity register by the corresponding number of tokens. When the sending end pauses transmission, it adds the number of received Type I tokens to the remaining register; after the sending end resumes transmission, it fills the Type I tokens stored in the remaining register back into the data packets sent in subsequent transmissions. Each data packet is filled with Type I tokens no larger than its own byte size and the value of the remaining register is reduced accordingly. When a switch receives a data packet carrying a Type 1 token, it subtracts the number of Type 1 tokens carried in the data packet from the value of the capacity register and compares the current queue length with the value of the capacity register. If the queue length still exceeds the value of the capacity register, it continues to generate new Type 1 tokens based on the difference.

6. The RoCE network precise congestion control method based on congestion source differentiation according to claim 5, characterized in that, The header of the data packet contains a token number field for recording the number of first-type tokens generated. When the data packet passes through multiple switches, the downstream switch, upon receiving the data packet, first increments its capacity register by the value of the token number field, and then executes the subsequent task verification mechanism.

7. The RoCE network precise congestion control method based on congestion source differentiation according to claim 5, characterized in that, The switch counts the number of data packets sent for each port since the last time the capacity register was cleared. When the number reaches the preset capacity register reset threshold and the capacity register is still not cleared, the capacity register is reset to zero.

8. The RoCE network precise congestion control method based on congestion source differentiation according to claim 1, characterized in that, It also includes steps to detect available bandwidth in order to increase the transmission rate: The switch measures the actual dequeue time interval between adjacent data packets at the port where the queue exists and compares it with a reference interval; Based on the comparison results, the corresponding acceleration level is marked in the acceleration field of the data packet header, and only the smaller acceleration level indicating higher link utilization is retained when the data packet passes through each port; When the sending end receives an acknowledgment packet carrying an acceleration field, it increases the sending rate according to the acceleration level. Wherein, the actual dequeue time interval is the difference between the arrival timestamp of the current data packet and the arrival timestamp of the previous data packet, and the reference interval is the ratio of the maximum transmission unit to the link bandwidth.

9. The RoCE network precise congestion control method based on congestion source differentiation according to claim 8, characterized in that, The step of marking the corresponding acceleration level in the acceleration field of the data packet header based on the comparison result specifically includes: Calculate the ratio of the actual outbound time interval to the baseline interval, map the ratio to several preset intervals, and then mark the corresponding acceleration level in the acceleration field of the data packet header according to the interval in which the ratio falls.

10. A system for implementing the RoCE network precise congestion control method based on congestion source differentiation as described in any one of claims 1-9, characterized in that, Includes the sending end, the switch, and the receiving end; The sending end is used to: when sending a data packet, mark the data packet with a signaling flag according to the remaining transmission status of the data stream to which the data packet belongs, so as to distinguish the data packet into a signaling packet and a non-signaling packet; when receiving an acknowledgment packet carrying a first type of token, suspend data transmission according to the number of first type of tokens carried, so as to clear the backlog queue; and reduce the transmission rate when receiving an acknowledgment packet indicating that a second type of token has been consumed. The switch is used to: use the increase in port queue length within a round-trip time as a congestion indication, and when a queue length increase is detected, generate independent first and second tokens on the port, and attach the first and second tokens to the acknowledgment packet returned along the reverse path, which is carried and consumed by the acknowledgment packet; The receiving end is used to: copy the custom header carrying token information in the received data packet into the corresponding confirmation packet and return it along the reverse path; The pause operation to clear the backlog queue and the deceleration operation to reduce the transmission rate are decoupled from each other.

Images