A bitmap-based packet loss detection system and method for RDMA networks

Abstract

This invention discloses a bitmap-based packet loss detection system and method for RDMA networks, belonging to the field of remote direct memory access network transmission. To address the problems of coarse-grained packet loss identification, high out-of-order false positive rate, and insufficient coordination efficiency between feedback and retransmission, this invention establishes a sending bitmap at the sending end and a receiving bitmap at the receiving end. The receiving end identifies suspected lost packets based on the state gaps formed by unset bits and subsequently set bits in the receiving bitmap, and makes a judgment based on out-of-order tolerance depth, duration, window stall, and path identifier to distinguish between out-of-order and actual packet loss. When a packet loss is determined, a fast feedback message containing gap location or interval information is generated and sent to the sending end. The sending end locates the corresponding packet in the sending bitmap based on the feedback message and performs single-packet, interval, or strategy-aggregated retransmission. This invention is applicable to high-bandwidth, multi-path, and large-window RDMA networks, improving packet loss detection accuracy, reducing false positive rate, shortening recovery latency, and reducing redundant retransmissions.

Description

Technical Field

[0001] This invention relates to the field of remote direct memory access network transmission technology, and in particular to a bitmap-based RDMA network packet loss detection system and method. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Remote Direct Memory Access (RDMA) networks refer to communication networks that employ RDMA technology. This technology allows network interfaces to directly access data between local and remote memory with minimal host kernel intervention. It features low latency, high throughput, and low CPU overhead, and has been widely applied in data center storage, distributed computing, artificial intelligence training, high-performance computing, and memory pooling scenarios. As network scale expands and link speeds increase, the RDMA operating environment gradually exhibits characteristics such as high bandwidth, high concurrency, large window size, multipathing, and a certain proportion of out-of-order delivery. Reliable connection-based RDMA typically relies on packet sequence numbers, acknowledgment feedback, and retransmission mechanisms to ensure correct data delivery.

[0004] However, the existing solutions have the following shortcomings: (1) The granularity of packet loss identification is relatively coarse: the traditional sequential confirmation and coarse-grained timeout mechanism is difficult to locate the specific missing group in time. Recovery can only be triggered after timeout or abnormal accumulation, resulting in high recovery latency.

[0005] (2) Difficulty in distinguishing out-of-order packets from actual packet loss: In scenarios of link jitter or multi-path forwarding, subsequent packets may arrive before preceding packets, which can easily lead to misjudging short-term out-of-order packets as packet loss, triggering unnecessary feedback and retransmission.

[0006] (3) Insufficient scalability in large window scenarios: Under high bandwidth delay product conditions, the window coverage is large, and the range of packet sequence numbers covered by the sending window and receiving window is large. Traditional packet-based maintenance or sparse state maintenance methods are highly complex in terms of searching, updating and sliding.

[0007] (4) High misjudgment rate in multi-path scenarios: When data packets of the same logical business flow are transmitted through different sub-paths, the time delay difference of each path will lead to cross-path out-of-order. When only maintaining the global order state, cross-path out-of-order is easily misidentified as packet loss.

[0008] (5) Insufficient efficiency of feedback and retransmission coordination: The feedback on suspected missing packets is not precise enough, the retransmission trigger is rough, and it is difficult to achieve accurate recovery of single packets or intervals.

[0009] The root cause of the above problems lies in the fact that traditional mechanisms lack the ability to compactly maintain the state of packets within the window, distinguish between out-of-order packets and packet loss, and have a finely coordinated design for feedback and retransmission, which brings technical challenges to high-throughput and low-latency transmission. Summary of the Invention

[0010] To address the shortcomings of existing technologies, this invention provides a bitmap-based packet loss detection system and method for RDMA networks. A transmitting bitmap is established at the sending end, and a receiving bitmap is established at the receiving end. The receiving end identifies suspected lost packets based on the state gaps formed by unset bits and subsequently set bits in the receiving bitmap. It then uses out-of-order tolerance depth, duration, window stall, and path identifiers to determine whether the packet is out of order or actually lost. When a packet loss is determined, a fast feedback message containing gap location or interval information is generated and sent to the sending end. The sending end locates the corresponding packet in the transmitting bitmap based on the feedback message and performs single-packet, interval, or strategy-aggregated retransmission. This invention is applicable to high-bandwidth, multi-path, and large-window RDMA networks, improving packet loss detection accuracy, reducing false positive rates, shortening recovery latency, and reducing redundant retransmissions.

[0011] On the one hand, a bitmap-based RDMA network packet loss detection system is provided, including: The sender and receiver are connected in a communication manner. The sender is configured to: establish a sending window and initialize a sending bitmap corresponding to the packet sequence number range covered by the sending window; send data packets and update the sending status of the corresponding packets in the sending bitmap according to the packet sequence number of the data packets; receive a fast feedback message from the receiver; locate the corresponding packet status item in the sending bitmap according to the fast feedback message and update the status of the unacknowledged corresponding packets to the pending retransmission status; and retransmit the data packets corresponding to the pending retransmission status. The receiving end is configured to: establish a receiving window and initialize a receiving bitmap corresponding to the packet sequence number range covered by the receiving window; receive data packets and update the set state of the corresponding position in the receiving bitmap according to the packet sequence number of the data packets; identify suspected lost packets based on the state gap formed between the unset position and the subsequent set position in the receiving bitmap; determine the out-of-order tolerance of suspected lost packets based on at least one of the following: out-of-order tolerance depth, duration threshold, window stall threshold, and path identifier; and when determined to be a real packet loss, generate a fast feedback message containing gap position information or interval information and send it to the sending end.

[0012] Furthermore, when there is an unset position in the received bitmap and the subsequent position of that position has been set, a state gap is formed; the state gap is a single-point gap corresponding to a single packet sequence number, or a gap interval composed of multiple consecutive unset positions.

[0013] Furthermore, the out-of-order tolerance determination includes: when the out-of-order depth corresponding to the suspected lost packet does not exceed the preset maximum out-of-order tolerance depth, the duration does not exceed the preset duration threshold, and the degree of window stagnation does not exceed the preset window stagnation threshold, the packet remains in a waiting state and is not immediately determined to be a lost packet; when the out-of-order depth exceeds the preset maximum out-of-order tolerance depth, or the duration exceeds the preset duration threshold, or the degree of window stagnation exceeds the preset window stagnation threshold, the suspected lost packet is determined to be a lost packet event.

[0014] Furthermore, the out-of-order depth is the difference between the sequence number of the largest received packet and the sequence number of the gap start packet; the duration is the time difference between the moment the state gap is first discovered and the current determination moment; and the window stall is the duration during which the receiving window reference packet sequence number cannot continue to advance because the state gap has not been filled, or the number of subsequent packets received during the period when the receiving window has not advanced.

[0015] Furthermore, the receiver supports single-path mode and multi-path mode. In single-path mode, a global bitmap mode is used to uniformly maintain the status of all packets within the window. In multi-path mode, a path-level bitmap mode is used to maintain the corresponding path-level receiving bitmap for each sub-path, and the path identifier is combined to distinguish between cross-path out-of-order packets and actual packet loss.

[0016] Furthermore, the received bitmap is extended using segmented bitmaps, variable-length bitmaps, or a combination of both to adapt to high-bandwidth delay product transmission. Among them, the segmented bitmap divides the logical large window into multiple continuous segments, each segment corresponding to a sub-bitmap; the variable-length bitmap dynamically adjusts the bitmap length according to round-trip delay, link bandwidth, bandwidth-delay product, congestion status, or service load.

[0017] Furthermore, the fast feedback message includes one or more of the following fields: queue pair identifier, gap start packet sequence number, gap length, current receive boundary, current window baseline, timestamp, path identifier, and statistics count.

[0018] Furthermore, a bitmap-based packet loss detection method for RDMA networks is also provided, applied at the receiving end, including: Establish a receiving window and initialize a receiving bitmap corresponding to the packet sequence number range covered by the receiving window; receive data packets and update the set status of the corresponding positions in the receiving bitmap according to the packet sequence number of the data packets; identify suspected lost packets based on the state gap formed between the unset positions and the subsequent set positions in the receiving bitmap; determine the out-of-order tolerance of suspected lost packets based on at least one of the following: out-of-order tolerance depth, duration threshold, window stall threshold, and path identifier; when determined to be a real packet loss, generate a fast feedback message containing gap position information or interval information and send it to the sending end.

[0019] Furthermore, a bitmap-based packet loss detection method for RDMA networks is also provided, applied to the sending end, including: Establish a sending window and initialize a sending bitmap corresponding to the packet sequence number range covered by the sending window; send data packets and update the sending status of the corresponding packets in the sending bitmap according to the packet sequence number of the data packets; receive a fast feedback message from the receiving end; locate the corresponding packet status item in the sending bitmap according to the fast feedback message, and update the status of the unacknowledged corresponding packets to the pending retransmission status; retransmit the data packets corresponding to the pending retransmission status.

[0020] Furthermore, a computer-readable storage medium is provided on which a computer program is stored, which, when executed by a processor, performs the method of the first aspect.

[0021] The above technical solution has the following advantages or beneficial effects: (1) A bitmap with the same width as the receiving window is used to directly index and compactly represent the arrival status of packets within the window, which can accurately locate missing packets at the bit level and overcome the problems of coarse packet loss identification and high recovery delay of traditional sequential confirmation and timeout mechanisms.

[0022] (2) By introducing a multi-dimensional out-of-order tolerance judgment mechanism based on out-of-order tolerance depth, duration, window stagnation and path identification, the gap waiting state is maintained when short-term out-of-order occurs without immediately triggering retransmission, which effectively distinguishes out-of-order from real packet loss and significantly reduces false alarm rate and unnecessary retransmission overhead.

[0023] (3) In multi-path scenarios, by maintaining a path-level bitmap and combining it with path identifiers, it is possible to accurately determine cross-path out-of-order packets and actual packet loss, avoiding misjudgments caused by relying solely on the global order state. When the fast feedback condition is met, the receiver generates a fine-grained fast feedback message carrying the gap start packet sequence number and gap length, which can notify the sender without waiting for coarse-grained timeout, thereby reducing the packet loss recovery delay to the microsecond level. The sender locates the corresponding packet status item in the sending bitmap based on the gap start packet sequence number, gap length, or gap interval information in the feedback; if the corresponding packet has not been acknowledged, it is updated from the sent but unacknowledged state or suspected loss state to the pending retransmission state, and single packet retransmission, interval retransmission, or strategy aggregation retransmission is performed, and duplicate, expired, and acknowledged retransmission requests are suppressed and deduplicated, reducing redundant data transmission and improving network bandwidth utilization.

[0024] (4) For high bandwidth delay product and large window scenarios, the expansion method of combining segmented bitmap and variable length bitmap is adopted, which can dynamically adapt to link bandwidth, round-trip delay and congestion status, ensuring the scalability and state maintenance efficiency of the system under large window conditions.

[0025] In summary, this invention provides an efficient, accurate, and low-false-error packet loss detection and rapid recovery solution for high-bandwidth, high-concurrency, multi-path, and large-window RDMA networks. Attached Figure Description

[0026] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0027] Figure 1 The present invention provides an overall architecture diagram of a bitmap-based RDMA network packet loss detection system; Figure 2 The present invention provides an execution flowchart of a bitmap-based RDMA network packet loss detection system; Figure 3 This invention provides a receiver bitmap state management and gap identification structure diagram; Figure 4 This invention provides a flowchart of the collaborative process for feedback processing and retransmission at the transmitting end. Figure 5 This invention provides a schematic diagram of path-level bitmap maintenance in a multi-path scenario; Figure 6 This invention provides a schematic diagram of the state expansion of a segmented bitmap or variable-length bitmap in a high-bandwidth delay product scenario. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings. Those skilled in the art should understand that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0029] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0030] Example 1 like Figure 1As shown, this embodiment provides a bitmap-based RDMA network packet loss detection system. The system includes a transmitter and a receiver, and the transmitter and receiver are connected in communication. The transmitter is configured to: establish a transmission window and initialize a transmission bitmap corresponding to the packet sequence number range covered by the transmission window; transmit data packets and update the transmission status of the corresponding packets in the transmission bitmap according to the packet sequence number of the data packets; receive a fast feedback message from the receiver; locate the corresponding packet status item in the transmission bitmap according to the fast feedback message, and update the status of unacknowledged corresponding packets to a pending retransmission status; and retransmit the data packets corresponding to the pending retransmission status. The receiving end is configured to: establish a receiving window and initialize a receiving bitmap corresponding to the packet sequence number range covered by the receiving window; receive data packets and update the set state of the corresponding position in the receiving bitmap according to the packet sequence number of the data packets; identify suspected lost packets based on the state gap formed between the unset position and the subsequent set position in the receiving bitmap; determine the out-of-order tolerance of suspected lost packets based on at least one of the following: out-of-order tolerance depth, duration threshold, window stall threshold, and path identifier; and when determined to be a real packet loss, generate a fast feedback message containing gap position information or interval information and send it to the sending end.

[0031] In this embodiment, the system adopts a layered architecture with a collaborative control management layer, a sending protocol processing layer, a receiving protocol processing layer, and a data forwarding layer. Specifically, the control management layer interacts with the sending and receiving protocol processing layers via configuration / query channels; the sending and receiving protocol processing layers interact via RDMA data streams and acknowledgment / fast feedback channels; and the data forwarding layer is used for the underlying forwarding of service data.

[0032] The sending end protocol processing layer includes a sending status management module and a retransmission coordination module. The sending status management module internally sets up a sending bitmap to maintain the sending status, acknowledgment status, and pending retransmission status of packets within the sending window; the retransmission coordination module is used to trigger fine-grained retransmissions based on feedback results.

[0033] The receiver protocol processing layer includes a receiver status management module and a feedback generation module. The receiver status management module internally sets up a receiver bitmap and can optionally set a path-level bitmap to maintain the arrival status, gap status, and path-related status of packets within the receiver window. The feedback generation module is used to generate acknowledgment information or rapid feedback information based on the bitmap status.

[0034] In this embodiment, the transmit bitmap and receive bitmap are the basic capability model, while the path-level bitmap is the enhanced capability model. The system can enable path-level bitmap, segmented bitmap, or variable-length bitmap depending on multipath and high-bandwidth delay product scenarios.

[0035] The execution flow of the bitmap-based RDMA network packet loss detection system provided in this embodiment will be further explained below with reference to the accompanying drawings. Figure 2 As shown, Figure 2 This is a flowchart of the system execution process.

[0036] Specifically, this includes S101, initializing the bitmap. For the queue pairs in the target RDMA connection, the transmitting end establishes a transmitting window, determines the starting packet sequence number, window length, and sliding step size of the transmitting window, and initializes the transmitting bitmap corresponding to the range of packet sequence numbers covered by the transmitting window. Simultaneously, the receiving end establishes a receiving window, determines the starting packet sequence number, window length, and sliding step size of the receiving window, and initializes the receiving bitmap corresponding to the range of packet sequence numbers covered by the receiving window. Each status bit or status item in the transmitting bitmap is mapped to one or more consecutive packet sequence numbers, used to record the acknowledgment status, suspected loss status, pending retransmission status, retransmission pending acknowledgment status, or completed status of transmitted packets on the transmitting end side. Each bit in the receiving bitmap is mapped to one or more consecutive packet sequence numbers; initially, all bits are set to 0, used to record the arrival status of each data packet on the receiving end side.

[0037] S102, Data packet transmission and reception status update. During data transmission, the sending end updates the corresponding status items in the transmission bitmap in real time based on the transmission, acknowledgment, fast feedback, or retransmission processing results of data packets; the receiving end updates the corresponding set status in the receiving bitmap based on the arrival status of data packets. By having the sending and receiving ends maintain their respective bitmap structures, a collaborative foundation is established for subsequent status gap identification, out-of-order tolerance determination, fast feedback generation, retransmission location, and window advancement.

[0038] The preset mapping rule is used to calculate the offset position of the data packet in the bitmap based on the current window base packet sequence number, the data packet sequence number, and the preset sequence number mapping granularity. Specifically, the following parameters are defined: BasePSN represents the starting packet sequence number of the current receiving window, PSN represents the packet sequence number of the currently received data packet, G represents the mapping granularity (i.e., the number of consecutive packet sequence numbers corresponding to one bit in the receiving bitmap), and L represents the length of the receiving bitmap. The receiving end calculates the target position of the packet in the receiving bitmap using the following steps: First, determine whether the PSN falls within the current receiving window, that is, whether BasePSN <= PSN < BasePSN + WindowSize is satisfied; if satisfied, calculate the distance Delta = PSN - BasePSN of the packet relative to the window start point; then calculate the bitmap offset Offset = floor(Delta / G) according to the mapping granularity; finally, take the bit corresponding to this Offset as the target position RBM[Offset].

[0039] When the sender sends a data packet, it obtains the packet sequence number of the data packet, and maps the data packet to the corresponding status item in the sending bitmap according to the window base packet sequence number of the sending window and the preset mapping rule. After sending, update the corresponding status item to the "sent but unacknowledged state", indicating that the data packet has been sent but no acknowledgement feedback has been received yet.

[0040] When the receiver receives a valid data packet, it obtains the packet sequence number of the data packet, and maps the data packet to the corresponding position in the receiving bitmap according to the window base packet sequence number of the receiving window and the preset mapping rule. If this position was not set before, set it, indicating that the data packet has arrived; if this position was already set, mark the data packet as a repeatedly arrived packet, and update the repeated statistics information. The repeated statistics information includes one or more of the repeated packet count, the packet sequence number corresponding to the repeated packet, the repeated arrival timestamp, the queue pair identifier, and the path identifier.

[0041] After completing the update of the receiving bitmap, the receiver further detects whether there is a continuous and complete set interval starting from the starting position of the current receiving window, that is, whether multiple consecutive bitmap positions starting from the receiving window base packet sequence number are in the set state. If there is a continuous and complete interval, the receiver takes this interval as the pushable interval, advances the receiving window base packet sequence number, and releases, clears or reuses the historical bitmap state corresponding to the completed interval to record the subsequent arriving data packets.

[0042] Correspondingly, when the sender receives the acknowledgement feedback returned by the receiver, it updates the acknowledgement status of the corresponding packet in the sending bitmap according to the acknowledgement boundary, acknowledgement interval or packet sequence number information in the acknowledgement feedback, and advances the sending window.

[0043] S103: State Gap Identification and Suspected Lost Packet Determination. Specifically, the receiving end continuously scans the receiving bitmap corresponding to the current receiving window. When an unset position is found in the receiving bitmap, and there is at least one subsequent set position in the increasing direction of the packet sequence number corresponding to the unset position, a state gap is determined to have formed at the unset position. A state gap can be a single point gap corresponding to a single unset position, or a gap interval composed of multiple consecutive unset positions. Each state gap corresponds to one or more suspected missing data packets and is identified as a suspected lost packet.

[0044] like Figure 3 As shown, the receiving end maintains a receive bitmap (RBM), which corresponds to the range of packet sequence numbers covered by the current receive window. Let the base packet sequence number of the current receive window be... The length of the receiving window is The preset mapping granularity is The range of packet sequence numbers covered by the current receiving window is: When the receiving end receives a data packet with sequence number PSN, if the PSN falls within the current receiving window, its offset position in the receiving bitmap is calculated according to a preset mapping rule, for example: The receiving end updates the set state of the corresponding position in the received bitmap RBM according to the offset position.

[0045] If there is a continuous set prefix interval in the RBM starting from the position corresponding to the BasePSN, the prefix interval is considered to be a continuous complete interval. The receiver can advance the receiver window base packet sequence number accordingly and release, clear, or reuse the bitmap space corresponding to the completed interval.

[0046] If an unset position exists in the received bitmap (RBM), and a set position already exists after that unset position, it indicates that subsequent data packets have arrived, while the data packet corresponding to the unset position has not yet arrived. The receiver thus forms a state gap. To facilitate subsequent processing, the receiver can create a gap record entry for the state gap. This gap record entry includes one or more of the following: gap start packet sequence number, gap length, first detection time, current gap status, path identifier, out-of-order depth, window stall information, and triggered feedback flags.

[0047] This state gap is used for subsequent out-of-order tolerance determination and fast feedback generation. When this state gap is determined to be a real packet loss event, the receiver generates a fast feedback message based on the gap record and sends it to the sender. This allows the sender to locate the corresponding packet state entry in its transmission bitmap based on the fast feedback message and perform subsequent retransmission processing.

[0048] S104, in order to avoid misjudging late-arriving packets caused by link jitter, multipath transmission, or short-term out-of-order delivery as real packet loss, the receiving end combines at least one of the following information: out-of-order tolerance depth, gap duration, window stagnation degree, and path identifier to determine the out-of-order tolerance of suspected packet loss packets identified in S103.

[0049] Specifically, out-of-order tolerance depth is a measure of the extent to which a state gap has been crossed, even if the gap has not yet been filled. For example, the number of subsequent packets that have arrived after the state gap can be used as the out-of-order depth; or, the difference between the sequence number of the currently received largest packet and the sequence number of the gap's starting packet can be used as the out-of-order depth; gap duration refers to the time difference between the moment the state gap is first discovered and the moment of the current determination; window stall degree refers to the duration during which the receiver window's baseline packet sequence number cannot continue to advance due to the unfilled state gap, or the number of subsequent packets received during the period when the receiver window has not advanced; path identifier refers to information used to distinguish the sub-path, forwarding path, or equivalent path to which the data packet belongs, used to determine whether the gap may be caused by cross-path out-of-order delivery.

[0050] The receiving end makes judgments based on a preset maximum out-of-order tolerance depth, a preset packet loss judgment waiting time, and a preset window stagnation threshold: When the out-of-order tolerance depth does not exceed the preset maximum out-of-order tolerance depth, the gap duration does not exceed the preset packet loss judgment waiting time, and the window stagnation degree does not exceed the preset window stagnation threshold, the state gap is maintained in a waiting state and is not immediately judged as a real packet loss. When the out-of-order tolerance depth exceeds the preset maximum out-of-order tolerance depth, or the gap duration exceeds the preset packet loss judgment waiting time, or the window stagnation degree exceeds the preset window stagnation threshold, or when it is determined based on the path identifier that subsequent packets on the same path have arrived but the state gap has not yet been filled, the suspected packet loss packet corresponding to the state gap is judged as a real packet loss event.

[0051] For example, when setting the maximum out-of-order tolerance depth to 8 packets, the packet loss determination waiting time to 50 microseconds, and the window stall threshold to 80 microseconds, if only 3 subsequent packets arrive after a gap, and the gap duration and window stall time are both 20 microseconds, the receiver maintains the gap waiting state. If 9 subsequent packets arrive after the gap, or the gap duration exceeds 50 microseconds, or the window baseline stalls for more than 80 microseconds due to the gap, the receiver upgrades the suspected packet loss to a real packet loss event. In multi-path scenarios, if subsequent arriving packets come from different paths, the waiting threshold can be appropriately relaxed; if subsequent packets on the same path have arrived consecutively and the gap has not yet been filled, it can be determined as a packet loss event in advance.

[0052] When a suspected packet loss is determined to be a genuine packet loss event, the receiving end generates a fast feedback message. The fast feedback message includes at least gap location information or gap interval information. It may also include one or more of the following: queue pair identifier, gap start packet sequence number, gap length, current receive boundary, current window baseline packet sequence number, timestamp, path identifier, out-of-order depth, gap status code, and statistical count.

[0053] It should be noted that when a continuous and complete interval is formed in the received bitmap and the received window is advanced, the receiving end generates a regular acknowledgment message to enable the sending end to advance the sending window normally; however, when a state gap that meets the packet loss judgment condition is detected, the receiving end generates a fast feedback message and sends the message to the sending end so that the sending end can locate the corresponding packet state item in the sending bitmap based on the fast feedback message and perform retransmission.

[0054] S105, after receiving the fast feedback message sent by the receiver, the sending end parses the gap location information or gap interval information in the fast feedback message, and locates the corresponding packet status item in the sending bitmap according to the gap start packet sequence number, gap length or gap interval.

[0055] If the corresponding packet status item is in the sent but unacknowledged state or the suspected lost state, and has not yet been acknowledged, the sender updates the packet status item to the pending retransmission state and works with the retransmission coordination module to locate and schedule the packet to be retransmitted; if the corresponding packet status item is already in the acknowledged state or the completed state, the sender ignores the retransmission request corresponding to the fast feedback message to avoid redundant retransmission of already acknowledged or completed packets.

[0056] The retransmission coordination module performs single-packet retransmission, interval retransmission, or strategy-aggregated retransmission based on the retransmission status in the transmission bitmap. Single-packet retransmission refers to retransmitting only a single missing packet; interval retransmission refers to retransmitting multiple consecutive missing packets; strategy-aggregated retransmission refers to the sender merging multiple discrete gaps or multiple fast feedback requests into one or a few retransmission operations based on gap adjacency, feedback arrival time, retransmission priority, congestion control status, or retransmission window limitations, thereby reducing redundant scheduling and retransmission overhead.

[0057] After retransmission, the sending end can update the status of the corresponding packet from "Pending Retransmission" to "Retransmitted but Awaiting Acknowledgment." When it subsequently receives acknowledgment feedback from the receiving end, the sending end updates the status of the corresponding packet in the transmission bitmap to "Acknowledged" or "Completed" based on the acknowledgment boundary, acknowledgment interval, or packet sequence number information, and advances the transmission window.

[0058] In addition, the sending end also performs the following collaborative processing: when receiving confirmation feedback, according to the confirmation boundary or confirmation interval, the status of the corresponding packet in the sending bitmap is updated from sent but not confirmed or retransmitted and awaiting confirmation to confirmed or completed, and the corresponding window status is released or reused; when a packet confirmation timeout is detected or the sending window has not advanced for a long time, the corresponding sent but not confirmed status can be updated to suspected loss status or awaiting retransmission status to trigger timeout retransmission.

[0059] Meanwhile, the sending end suppresses and deduplicates duplicate feedback, expired feedback, retransmission requests corresponding to confirmed packets, and duplicate trigger requests for the same gap within a set time window. When the sending end is under congestion control constraints, it incorporates retransmission packets and normal transmission packets into the current queue pair or the transmission rate, transmission window, congestion window, or token bucket constraints of the service flow, so that retransmission traffic and normal transmission traffic are jointly restricted by the congestion control strategy, avoiding retransmission traffic from disrupting the existing congestion control mechanism.

[0060] Understandably, the system executes S102 to S105 in a loop to achieve continuous packet loss detection, rapid feedback, accurate retransmission, and window advancement.

[0061] like Figure 4 As shown, when the receiver identifies a gap that meets the fast feedback condition, it generates a fast feedback message (FFM). The fast feedback message may carry one or more of the following fields: queue pair (QP) identifier, current receive boundary, gap start packet sequence number (PSN), gap length, gap status code, timestamp, path identifier, and one or more of the accumulated repetition count or out-of-order depth.

[0062] Upon receiving the fast feedback message, the sending end retrieves the corresponding packet status from the transmission bitmap. If the packet corresponding to the gap has not yet been acknowledged and is not in the completed retransmission state, it is set to the pending retransmission state, and the retransmission action is performed. To avoid excessive retransmissions, the following control strategy can be adopted: (1) Do not retransmit packets that have already received confirmation information; (2) Only one valid retransmission trigger is allowed for the same gap within the set time window; (3) Multiple adjacent gaps are merged and inter-interval retransmission is used; (4) When congestion control is in effect, retransmission traffic is included in the overall transmission rate constraint to prevent retransmission traffic from disrupting the existing congestion control strategy. "When congestion control is in effect" means that the target RDMA connection has enabled congestion control or rate control mechanisms, and the sender is in a congestion-constrained transmission state due to receiving congestion indications, fast feedback, acknowledgment stagnation, retransmission timeouts, or limitations on transmission rate or transmission window. Specifically, the sender integrates retransmission packets and normal transmission packets into the current QP or service flow's transmission rate, transmission window, and congestion window constraints, ensuring that retransmission traffic and normal transmission traffic are both subject to congestion control mechanisms.

[0063] Through the above mechanism, the system can perform single packet retransmission, interval retransmission, or strategy aggregation retransmission based on rapid feedback, confirmation of stagnation, timer events, or management plane trigger results. It can also suppress and deduplicate retransmission requests corresponding to duplicate feedback, expired feedback, and confirmed packets, thereby achieving refined retransmission coordination.

[0064] Understandably, the above steps are repeated during subsequent continuous data packet transmission to achieve continuous bitmap updates, gap identification, feedback generation, and retransmission processing.

[0065] For different network scenarios, this invention provides corresponding optimized implementation methods.

[0066] In a single-path scenario, the receiving end uses a global bitmap mode to uniformly maintain the status of all packets within the window; like Figure 5 As shown, in a multi-path RDMA scenario, data packets of the same logical service flow may be forwarded to the receiving end via multiple different sub-paths. Due to differences in propagation delay, queuing delay, and congestion status on different sub-paths, the arrival order of packets may change. To address this, this embodiment further proposes a path-level bitmap mechanism: maintaining a corresponding path-level receive bitmap for each sub-path. Each path-level bitmap corresponds to the same logical packet sequence number space, but records the arrival status of packets on different paths.

[0067] For example, when the receiving end receives a path identifier When grouping data, not only is the global bitmap (GBM) updated, but also the data grouping process is updated. Corresponding path-level bitmap If a packet forms a gap in the global bitmap, but the adjacent subsequent packets of that gap come from other paths, it is preferentially determined as cross-path out-of-order rather than immediately considered lost. When path-level bitmap capabilities are unavailable, path identifiers are missing, or the device does not enable this function, it can degenerate into performing detection using only the global bitmap mode.

[0068] Through the aforementioned path-level bitmap mechanism, this invention effectively distinguishes between cross-path out-of-order packets and actual packet loss in multipath transmission, further reducing the false positive rate.

[0069] like Figure 6 As shown, in high-bandwidth delay product scenarios, since the sending and receiving windows may cover a large number of packet sequence numbers, using a fixed-length single-piece bitmap can easily lead to problems such as excessively long bitmaps, increased access latency, and excessive state storage. Therefore, this embodiment proposes using segmented bitmaps, variable-length bitmaps, or a combination of both to extend the bitmap, thereby improving window coverage and state maintenance scalability.

[0070] Segmented bitmaps divide a large logical window into multiple consecutive segments, each corresponding to a sub-bitmap. Data packets first locate their respective segments based on their packet sequence numbers, then determine their offset positions within those segments, thereby reducing the complexity of a single access and facilitating parallel processing in hardware pipelines. Variable-length bitmaps dynamically adjust their length based on round-trip latency, link bandwidth, bandwidth-delay product, congestion status, or service load: shorter bitmaps can be used to conserve resources when network load is low or the window is small; when facing high bandwidth-delay product scenarios, extended bitmap lengths can be added to the base bitmap length. This ensures sufficient detection coverage. For ultra-large window scenarios, the number of segments can be dynamically adjusted based on the total window length, and a fixed or adaptive length bitmap structure can be maintained within each segment to achieve better scalability.

[0071] In some implementations, the system also supports management and monitoring capabilities for operation and maintenance deployment and policy control. Configurable parameters include, but are not limited to: base bitmap length and extended bitmap length. The parameters include: bitmap segment size, window start packet sequence number (PSN), window sliding step size, maximum out-of-order tolerance depth, packet loss judgment waiting time, fast feedback trigger threshold, single queue QP state limit, maximum number of concurrent queue QPs per device, and number of path-level bitmap instances.

[0072] It can also provide statistical information such as the currently enabled bitmap mode, the current base bitmap length, and the current extended bitmap length. The data includes the current actual bitmap length, number of segments, number of packet loss events, number of strategy aggregation retransmissions, number of retransmissions, number of duplicate groupings, number of misjudgment corrections, statistics on different paths, and resource usage.

[0073] In addition, as an optional enhancement to this system, a management and monitoring module can be set up. This module is located in the control and management layer and communicates bidirectionally with both the sending and receiving protocol processing layers. It is used to complete parameter configuration, status query, capability discovery, event logging, statistical export, and alarm management. Specifically, the management and monitoring module supports the configuration and querying of parameters such as bitmap length, bitmap segment size, window start packet sequence number, window sliding step size, fast feedback trigger threshold, packet loss judgment waiting time, maximum out-of-order tolerance depth, number of path-level bitmap instances, single queue pair status limit, and maximum number of concurrent queue pairs per device. At the same time, this module can provide statistical information, including the currently enabled bitmap mode, current bitmap length and number of segments, number of packet loss events, number of policy aggregation retransmissions, number of retransmissions, number of duplicate groupings, number of misjudgment corrections, statistical information of different paths, and resource usage.

[0074] It should be noted that this management and monitoring module is not a core module necessary for packet loss detection and retransmission. Its main function is for system operation and maintenance and policy optimization. In actual deployment, it can be enabled or disabled according to needs.

[0075] An exemplary application scenario.

[0076] To make the technical solution of the present invention clearer, the following description takes the RDMA reliable connection data transmission scenario as an example, but the present invention is not limited thereto.

[0077] In this scenario, the sending end establishes a sending window based on the target queue pair (QP) and initializes a sending bitmap. This sending bitmap is used to record the acknowledgment status, suspected loss status, retransmission status, or pending status of sent packets. The sending end maintains information such as sent packets, acknowledged packets, suspected lost packets, packets awaiting retransmission, and completed packets through the sending bitmap. The receiving end establishes a receiving window based on the same QP and initializes a receiving bitmap to record the arrival status of each data packet.

[0078] When the sending end sends continuously to After the data is grouped, the corresponding status item in the sent bitmap is updated to "sent but unacknowledged". The receiving end receives... Then, the corresponding bit in the received bitmap is set, and the information is identified. A state gap occurs when a corresponding position is not set while a subsequent position is set. If this gap is escalated to a packet loss event after out-of-order tolerance determination, the receiver generates a packet loss event containing the start of the gap. And quick feedback messages on the gap length.

[0079] After receiving the fast feedback message, the sending end retrieves the corresponding status item in the sending bitmap based on the gap position or interval information. If If a packet's status is "sent but unacknowledged" or "suspected lost," and it has not yet been acknowledged, it is updated to "pending retransmission," and the retransmission coordination module locates and retransmits the packet. After retransmission, it is updated to "retransmitted but awaiting acknowledgment." The receiving end receives the retransmitted packet. Then, the corresponding position in the receiving bitmap is set, and the receiving window is advanced after a continuous and complete interval is formed. At the same time, an acknowledgment feedback is sent back to the sending end. After receiving the acknowledgment feedback, the sending end updates the status of the corresponding group in the sending bitmap to the acknowledged or completed status and advances the sending window.

[0080] Through the above process, the receiving bitmap is responsible for recording the arrival status, identifying status gaps and triggering rapid feedback, while the sending bitmap is responsible for recording the confirmation status, suspected loss status, retransmission status and pending status, and updating the status according to the confirmation feedback or rapid feedback. Together with the retransmission coordination module, it completes the positioning of pending groups, accurate retransmission and window advancement.

[0081] Example 2 This embodiment provides a bitmap-based packet loss detection method for RDMA networks, applied at the receiving end, including: Establish a receiving window and initialize a receiving bitmap corresponding to the packet sequence number range covered by the receiving window; receive data packets and update the set status of the corresponding positions in the receiving bitmap according to the packet sequence number of the data packets; identify suspected lost packets based on the state gap formed between the unset positions and the subsequent set positions in the receiving bitmap; determine the out-of-order tolerance of suspected lost packets based on at least one of the following: out-of-order tolerance depth, duration threshold, window stall threshold, and path identifier; when determined to be a real packet loss, generate a fast feedback message containing gap position information or interval information and send it to the sending end.

[0082] Example 3 This embodiment provides a bitmap-based packet loss detection method for RDMA networks, applied at the sending end, characterized by including: Establish a sending window and initialize a sending bitmap corresponding to the packet sequence number range covered by the sending window; send data packets and update the sending status of the corresponding packets in the sending bitmap according to the packet sequence number of the data packets; receive a fast feedback message from the receiving end; locate the corresponding packet status item in the sending bitmap according to the fast feedback message, and update the status of the unacknowledged corresponding packets to the pending retransmission status; retransmit the data packets corresponding to the pending retransmission status.

[0083] Example 4 This embodiment also provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the method of Embodiment 1.

[0084] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A bitmap-based RDMA network packet loss detection system, characterized in that, include: The sender and receiver are connected in a communication manner. The sender is configured to: establish a sending window and initialize a sending bitmap corresponding to the packet sequence number range covered by the sending window; Send data packets and update the sending status of the corresponding packets in the sending bitmap according to the packet sequence number of the data packets; Receive rapid feedback messages from the receiving end; Based on the rapid feedback message, locate the corresponding packet status item in the sending bitmap and update the status of the unacknowledged corresponding packet to the pending retransmission status; Retransmit the data packets corresponding to the retransmission status; The receiver is configured to: establish a receive window and initialize a receive bitmap corresponding to the packet sequence number range covered by the receive window; The system receives data packets and updates the set status of the corresponding position in the received bitmap according to the packet sequence number of the data packets. Based on the state gap formed between the unset position and the subsequent set position in the received bitmap, it identifies suspected lost packets. Based on at least one of the following information, out-of-order tolerance depth, duration threshold, window stall threshold and path identifier, it determines the out-of-order tolerance of the suspected lost packets. When it is determined to be a real packet loss, it generates a fast feedback message containing gap position information or interval information and sends it to the sender.

2. The system according to claim 1, characterized in that, A state gap is formed when there is an unset position in the received bitmap and the subsequent position of that position is set. A state gap is a single-point gap corresponding to a single packet sequence number, or a gap interval composed of multiple consecutive unset positions.

3. The system according to claim 1, characterized in that, Out-of-order tolerance determination includes: when the out-of-order depth corresponding to the suspected lost packet does not exceed the preset maximum out-of-order tolerance depth, the duration does not exceed the preset duration threshold, and the degree of window stagnation does not exceed the preset window stagnation threshold, the packet remains in a waiting state and is not immediately determined to be a lost packet; when the out-of-order depth exceeds the preset maximum out-of-order tolerance depth, or the duration exceeds the preset duration threshold, or the degree of window stagnation exceeds the preset window stagnation threshold, the suspected lost packet is determined to be a lost packet event.

4. The system according to claim 3, characterized in that, Out-of-order depth is the difference between the sequence number of the largest received packet and the sequence number of the gap start packet; duration is the time difference between the moment the state gap is first discovered and the moment of current determination; window stall is the duration during which the receive window reference packet sequence number cannot continue to advance because the state gap has not been filled, or the number of subsequent packets received during the period when the receive window has not advanced.

5. The system according to claim 1, characterized in that, The receiver supports single-path and multi-path modes. In single-path mode, a global bitmap mode is used to uniformly maintain the status of all packets within the window. In multi-path mode, a path-level bitmap mode is used to maintain the corresponding path-level receive bitmap for each sub-path, and the path identifier is combined to distinguish between cross-path out-of-order packets and actual packet loss.

6. The system according to claim 1, characterized in that, The received bitmap is extended using segmented bitmaps, variable-length bitmaps, or a combination of both to adapt to high-bandwidth delay product transmission. The segmented bitmap divides the logical large window into multiple continuous segments, each segment corresponding to a sub-bitmap. The variable-length bitmap dynamically adjusts the bitmap length based on round-trip delay, link bandwidth, bandwidth-delay product, congestion status, or service load.

7. The system according to claim 1, characterized in that, The quick feedback message includes one or more of the following fields: queue pair identifier, gap start packet sequence number, gap length, current receive boundary, current window baseline, timestamp, path identifier, and statistics count.

8. A bitmap-based packet loss detection method for RDMA networks, applied at the receiving end, characterized in that, include: Establish a receive window and initialize the receive bitmap corresponding to the packet sequence number range covered by the receive window; The system receives data packets and updates the set status of the corresponding position in the received bitmap according to the packet sequence number of the data packets. Based on the state gap formed between the unset position and the subsequent set position in the received bitmap, it identifies suspected lost packets. Based on at least one of the following information, out-of-order tolerance depth, duration threshold, window stall threshold and path identifier, it determines the out-of-order tolerance of the suspected lost packets. When it is determined to be a real packet loss, it generates a fast feedback message containing gap position information or interval information and sends it to the sender.

9. A bitmap-based packet loss detection method for RDMA networks, applied at the transmitting end, characterized in that, include: Establish a sending window and initialize a sending bitmap corresponding to the packet sequence number range covered by the sending window; Send data packets and update the sending status of the corresponding packets in the sending bitmap according to the packet sequence number of the data packets; Receive rapid feedback messages from the receiving end; Based on the rapid feedback message, locate the corresponding packet status item in the sending bitmap and update the status of the unacknowledged corresponding packet to the pending retransmission status; Retransmit the data packets corresponding to the retransmission status.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements a bitmap-based RDMA network packet loss detection method as described in claim 8 or 9.

Images