RDMA retransmission method and network card

Abstract

The invention relates to the technical field of computer networks, and provides an RDMA retransmission method and a network card, and the method comprises the steps: when RDMA request messages are sent, distributing a work queue element WQE for each RDMA request message, recording the initial message sequence number of the corresponding RDMA request message in the WQE, and dividing each RDMA request message into a plurality of messages to be sent in sequence; when it is determined that retransmission is needed, determining a sequence number of a message to be retransmitted and a target WQE corresponding to the message to be retransmitted; and determining a target fragment position of the message to be retransmitted in the RDMA request message corresponding to the target WQE according to the serial number of the message to be retransmitted and the serial number of the initial message recorded in the target WQE, and retransmitting based on the target fragment position. According to the invention, the position of the retransmission message can be quickly and accurately determined, network card storage resources are saved, and the processing efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of computer network technology, and more specifically, to an RDMA retransmission method and a network interface card (NIC). Background Technology

[0002] In the RDMA (Remote Direct Memory Access) protocol and its derivatives RoCEv1 and RoCEv2, RDMA response messages typically only carry the expected sequence number (PSN) and the message number indicating successful reception (MSN). However, RDMA operations (such as Send and RDMAWrite) are limited by the path MTU (PMTU) size and the actual network MTU size, causing a message to be segmented into several packets, thus occupying multiple PSNs. Therefore, when the RDMA requesting end receives an RDMA response message (ACK / NAK message) and triggers a retransmission, it often cannot directly identify which specific fragment among the segments of a message has not been responded to by the receiving end, thus affecting the delineation of the RDMA message retransmission point.

[0003] Existing RDMA protocols and network cards typically employ two methods for retransmission delimitation: one is to record the starting PSN of each message and calculate the difference by traversing the queue to determine the origin of the message to be retransmitted. However, this requires the network card to use a very large amount of storage space, and if storage space is constrained, the number of concurrent requests under a single QP needs to be limited; the other is to find the correspondence between the response message and the small messages segmented by traversing the send request queue. However, this consumes bandwidth between the RDMA network card and the host when receiving the response message, resulting in limited processing efficiency. Summary of the Invention

[0004] The purpose of this application is to provide an RDMA retransmission method and a network card to solve the above-mentioned technical problems.

[0005] In a first aspect, the present invention provides an RDMA retransmission method, applied to an RDMA requesting end, comprising: When sending an RDMA request message, a work queue element (WQE) is allocated for each RDMA request message, and the starting sequence number of the corresponding RDMA request message is recorded in the WQE. Each RDMA request message is divided into multiple messages and sent in sequence. When it is determined that retransmission is required, determine the sequence number of the message to be retransmitted and the target WQE corresponding to the message to be retransmitted. Based on the sequence number of the message to be retransmitted and the starting sequence number recorded in the target WQE, the target fragment position of the message to be retransmitted in the RDMA request message corresponding to the target WQE is determined, and retransmission is performed based on the target fragment position.

[0006] In an optional implementation, the starting sequence number of the corresponding RDMA request message recorded in the WQE includes: The RDMA driver calculates the starting sequence number of the current RDMA request message, writes the starting sequence number into the specified field of WQE, and then sends WQE to the network card hardware.

[0007] In an optional implementation, the specified field is an extended field located in the WQE control segment or task segment. The extended field is 24 bits and is used to record the PSN of the first message slice.

[0008] In an optional implementation, it further includes: Determine whether a retransmission is needed based on the response message.

[0009] In an optional implementation, the response message is an ACK message or a NAK message; if the response message is a NAK message, it is determined that retransmission is required; or, if the sender's timer expires and no corresponding response message is received, it is determined that retransmission is required.

[0010] In an optional implementation, determining the sequence number of the message to be retransmitted and the target WQE corresponding to the message to be retransmitted includes: The sequence number of the message to be retransmitted is determined based on the unacknowledged message sequence number psn_una maintained in the queue context QPC, and the corresponding target WQE is determined based on the unacknowledged WQE index wqe_idx_una.

[0011] In an optional implementation, after receiving the response message, the unacknowledged message sequence number ssn_una and the unacknowledged WQE index wqe_idx_una in the QPC are updated according to the message sequence number in the response message.

[0012] In an optional implementation, determining the target fragment location based on the sequence number of the message to be retransmitted and the starting sequence number recorded in the target WQE includes: Calculate the difference between the sequence number of the message to be retransmitted and the sequence number of the starting message, and determine the fragment number of the message to be retransmitted in the message based on the difference.

[0013] In an optional implementation, retransmission based on the target fragment location includes: starting from the target fragment location, sending a retransmission message in a back N-step manner.

[0014] In a second aspect, the present invention provides a network interface card (NIC) including a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the method of any of the foregoing embodiments.

[0015] Beneficial effects: This invention extends the WQE structure to carry the initial PSN information of the Message, so that the network card does not need to traverse the send request queue when processing retransmissions, nor does it need to maintain a large PSN record table for each Message. This significantly saves the network card's storage space resources and removes the limitation on the number of concurrent Send / Write requests under a single QP.

[0016] When a response message is received and a retransmission is triggered, this invention directly uses the index in the QPC to locate the target WQE, and combines it with the starting PSN recorded in the WQE to quickly calculate the fragment position to be retransmitted. This avoids reading a large number of irrelevant RDMA requests, thereby saving bandwidth between the network card and the host, reducing processing latency, and improving the processing efficiency of the RDMA protocol.

[0017] This invention calculates and records the initial PSN during the driver deployment phase, thus moving the complex delimitation calculation forward. This makes the retransmission processing logic of the network card hardware simpler and more efficient, which is beneficial to improving the hardware's operating frequency and throughput. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of an RDMA request and response architecture according to an embodiment of the present invention; Figure 2 This is a schematic flowchart of an RDMA retransmission method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of another RDMA retransmission method according to an embodiment of the present invention; Figure 4 This is a schematic diagram of another RDMA retransmission method according to an embodiment of the present invention; Figure 5 This is a schematic diagram of a network card structure according to an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0021] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0022] This embodiment provides an RDMA retransmission method applied to the RDMA requesting end to solve the problem of retransmission delimitation difficulties caused by messages being segmented into multiple packets in the RDMA protocol. Figure 1 As shown, the RDMA requesting end encapsulates the RDMA request message into a WQE and submits the WQE to the transmit queue (SQ); the network card hardware retrieves the WQE from the transmit queue and parses the information in the WQE; the network card generates one or more request packets based on the WQE and sends them to the responding end through the network; the responding end receives and processes the request packets and performs the corresponding memory access operations; the responding end generates a response packet (ACK / NAK) and returns it to the requesting end; the requesting end receives the response packet, updates the context state, and completes message confirmation or triggers retransmission.

[0023] The Send Queue (SQ) is a circular queue maintained by the requesting end to store work queue elements (WQEs) to be sent. After the RDMA driver encapsulates the RDMA request message into a WQE, it submits the WQE to the send queue, awaiting retrieval and processing by the network interface card (NIC) hardware. The WQEs in the send queue are arranged in the submission order, corresponding to the sending order of the RDMA request messages.

[0024] A work queue element (WQE) is a "task descriptor" issued by software to the hardware. Figure 1 The two boxes arranged vertically on the left represent WQEs arranged sequentially in the transmission queue. Each WQE corresponds to an RDMA request message (such as Send, RDMAWrite, etc.). According to the technical solution of the present invention, each WQE is extended with the start sequence number (Start PSN) of the message corresponding to that WQE, which is used for precise delimitation during subsequent retransmissions.

[0025] The order in which WQEs are arranged in the sending queue is consistent with the order in which message sequence numbers (MSNs) are assigned; that is, WQEs delivered earlier correspond to smaller MSNs, and WQEs delivered later correspond to larger MSNs. The dequeue order of WQEs is controlled by the network interface card (NIC) hardware, which processes and sends the corresponding request messages sequentially according to the order of the WQEs in the queue.

[0026] Figure 1The arrow pointing from left to right in the diagram represents an RDMA request message sent from the requesting end to the responding end. Each RDMA request message may be divided into multiple message fragments for transmission due to PMTU / MTU limitations, but the diagram uses a single arrow to schematically indicate the transmission direction of the request message. Each request message header carries a Sequence Number (PSN) to identify the message's sequential position within the message.

[0027] Figure 1 The box on the right represents the RDMA response end, which is the peer device that receives the request message and returns a response. After receiving the request message, the response end performs the corresponding memory access operation (such as RDMA Write for data writing, RDMA Read for data reading, etc.) according to the message content, and returns a response message to the requesting end.

[0028] Figure 1 The arrow pointing from right to left in the diagram represents the response message returned by the responding end, including ACK (acknowledgment) and NAK (negative acknowledgment) messages. The response message carries at least two key pieces of information: Message Sequence Number (PSN): the sequence number of the next message the responding end expects to receive, used for flow control and retransmission decisions by the requesting end; Message Sequence Number (MSN): the sequence number of the message the responding end has successfully received, used by the requesting end to acknowledge the completion of the message. After receiving the response message, the requesting end parses the PSN and MSN information, updates the status variables (psn_una, ssn_una, wqe_idx_una) in the local queue pair context (QPC), and triggers a retransmission process as needed.

[0029] like Figure 2 As shown, the method includes the following steps: In step S100, when sending an RDMA request message, a work queue element WQE is allocated for each RDMA request message, and the starting sequence number of the corresponding RDMA request message is recorded in the WQE. Each RDMA request message is divided into multiple messages and sent in sequence.

[0030] Specifically, in RDMA network communication, due to the limitation of the network MTU (Maximum Transmission Unit) size, a complete RDMA request message often cannot be transmitted in one message and needs to be divided into multiple message slices, each message slice having a unique message sequence number (PSN).

[0031] In this embodiment, when the RDMA driver is ready to send a message, it allocates a WQE for the message, which serves as a descriptor for the hardware execution task. Crucially, during the construction of the WQE, the PSN (Starting Sequence Number) of the first segment contained in the message, i.e., the "starting sequence number," can be recorded in a specific field of the WQE. This starting sequence number serves as the logical anchor point for the message within the entire PSN sequence stream, uniquely identifying the message's starting boundary.

[0032] By carrying this information in the WQE, the starting position information of the message can be obtained directly by reading the WQE when processing retransmissions, without having to traverse the entire sending queue or maintain an additional mapping table.

[0033] Step S200: When it is determined that retransmission is required, determine the sequence number of the message to be retransmitted and the target WQE corresponding to the message to be retransmitted.

[0034] Specifically, after sending a message, the RDMA requesting end continuously listens for feedback from the responding end. When packet loss or congestion occurs in the network, the requesting end receives a signal indicating retransmission, such as receiving a NAK (Non-Acknowledgement) message, or the sender's timer expires without receiving a corresponding ACK message, thus determining that retransmission is necessary. At this point, it is first necessary to determine which specific message needs to be retransmitted, i.e., to determine the "sequence number of the message to be retransmitted". Simultaneously, it is necessary to find the original message request to which the message to be retransmitted belongs, i.e., to find the corresponding "target WQE". This process can be achieved by querying the unacknowledged WQE index maintained in the context QPC, thereby quickly locating the WQE that has not yet been fully acknowledged by the responding end.

[0035] Step S300: Based on the sequence number of the message to be retransmitted and the starting sequence number recorded in the target WQE, determine the target fragment position of the message to be retransmitted in the RDMA request message corresponding to the target WQE, and perform retransmission based on the target fragment position.

[0036] Specifically, after determining the sequence number of the message to be retransmitted and the target WQE, the core calculation logic for retransmission delimitation can be executed.

[0037] The pre-recorded "starting message sequence number" can be read from the target WQE, and then the difference between the sequence number of the message to be retransmitted and the starting message sequence number can be calculated. This difference directly reflects the offset of the message to be retransmitted in the original message, that is, which fragment it is after the message was segmented.

[0038] For example, if the initial message sequence number is 100 and the message to be retransmitted has a sequence number of 102, the difference is 2, indicating that the message to be retransmitted is either the 3rd fragment (assuming counting starts from 0) or the 2nd offset fragment of the message. This calculation method allows for precise location of the "target fragment position." Subsequently, based on this target fragment position, the retransmission process is initiated starting from that fragment, for example, using a Go-Back-N strategy, retransmitting that fragment and all subsequent messages to ensure reliable data transmission.

[0039] This embodiment establishes an efficient "anchor-offset" positioning mechanism by recording the initial message sequence number in the WQE and calculating the difference between this sequence number and the sequence number to be retransmitted during retransmission. This mechanism avoids the resource consumption and bandwidth waste caused by traversing the transmission queue or maintaining a large-scale PSN mapping table in existing technologies, and significantly improves the response speed and processing efficiency of RDMA network cards when handling retransmissions.

[0040] In some embodiments, the specific implementation process of recording the starting message sequence number in WQE is described in detail. For example... Figure 3 As shown, the method includes the following steps: Step S301: Calculate the starting sequence number of the current RDMA request message through the RDMA driver, write the starting sequence number into the specified field of WQE, and then send WQE to the network card hardware.

[0041] Specifically, during the initialization phase of RDMA communication or before a transmission request, the host-side RDMA driver performs the computational task. The driver reads the sequence number of the next message to be transmitted (psn_nxt) maintained in the current queue context QPC. This value represents the sequence number of the first segment of the upcoming RDMA request message. When constructing the WQE in memory, the driver fills the calculated starting sequence number into a specific location within the WQE. After this filling operation, the driver distributes the WQE containing this crucial information to the network interface card (NIC) hardware's transmit queue via configuration registers or direct memory access (DMA). This process ensures that the NIC hardware can directly obtain the message's start boundary information without additional computation or lookup when starting to process the transmission task.

[0042] In some embodiments, the specified field is an extended field located in the WQE control segment or task segment. The extended field is 24 bits and is used to record the PSN of the first message slice of the message.

[0043] Specifically, to efficiently store and retrieve the starting packet sequence number at the hardware level, this embodiment extends the WQE data structure. A dedicated 24-bit storage area is allocated in the WQE storage layout as an extended field. This extended field can be set in the WQE control section (CTRL) or the task segment, depending on the network card hardware design architecture, as long as the hardware can correctly parse the field. Choosing 24 bits for this field width is based on the RDMA protocol standard, because the PSN (Packet Sequence Number) defined in the protocol specification is typically a 24-bit unsigned integer, which can support a sufficiently large sequence number space to prevent wraparound issues. By storing the starting packet sequence number in this 24-bit extended field, the network card hardware can directly extract this value when parsing the WQE and use it as the base value for subsequent retransmission delimitation calculations. This design not only is compatible with the standard RDMA protocol format but also avoids additional bit-width conversion overhead by reusing the standard PSN bit width, improving hardware processing efficiency.

[0044] In some embodiments, the judgment logic for retransmission triggering conditions is further elaborated in detail. For example... Figure 3 As shown, the method includes the following steps: Step S302: Receive a response message and determine whether retransmission is required based on the response message.

[0045] Specifically, during RDMA communication, after sending a request message, the requesting end continuously listens for and receives feedback messages from the responding end. These response messages are the core mechanism for maintaining transmission reliability in the RDMA protocol, used to inform the sending end of the data packet reception status. In this embodiment, response messages are mainly divided into two types: ACK (acknowledgment) messages and NAK (non-acknowledgment) messages. Upon receiving either type of response message, retransmission is not immediately initiated blindly. Instead, a judgment logic is entered to parse the message content and combine it with the current sending status to determine whether a retransmission process needs to be initiated. This on-demand triggering mechanism avoids invalid retransmission operations and saves network bandwidth resources.

[0046] In some embodiments, the response message is an ACK message or a NAK message; if the response message is a NAK message, it is determined that retransmission is required; or, if the sender's timer expires and no corresponding ACK message is received, it is determined that retransmission is required.

[0047] When a received response message is identified as a NAK message, it usually means that the responding end has detected a message loss, a reception error, or other unrecoverable anomaly. In this case, the NAK message indicates that retransmission is required.

[0048] ACK messages are typically used to acknowledge that the receiver has successfully received a sequence of messages. If no acknowledgment is received in time, that is, if the sender's timer expires and no corresponding response message is received, then retransmission is required.

[0049] In some embodiments, the specific implementation logic of retransmission delimitation and retransmission strategy based on queue-based context QPC is further described. For example... Figure 3 As shown, the method includes the following steps: Step S303: After receiving the response message, update the unacknowledged message sequence number ssn_una and the unacknowledged WQE index wqe_idx_una in the QPC according to the message sequence number in the response message.

[0050] Specifically, upon receiving an ACK or NAK message from the responding end, the network interface card (NIC) hardware parses the MSN (Message Sequence Number) carried in the message. The MSN represents the sequence number of consecutive messages received by the responding end. The MSN in the response message is compared with the ssn_una (Unacknowledged Send Sequence Number) maintained locally in the QPC to calculate how many messages were acknowledged in this response message. Subsequently, based on the calculated difference, ssn_una and wqe_idx_una in the QPC are updated synchronously. For example, if 3 messages are acknowledged, wqe_idx_una is incremented by 3, pointing to the next unacknowledged WQE. This update process ensures that the pointer in the QPC always points to the "unacknowledged" boundary, providing an accurate starting point for possible subsequent retransmission operations and guaranteeing the consistency of the state machine.

[0051] As an example, the requesting end parses the response message and obtains the message sequence number (ack.msn). This value is compared with the unacknowledged message sequence number (ssn_una) maintained in the local queue pair context (QPC) to calculate the number of messages successfully acknowledged in this response message (wqe_cmpl_num). The calculation formula is: wqe_cmpl_num = ack.msn - ssn_una + 1; this value represents how many messages have been successfully received by the receiving end since the current unacknowledged message.

[0052] Based on the response message and calculation results, update the three key state variables in the Receive Direction Queue Pair Context (QPC): Unacknowledged message sequence number (psn_una): Updated to the sequence number in the response message plus 1, i.e., psn_una = ack.psn + 1, representing the next expected acknowledgment message sequence number; Unacknowledged message sequence number (ssn_una): Updated to the message sequence number in the response message plus 1, i.e., ssn_una = ack.msn + 1, representing the next expected acknowledgment message sequence number; Unconfirmed WQE Index (wqe_idx_una): This index is the current value plus the number of messages completed this time, i.e., wqe_idx_una += wqe_cmpl_num, pointing to the WQE corresponding to the first unconfirmed message.

[0053] The network interface card (NIC) notifies the driver of completion by generating a Completion Queue (CQE) element, informing the driver which messages have been successfully received by the peer and related resources can be released. The driver can then perform subsequent processing based on the CQE, such as cleaning up completed work requests and notifying upper-layer applications.

[0054] Step S304: Determine the sequence number of the message to be retransmitted based on the unacknowledged message sequence number psn_una maintained in the queue context QPC, and determine the corresponding target WQE based on the unacknowledged WQE index wqe_idx_una.

[0055] Specifically, the QPC is a high-speed storage area in the RDMA network card hardware that maintains queue pair status information, recording key status variables for transmission and reception in real time. In this embodiment, when it is determined that a retransmission process needs to be initiated, it is not necessary to traverse the entire transmission queue, but to directly read the variables in the QPC. Among them, psn_una (Unacknowledged PacketSequence Number) records the smallest packet sequence number that has not yet been acknowledged by the responding end, and this value is the sequence number of the packet to be retransmitted. At the same time, wqe_idx_una (Unacknowledged WQE Index) records the index position of the WQE to which the unacknowledged packet belongs in the transmission queue. Through wqe_idx_una, the network card hardware can directly address the target WQE in memory or on-chip cache without checking each WQE one by one. This direct indexing mechanism based on QPC pointers reduces the time complexity of retransmission delimitation from O(N) to O(1), greatly improving the response speed of the network card in handling high packet loss network environments.

[0056] Step S305: Calculate the difference between the sequence number of the message to be retransmitted and the sequence number of the starting message, and determine the fragment number of the message to be retransmitted in the message based on the difference.

[0057] Specifically, after reading the target WQE via wqe_idx_una, the starting sequence number of the message (i.e., wqestartpsn recorded in the aforementioned embodiment) is extracted from the extended fields of the WQE. At this point, two key PSN values ​​are available: one is the sequence number of the message to be retransmitted (psn_una), and the other is the starting sequence number of the message. By performing a subtraction operation, i.e., "the sequence number of the message to be retransmitted minus the starting sequence number," a difference is obtained. This difference accurately reflects the offset position of the message to be retransmitted within the original message. For example, if the starting sequence number is 1000 and the sequence number of the message to be retransmitted is 1002, the difference is 2, indicating that the message to be retransmitted is the 3rd fragment of the message (assuming counting starts from 0). This delimitation method based on mathematical calculation is logically simple and highly deterministic, making it very suitable for hardware logic circuit implementation and avoiding complex table lookup operations.

[0058] Step S306: Starting from the target fragment position, send a retransmission message in a back N-step manner.

[0059] Specifically, after determining the location of the target fragment, the retransmission process is initiated. This embodiment can employ a Go-Back-N retransmission strategy. This means that once it is determined that a fragment needs to be retransmitted, not only the fragment itself will be retransmitted, but also all subsequent fragments that have been sent but may not have been acknowledged will be retransmitted.

[0060] In practical implementation, the network card hardware locates the offset address in the data buffer corresponding to the target WQE based on the calculated fragment sequence number, and rereads the data from that address and encapsulates it into a packet for transmission. Although this strategy may result in the duplicate transmission of some received packets, its control logic is simple, it does not require maintaining a complex buffer reordering mechanism at the receiving end, and combined with the fast delimitation method described above, it can minimize the retransmission response time and meet the stringent low-latency requirements of the RDMA protocol.

[0061] Through the above steps, this embodiment constructs a complete closed-loop mechanism from state update, rapid positioning to accurate calculation and efficient retransmission, making full use of the advantages of QPC hardware context and WQE extended information, and solving the technical problem of low retransmission delimitation efficiency in the prior art.

[0062] In some embodiments, an RDMA retransmission method can also be provided, applied to scenarios where the network MTU changes dynamically, to solve the retransmission delimitation problem when the number of message fragments changes due to a change in the network path MTU. For example... Figure 4 As shown, the method includes the following steps: In step S410, when sending an RDMA request message, a work queue element WQE is allocated for each RDMA request message, and the starting message sequence number, current MTU configuration information, and MTU version identifier of the corresponding RDMA request message are recorded in the WQE.

[0063] Specifically, in RDMA network communication, the Maximum Transmission Unit (MTU) of a network path may dynamically adjust due to network topology changes, route switching, or the Path MTU Discovery (PMTUD) mechanism. When the MTU changes, the number of fragments of the same message and the size of each fragment will change, which brings new challenges to PSN-based retransmission delimitation. In this embodiment, when constructing the WQE, in addition to recording the message's starting sequence number, two additional key pieces of information are recorded: the current network-configured MTU value and the MTU version identifier. The MTU version identifier is a monotonically increasing counter, which increments by 1 whenever an MTU change is detected. By carrying this information in the WQE, during subsequent retransmission processing, it is possible to identify under what MTU configuration the WQE was generated, thereby adopting an appropriate delimitation strategy. For example, the extended fields of the WQE can be designed to include a 24-bit starting PSN field, a 16-bit MTU value field, and an 8-bit MTU version identifier field, occupying a total of 48 bits of storage space.

[0064] Step S420: Listen for network MTU change events. When an MTU change is detected, determine the scope of the impact of the MTU change and perform differentiated adaptive processing based on the WQE transmission status.

[0065] Specifically, the network interface card (NIC) hardware or driver continuously monitors for changes in the network MTU. MTU changes can be triggered by various reasons, such as receiving an ICMP Fragmentation Needed message, path changes due to routing table updates, or proactive configuration adjustments by the management plane. When a change in MTU from the old value MTU_old to the new value MTU_new is detected, it is necessary to determine the scope of the impact of this change on the WQEs currently in the transmission queue. This embodiment categorizes WQEs into three types based on their transmission status: the first type is "not started transmitting," meaning the WQE has been sent to the NIC but has not yet begun processing; the second type is "transmitting," meaning the message corresponding to the WQE is being segmented and sent, but has not yet been completely transmitted; and the third type is "sent but not acknowledged," meaning all packets corresponding to the WQE have been sent, but no acknowledgment has been received from the responding end.

[0066] For WQEs in the first "Not Started Sending" state, a recalculation logic is executed: the number of message fragments is recalculated based on the new MTU_new value, and the starting PSN recorded in the WQE is updated. Since the WQE has not yet started sending, its starting PSN can be reallocated based on the psn_nxt value in the current QPC, without considering the continuity issue with already sent messages. Simultaneously, the MTU version identifier of the WQE is updated to the latest MTU version number.

[0067] For WQEs in the second "transmitting" state, a tag protection logic is executed: the initial PSN and MTU_old values ​​recorded in the WQE remain unchanged, but an "MTU change flag" is set in the WQE's status field to indicate that the message was started before the MTU change. Simultaneously, the PSN of the last message sent at the time of the MTU change is recorded, denoted as psn_switch. For this WQE, messages before (and including) psn_switch are segmented according to MTU_old, while messages after psn_switch need to be re-segmented according to MTU_new. This processing method ensures that the PSN sequence of sent messages is not disrupted by the MTU change, while allowing subsequent messages to adapt to the new MTU configuration.

[0068] For WQEs in the third "sent but not acknowledged" state, the unchanged logic is executed: all packets of these WQEs have been sent according to MTU_old, and their initial PSN and fragmentation information remain unchanged, requiring no adjustments. This ensures the consistency of retransmission delimitation and avoids the invalidation of historical packet delimitation information due to MTU changes.

[0069] Step S430: When it is determined that retransmission is required, determine the sequence number of the message to be retransmitted and the target WQE corresponding to the message to be retransmitted, and check the MTU change flag of the target WQE.

[0070] Specifically, when the network interface card (NIC) receives a NAK packet or detects an ACK packet indicating the presence of unacknowledged packets, it determines that a retransmission process is required. First, the sequence number of the packet to be retransmitted is determined based on the psn_una maintained in the QPC, and the corresponding target WQE is determined based on wqe_idx_una. Unlike Example 1, this example, after reading the target WQE, additionally checks the MTU change flag and MTU version identifier of the WQE. If the MTU change flag is not set, it means that all packets of this WQE are sent under the same MTU configuration, and the delimitation calculation can be performed directly according to the method in Example 1. If the MTU change flag is set, it means that the packets of this WQE span the MTU change time, and special delimitation logic needs to be executed.

[0071] Step S440: If the MTU change flag of the target WQE is set, then select the corresponding delimitation calculation strategy according to the relationship between the sequence number of the message to be retransmitted and the MTU change switching point psn_switch.

[0072] Specifically, when the MTU change flag of the target WQE is set, it is necessary to determine whether the packet to be retransmitted was sent before or after the MTU change. The sequence number psn_una of the packet to be retransmitted is compared with the psn_switch recorded in the target WQE. If psn_una ≤ psn_switch, it means the message to be retransmitted was segmented and sent according to MTU_old before the MTU change. In this case, the initial PSN and MTU_old values ​​recorded in the target WQE are used for delimitation calculation. The specific calculation formula is: Fragment offset = (psn_una - Initial PSN). Since MTU_old is known, the byte offset of the fragment in the original message data buffer can be further calculated: Byte offset = Fragment offset × MTU_old. This calculation method ensures accurate location of historical messages.

[0073] If psn_una > psn_switch, it means the message to be retransmitted was sent after the MTU change, segmented according to MTU_new. In this case, the delimitation calculation needs to be performed in two steps. First, calculate the amount of data sent before the MTU change: data_old = (psn_switch - initial PSN + 1) × MTU_old. Second, calculate the offset of the message to be retransmitted in the MTU_new segmentation sequence: fragment offset_new = psn_una - psn_switch - 1. Third, calculate the byte offset of this fragment in the original message data buffer: byte offset = data_old + fragment offset_new × MTU_new. This segmented calculation method accurately reflects the impact of the MTU change on the fragment boundaries.

[0074] Step S450: Based on the calculated target fragment position, starting from the corresponding offset in the original message data buffer, encapsulate and send the retransmitted message according to the currently valid MTU configuration.

[0075] Specifically, after determining the target shard position and byte offset, the retransmission sending process is initiated. Different from Embodiment 1, in this embodiment, the current configuration of the MTU needs to be considered during retransmission. If the packet to be retransmitted was sent under MTU_old, but the current network MTU has changed to MTU_new, it is necessary to decide whether to retransmit according to MTU_old (maintaining consistency with the original packet) or according to MTU_new (adapting to the current network). This embodiment adopts an adaptive strategy: if MTU_new < MTU_old, then re-segment and retransmit according to MTU_new, which can avoid the packet being fragmented again in the network due to being too large; if MTU_new > MTU_old, then still retransmit according to MTU_old to maintain consistency with what the receiving end expects. In the Go-Back-N retransmission mode, starting from the target shard position, resend this shard and all subsequent packets to ensure the reliability of data transmission.

[0076] In this embodiment, a complete adaptive retransmission delimitation mechanism is established by recording the MTU configuration information and version identifier in the WQE and performing differential status management on the WQE when the MTU changes. This mechanism can cope with the dynamic changes of the network environment, ensure that the retransmission point can still be accurately located in the case of MTU changes, avoid delimitation errors caused by changes in the shard boundary, and improve the robustness and adaptability of the RDMA network in complex environments.

[0077] In some embodiments, a network card is also provided for implementing the RDMA retransmission method described in the above embodiments. As Figure 5 shown, the network card device includes a processor and a memory. The network card 600 provided by the embodiments of the present application at least includes: a processor 601, a memory 602, and a computer program stored on the memory 602 and executable on the processor 601. When the processor 601 executes the computer program, the method provided by the embodiments of the present application is implemented.

[0078] The network card 600 provided by the embodiments of the present application may further include a bus 603 connecting different components (including the processor 601 and the memory 602). Among them, the bus 603 represents one or more of several types of bus structures, including a memory bus, a peripheral bus, a local bus, etc.

[0079] Memory 602 may include a readable storage medium in the form of volatile memory, such as random access memory (RAM) 6021 and / or cache memory 6022, and may further include read-only memory (ROM) 6023. Memory 602 may also include a program tool 6025 having a set (at least one) of program modules 6024, including but not limited to an operating subsystem, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0080] Processor 601 can be a single processing element or a collective term for multiple processing elements. For example, processor 601 can be a central processing unit (CPU) or one or more integrated circuits configured to implement the methods provided in the embodiments of this application. Specifically, processor 601 can be a general-purpose processor, including but not limited to CPUs, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0081] Network interface card 600 can communicate with one or more external devices 604 (e.g., keyboard, remote control, etc.), and also with one or more devices that enable user interaction with network interface card 600 (e.g., mobile phone, computer, etc.), and / or with any device that enables network interface card 600 to communicate with one or more other network interface cards 600 (e.g., router, modem, etc.). This communication can be performed through input / output (I / O) interface 605. Furthermore, network interface card 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) through network adapter 606. Figure 5 As shown, network adapter 606 communicates with other modules of network card 600 via bus 603. It should be understood that, although... Figure 5As not shown in the diagram, it can be used in conjunction with the NIC 600 to utilize other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, Redundant Arrays of Independent Disks (RAID) subsystems, tape drives, and data backup storage subsystems.

[0082] It should be noted that, Figure 5 The network card 600 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0083] Specifically, the network interface card (NIC) can be a smart NIC or a network interface card integrated into the host. The processor is the core control unit of the NIC and can be a network processor, application-specific integrated circuit (ASIC), or field-programmable gate array (FPGA), etc., responsible for executing packet encapsulation, parsing, and protocol processing logic. The memory is connected to the processor and is used to store computer programs and data generated during operation. This memory can include random access memory (RAM), flash memory, or other non-volatile storage media.

[0084] The memory stores a computer program, and when the processor executes the computer program, it implements the method described in any of the above embodiments.

[0085] Specifically, the computer program stored in the memory contains all the logical instructions detailed in the foregoing embodiments. When the processor calls and executes these programs, the network interface card (NIC) device can perform the following operations: First, when sending an RDMA request message, it generates the starting sequence number of the message through driver layer calculation or hardware logic and writes it into the extended field of the work queue element WQE. This extended field has a width of 24 bits and is located in the control segment or task segment of the WQE. Second, after receiving a response message, it parses the ACK or NAK information in the message and, in conjunction with the variables such as psn_una, ssn_una, and wqe_idx_una maintained in the queue context QPC, determines whether retransmission is required. Finally, when it is determined that retransmission is required, it uses the QPC index to quickly locate the target WQE, reads the starting sequence number recorded therein, accurately determines the fragment position of the message to be retransmitted in the message by calculating the difference, and performs a retransmission operation that goes back N steps based on that position.

[0086] The network interface card (NIC) provided in this embodiment executes the retransmission method directly through hardware, offloading the complex protocol logic that originally required host CPU processing to the NIC hardware. This hardware-software co-processing design not only greatly frees up host CPU resources, allowing it to focus on upper-layer application services (such as AI training model computation and distributed storage data processing), but also significantly improves the response speed and processing efficiency of RDMA retransmission by leveraging the high parallelism and low latency characteristics of the hardware. This NIC device is particularly suitable for scenarios that are extremely sensitive to network throughput and latency, such as high-performance computing clusters, large-scale distributed storage, and artificial intelligence training clusters. It can effectively solve network congestion and packet loss retransmission problems under high concurrency, ensuring overall stability and performance. The computer-readable storage medium provided in this application embodiment is described below. The computer-readable storage medium provided in this application embodiment stores computer instructions, which, when executed by a processor, implement the method provided in this application embodiment. Specifically, the computer instructions can be built into or installed in the processor, so that the processor can implement the method provided in this application embodiment by executing the built-in or installed computer instructions.

[0087] Furthermore, the method provided in this application embodiment can also be implemented as a computer program product, which includes program code that implements the method provided in this application embodiment when run on a processor.

[0088] The computer program product provided in this application embodiment may employ one or more computer-readable storage media, which may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination thereof. Specifically, more specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires, portable disks, hard disks, RAM, ROM, erasable programmable read-only memory (EPROM), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0089] The computer program product provided in this application embodiment can be a CD-ROM and include program code, and can also run on electronic devices such as computers. However, the computer program product provided in this application embodiment is not limited thereto. In this application embodiment, the computer-readable storage medium can be any tangible medium that contains or stores program code, which can be used by or in conjunction with an instruction execution system, device, or apparatus.

[0090] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0091] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0092] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0093] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. An RDMA retransmission method, characterized in that, Applied to the RDMA requesting end, including: When sending an RDMA request message, a work queue element (WQE) is allocated for each RDMA request message, and the starting sequence number of the corresponding RDMA request message is recorded in the WQE. Each RDMA request message is divided into multiple messages and sent in sequence. When it is determined that retransmission is required, the sequence number of the message to be retransmitted and the target WQE corresponding to the message to be retransmitted are determined. Based on the sequence number of the message to be retransmitted and the starting message sequence number recorded in the target WQE, the target fragment position of the message to be retransmitted in the RDMA request message corresponding to the target WQE is determined, and retransmission is performed based on the target fragment position.

2. The method according to claim 1, characterized in that, The starting sequence number of the corresponding RDMA request message recorded in the WQE includes: The RDMA driver calculates the starting sequence number of the current RDMA request message, writes the starting sequence number into a specified field of WQE, and then sends WQE to the network card hardware.

3. The method according to claim 2, characterized in that, The specified field is an extended field located in the WQE control segment or task segment. The extended field is 24 bits and is used to record the PSN of the first message slice of the message.

4. The method according to claim 1, characterized in that, Also includes: Determine whether retransmission is needed based on the response message.

5. The method according to claim 4, characterized in that, If the response message is a NAK message, then it is determined that retransmission is required; or, if the sender's timer expires and no corresponding response message is received, then it is determined that retransmission is required.

6. The method according to claim 1, characterized in that, The process of determining the sequence number of the message to be retransmitted and the target WQE corresponding to the message to be retransmitted includes: The sequence number of the message to be retransmitted is determined based on the unacknowledged message sequence number psn_una maintained in the queue context QPC, and the corresponding target WQE is determined based on the unacknowledged WQE index wqe_idx_una.

7. The method according to claim 6, characterized in that, After receiving the response message, update the unacknowledged message sequence number ssn_una and the unacknowledged WQE index wqe_idx_una in the QPC according to the message sequence number in the response message.

8. The method according to claim 6, characterized in that, The step of determining the target fragment location based on the sequence number of the message to be retransmitted and the starting sequence number recorded in the target WQE includes: Calculate the difference between the sequence number of the message to be retransmitted and the sequence number of the starting message, and determine the fragment number of the message to be retransmitted in the message based on the difference.

9. The method according to claim 1, characterized in that, The retransmission based on the target fragment position includes: starting from the target fragment position, sending a retransmission message in a backtracking manner of N steps.

10. A network interface card (NIC), characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 9.

Images