Remote direct memory access data retransmission method and terminal

Abstract

The present invention relates to a data retransmission method and terminal for remote direct memory access (RDMA). When the requesting end detects that a retransmission operation needs to be performed, it immediately blocks the new WQE command channel and assists the ORT (Outstanding Request Table) to open the retransmission WQE command channel to achieve a rapid response to the retransmission. The ORT actively takes out the WQE to be retransmitted through the retransmission WQE command channel; the sending protocol stack shields the new WQE of the retransmission QP and only executes the retransmission WQE to achieve retransmission queue interruption. The present invention designs and optimizes the data retransmission system for remote direct memory access. The set ORT table can realize the rapid positioning of the retransmission packet and the rapid acquisition of the metadata information of the retransmission data packet during retransmission, thereby reducing the waiting time for the retransmission command of the requesting end. The three WQE command channels are set to ensure the priority execution of the retransmission WQE, and the retransmission queue shield is set to eliminate the repeated execution problem of the requesting end, reduce the network delay of the retransmission packet, and improve the effective network bandwidth.

Description

Technical Field

[0001] The present invention belongs to the technical field of data transmission networks, and in particular relates to a data retransmission method and a terminal for remote direct memory access. Background Art

[0002] With the rapid development of technologies such as artificial intelligence, cloud computing, and distributed computing, the scale and data volume of data centers are growing larger and larger. These big data applications require large amounts of data transmission for distributed and parallel processing. In this context, the network is becoming a major bottleneck for big data applications.

[0003] To address this issue, Remote Direct Memory Access (RDMA) technology was introduced and applied in data centers to achieve better performance. RDMA enables a zero-copy mechanism and CPU bypass capabilities, achieving higher bandwidth and lower latency.

[0004] RDMA requires hardware implementation of a reliable transmission protocol. A reliable service guarantees that messages are delivered at most once, in order, and without corruption, from the requester to the responder. Key elements of a reliable service include a protection scheme (CRC) that detects corrupted data, an acknowledgment mechanism that allows the requester to confirm that the message was successfully delivered, and a data sequence number and retransmission mechanism that detects lost packets and allows the requester to reissue messages.

[0005] Currently, RoCEv2 (RDMA over Converged Ethernet v2) is the most widely used RDMA network. However, due to the inherent unreliability of Ethernet and the sensitivity of RDMA networks to packet loss and congestion, a packet loss rate exceeding one thousandth will cause the effectiveness of RDMA networks to drop sharply.

[0006] The protocol's back-off-N retransmission scheme retransmits all subsequent packets starting from the point of packet loss, resulting in significant bandwidth waste and low efficiency. To address this issue, many optimization schemes have been proposed, such as retransmitting only lost packets. However, these modifications also come with numerous drawbacks, including slow response to timeout retransmissions, the need for additional off-chip buffering to store retransmitted data and information, and the increased complexity of the mechanism required for hardware implementation.

[0007] Another potential issue with the Fallback N retransmission scheme is duplicate execution. When the requesting end detects the need to retransmit a data packet, it requires a certain amount of processing time to locate the retransmission WQE, retrieve the retransmission WQE content, and retrieve the retransmission data. During this time, the requesting end continues to operate, and therefore may still send a large number of RDMA requests. These RDMA requests reach the responding end, which continues to process the received requests in sequence. However, these requests are resent to the receiving end using the Retransmission N scheme, causing the receiving end to execute the same RDMA request twice. If these are large numbers of RDMA READ requests, the responding end will need to spend a considerable amount of PCIe read latency to process unnecessary requests, severely hindering the execution of retransmission requests. This will result in a large amount of invalid data on the link, reducing the effective network bandwidth while also increasing network latency and retransmission recovery time. Summary of the Invention

[0008] In order to solve the above problems existing in the prior art, the present invention provides a data retransmission method and terminal for remote direct memory access. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0009] The present invention provides a data retransmission method for remote direct memory access, which is applied to an RDMA network card request terminal and includes: a normal transmission mode and a retransmission mode, wherein:

[0010] In normal transmission mode, a new WQE command is obtained and cached in the QP queue of the first WQE cache module. The new WQE command is obtained from the QP queue in the cache order to generate corresponding framing and frame sending requests. The sending side protocol stack completes framing according to the framing request and sends it to the RDMA network card response terminal. The new WQE command in the QP queue is transmitted through the new WQE command channel.

[0011] In the retransmission mode, a retransmission command is generated in response to detecting a NACK response or a timeout pulse, the QP queue corresponding to the retransmission command is marked as a retransmission QP queue, the new WQE command channel of the retransmission QP queue is blocked, its retransmission WQE command channel is opened, and the retransmission queue-jumping task of the retransmission QP queue is executed based on the stored ORT; after the retransmission queue-jumping task is completed, the retransmission WQE command channel of the retransmission QP queue is closed, its new WQE command channel is opened, and normal transmission mode operation is resumed.

[0012] In one embodiment of the present invention, each QP queue stores an ORT corresponding to the ORT, which is used to store data information of WQE commands in the QP queue that have sent requests but have not received responses from the response terminal, wherein the data information is used to indicate metadata information of the data packet corresponding to the WQE command or a mapping relationship between the data packet corresponding to the WQE command and the metadata information.

[0013] In one embodiment of the present invention, the process of retransmitting the queue-jumping task of the retransmission QP queue includes:

[0014] determining metadata information of the retransmitted data packet based on detecting a NACK response or a timeout pulse;

[0015] According to the metadata information of the retransmitted data packet, searching the ORT of the retransmitted QP queue to determine the retransmitted WQE command corresponding to the metadata information of the retransmitted data packet;

[0016] Blocking a new WQE command channel of the retransmission QP queue to shield the transmission of new WQE commands of the retransmission QP queue;

[0017] The corresponding framing and sending frame request are generated according to the obtained retransmission WQE command, and the sending side protocol stack completes the framing according to the framing request and sends it to the RDMA network card response terminal; wherein, the retransmission WQE command is transmitted through the retransmission WQE command channel.

[0018] In one embodiment of the present invention, in response to detecting a NACK response or a timeout pulse, generating a retransmission command, marking a QP queue corresponding to the retransmission command as a retransmission QP queue, includes:

[0019] In response to detecting a NACK response or a timeout pulse, a retransmission command is generated. The queue mask of the sending side protocol stack is configured to turn on the masking enable of the QP queue corresponding to the retransmission command and mark it as a retransmission QP queue.

[0020] In one embodiment of the present invention, for the retransmission QP queue, during the execution of the retransmission queue interruption task, the sending side protocol stack silently discards the framing and frame sending request of the new WQE command, and only executes the framing and frame sending request corresponding to the retransmission WQE command to achieve retransmission queue interruption.

[0021] In one embodiment of the present invention, after the retransmission queue-jumping task is completed, the retransmission WQE command channel of the retransmission QP queue is closed and its new WQE command channel is opened, the further step further includes: configuring the queue mask of the sending side protocol stack, disabling the mask enable of the QP queue corresponding to the retransmission command, and canceling the retransmission mark of the QP queue.

[0022] In one embodiment of the present invention, when the ORT reaches a storage threshold, a new WQE command obtained from the QP queue of the first WQE cache module is stored in the second WQE cache module as an unexecuted WQE command of the QP queue;

[0023] Correspondingly, after the retransmission queue-jumping task is completed, if there are unexecuted WQE commands stored in the second WQE cache module, all unexecuted WQE commands are obtained and sent, and then the sending task is executed for the new WQE command transmitted through the new WQE command channel; wherein, the unexecuted WQE command is transmitted through the unexecuted WQE command channel.

[0024] In one embodiment of the present invention, the ORT writing process includes:

[0025] The host driver dynamically configures the storage threshold of the ORT corresponding to each QP queue based on the activity level of the QP queue and the bandwidth-delay product of the current network topology. The ORT is set with a head pointer, tail pointer, and retransmission pointer indicating QP storage;

[0026] After generating corresponding framing and sending frame requests for the acquired new WQE command or the unexecuted WQE command, the data information of the WQE command is written into the ORT;

[0027] The data information of the WQE command is WQE command information or WQE metadata information; the WQE command information includes a WQE completion QP number, complete WQE content, and a WQE first sequence number; the WQE metadata information includes a WQE sequence number, a WQE first sequence number, a WQE total length, and a WQE offset;

[0028] After writing is completed, the head pointer of the QP queue corresponding to the ORT is increased by one.

[0029] In one embodiment of the present invention, in normal transmission mode, the ORT reading process includes:

[0030] In response to detecting the ACK response, generating a read request message based on the received QP number, message number, and sequence number;

[0031] According to the read request information, index and read the data information of the WQE command stored in the ORT;

[0032] After the read is completed, a read reply feedback message is generated, and the tail pointer of the QP queue corresponding to the ORT is increased by one;

[0033] In the retransmission mode, the ORT reading process includes:

[0034] In response to detecting a NACK response or a timeout pulse, reading the ORT according to the QP number, message number, and sequence number corresponding to the NACK response or the timeout pulse;

[0035] Move the retransmission pointer of the ORT corresponding to the QP queue that needs to be retransmitted to the current read pointer;

[0036] Index and read the data information of the WQE command stored in QRT from the retransmission pointer;

[0037] The data information of the read WQE command is transmitted through the retransmission WQE command channel;

[0038] Each time the ORT is read, the retransmission pointer of the QP queue corresponding to the ORT is increased by one until the retransmission pointer moves to the write pointer.

[0039] The present invention provides a remote direct memory access data retransmission terminal, which serves as an RDMA network card request terminal to implement the remote direct memory access data retransmission method described in any of the above embodiments. The data retransmission terminal includes:

[0040] The WQE acquisition module uses DMA to obtain WQE commands in the queue ring buffer from the host memory;

[0041] A first WQE cache module, configured to temporarily store the WQE commands acquired by the WQE acquisition module and maintain order, wherein the cache is implemented in a first-in-first-out memory format;

[0042] ORT management module, used to store ORT and implement ORT reading and writing and sending retransmission commands;

[0043] A second WQE cache module, configured to store unexecuted WQE commands when the ORT reaches a storage threshold, to alleviate the head-of-queue blocking problem of the first WQE cache module; wherein the cache is implemented in a first-in-first-out memory format;

[0044] a command execution module, configured to generate corresponding framing and frame sending requests for acquired new WQE commands, retransmitted WQE commands, and unexecuted WQE commands, and send them to a transmitting-side protocol stack, and to perform corresponding operations based on received information from a receiving-side protocol stack, wherein the received information includes a NACK response, a timeout pulse, and an ACK response;

[0045] The timer module is used to time the response time of the WQE command request in each QP queue and count the number of retransmission operations in each QP queue;

[0046] The retransmission management module is used to control the WQE acquisition module, the first WQE cache module, the ORT management module, the command execution module, the second WQE cache module and the timer module to implement the data retransmission method of remote direct memory access.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The present invention's remote direct memory access (RDMA) data retransmission method and terminal design and optimize a remote direct memory access (RDMA) data retransmission system. The ORT (Optical Response Time) allows for rapid location of retransmitted packets and rapid acquisition of metadata information during retransmission, reducing the requester's wait time for retransmitted commands. The three WQE (Warning Queuing Expectation) command channels ensure prioritized WQE retransmissions, while a retransmission queue mask eliminates duplicate execution issues on the requester side, reducing network latency for retransmitted packets and increasing effective network bandwidth.

[0049] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following preferred embodiments are specifically cited and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a flow chart of a data retransmission method for remote direct memory access provided by an embodiment of the present invention;

[0051] Figure 2 This is a typical connection topology diagram of two RDMA-enabled terminals provided by an embodiment of the present invention;

[0052] Figure 3 This is a schematic diagram of a requesting end timeout retransmission caused by a responding end response being lost in the network, provided by an embodiment of the present invention;

[0053] Figure 4 This is a schematic diagram of a requesting end NACK retransmission caused by a request from the requesting end being lost in the network, provided by an embodiment of the present invention;

[0054] Figure 5 This is a schematic diagram of command execution on the request side when retransmission occurs in an existing RDMA network card;

[0055] Figure 6 This is a schematic structural diagram of a data retransmission terminal for remote direct memory access provided by an embodiment of the present invention;

[0056] Figure 7 This is a schematic diagram of the structure of an ORT (Outstanding Request Table) provided in an embodiment of the present invention;

[0057] Figure 8 This is a core working flow diagram of a remote direct memory access data retransmission terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following is a detailed description of a remote direct memory access data retransmission method and terminal proposed in accordance with the present invention in conjunction with the accompanying drawings and specific implementation methods.

[0059] The aforementioned and other technical contents, features, and effects of the present invention are clearly presented in the following detailed description of the specific embodiments in conjunction with the accompanying drawings. Through the description of the specific embodiments, a deeper and more specific understanding of the technical means and effects adopted by the present invention to achieve the intended purpose can be obtained. However, the accompanying drawings are provided for reference and illustration purposes only and are not intended to limit the technical solutions of the present invention.

[0060] In order to facilitate understanding of the solution of the present invention, the RDMA functional terminal and the fallback N retransmission solution specified by the RDMA protocol are first described. Figure 2 The embodiment of the present invention shown in the figure provides a typical connection topology diagram of two terminals with RDMA functions. RDMA requires a reliable transmission protocol to be implemented in hardware. The reliable service ensures that the message is delivered from the requester to the responder at most once, in order and without damage.

[0061] The RDMA protocol's back-off N retransmission scheme requires retransmission of all packets starting from the source of packet loss, resulting in significant bandwidth waste and low efficiency. To address this issue, various optimization schemes have been proposed, such as retransmitting only lost packets. However, these modifications also come with numerous drawbacks, including slow response to timeout retransmissions, the need for additional off-chip buffering to store retransmitted data and information, and the increased complexity of the mechanism required for hardware implementation.

[0062] In addition, another potential problem of the back-off N retransmission scheme is repeated execution. Figure 3 and Figure 4 The following figure shows a schematic diagram of a requester timeout retransmission caused by a response loss in the network and a schematic diagram of a requester NACK retransmission caused by a request loss in the network. As shown in the figure, when the requester detects that a data packet needs to be retransmitted, it takes a certain amount of processing time to complete the positioning of the retransmission WQE, the acquisition of the retransmission WQE content, and the acquisition of the retransmission data. The behavior of the requester in the retransmission case is as follows: Figure 5The diagram shows the command execution of the requesting end when retransmission occurs in an existing RDMA network card. During the retransmission period, the requesting end continues to work, so a large number of RDMA requests may still be sent out. These RDMA requests reach the responding end, and the responding end will still process the received requests in sequence. However, these requests will be sent to the receiving end again by the retransmission N scheme, causing the receiving end to execute the same RDMA request twice. If it is a large number of RDMA READ requests, the responding end will need to spend a lot of PCIe read latency to process unnecessary requests, which seriously hinders the execution of retransmission requests. This will bring a large amount of invalid data to the link, reducing the effective bandwidth of the network, and also increasing the network latency and retransmission recovery time.

[0063] To this end, the present invention provides a data retransmission method for remote direct memory access, which is applied to the RDMA network card request terminal. Figure 1 , Figure 1 An embodiment of the present invention provides a flowchart of a data retransmission method for remote direct memory access. As shown in the figure, the data retransmission method includes: a normal transmission mode and a retransmission mode.

[0064] In normal transmission mode, new WQE commands are obtained and cached in the QP queue of the first WQE cache module. New WQE commands are obtained from the QP queue in the cache order to generate corresponding framing and frame sending requests. The sending side protocol stack completes framing according to the framing request and sends it to the RDMA network card response terminal. The new WQE commands in the QP queue are transmitted through the new WQE command channel.

[0065] In the retransmission mode, a retransmission command is generated in response to detecting a NACK response or a timeout pulse, the QP queue corresponding to the retransmission command is marked as a retransmission QP queue, the new WQE command channel of the retransmission QP queue is blocked, its retransmission WQE command channel is opened, and the retransmission queue-jumping task of the retransmission QP queue is executed based on the stored ORT; after the retransmission queue-jumping task is completed, the retransmission WQE command channel of the retransmission QP queue is closed, its new WQE command channel is opened, and normal transmission mode operation is resumed.

[0066] In an optional embodiment, each QP queue stores an ORT, which is used to store data information of a WQE command in the QP queue that has generated a send request but has not received a response from a response terminal, wherein the data information is used to indicate metadata information of a data packet corresponding to the WQE command or a mapping relationship between a data packet corresponding to the WQE command and the metadata information.

[0067] Optionally, when the RDMA network card needs to serve a large number (greater than 1k) of QP queues, the ORT may only store the mapping relationship between the data packets corresponding to the WQE commands and their metadata.

[0068] In an optional implementation, the process of retransmitting a queue-jumping task in a retransmission QP queue includes:

[0069] Step 1: Determine metadata information of the retransmitted data packet based on detection of a NACK response or a timeout pulse;

[0070] Step 2: According to the metadata information of the retransmitted data packet, find the ORT of the retransmitted QP queue to determine the retransmitted WQE command corresponding to the metadata information of the retransmitted data packet;

[0071] Step 3: Block the new WQE command channel of the retransmission QP queue to shield the new WQE command transmission of the retransmission QP queue;

[0072] Step 4: Generate corresponding framing and frame sending requests based on the obtained retransmission WQE command. The sending side protocol stack completes framing according to the framing request and sends it to the RDMA network card response terminal. The retransmission WQE command is transmitted through the retransmission WQE command channel.

[0073] In this embodiment, for the retransmission QP queue, during the execution of the retransmission queue interruption task, the sending side protocol stack silently discards the framing and frame sending requests of the new WQE command, and only executes the framing and frame sending requests corresponding to the retransmission WQE command to achieve retransmission queue interruption.

[0074] In an optional embodiment, the retransmission QP queue marking process includes: generating a retransmission command in response to detecting a NACK response or a timeout pulse, configuring a queue mask of the sending side protocol stack, turning on the mask enable of the QP queue corresponding to the retransmission command, and marking it as a retransmission QP queue.

[0075] Accordingly, after the retransmission queue-jumping task is completed, the retransmission WQE command channel of the retransmission QP queue is closed, and its new WQE command channel is opened, the process also includes: configuring the queue shield of the sending side protocol stack, turning off the shielding enable of the QP queue corresponding to the retransmission command, and canceling the retransmission mark of the QP queue.

[0076] It should be noted that when the ORT reaches the storage threshold, the new WQE command obtained from the QP queue of the first WQE cache module needs to be stored in the second WQE cache module as the unexecuted WQE command of the QP queue to alleviate the head-of-queue blocking problem of the first WQE cache module.

[0077] Correspondingly, after the retransmission queue-jumping task is completed, if there are unexecuted WQE commands stored in the second WQE cache module, all unexecuted WQE commands are obtained and sent, and then the sending task is executed for the new WQE command transmitted through the new WQE command channel; wherein, the unexecuted WQE commands are transmitted through the unexecuted WQE command channel.

[0078] That is, in this embodiment, three types of WQE command channels are set up on the RDMA network card request terminal, namely, a new WQE command channel, an unexecuted WQE command channel, and a retransmitted WQE command channel. The new WQE command channel transmits brand-new WQE commands obtained from the network card host driver; the unexecuted WQE command channel transmits cached WQE commands that have been obtained but not executed; and the retransmitted WQE command channel transmits WQE commands that are currently waiting to be retransmitted.

[0079] See Figure 8 The core workflow diagram of a remote direct memory access data retransmission terminal provided by an embodiment of the present invention is shown. As shown in the figure, in this embodiment, the retransmission WQE command channel has the highest admission priority, that is, at any time, the retransmission WQE command transmitted by the retransmission WQE command channel needs to be executed first. The admission priority of the unexecuted WQE command channel is only lower than that of the retransmission WQE command channel. That is, at any time, when there is no command request entering the retransmission WQE command channel and the ORT corresponding to the request to enter the QP queue does not reach the storage threshold, the unexecuted WQE command from the unexecuted WQE command channel can be executed first. The new WQE command channel has the lowest admission priority, that is, only when there is no command request entering the retransmission WQE command channel and the unexecuted WQE command channel and the ORT corresponding to the request to enter the QP queue does not reach the storage threshold, the new WQE command from the new WQE command channel will be executed.

[0080] It should be noted that if the number of retransmission operations exceeds a preset threshold during the execution of a retransmission queue-jumping task in a retransmission QP queue, the communication transmission of the QP queue is considered to have a problem, and all WQE commands in the QP queue are admitted and discarded. When the unexecuted WQE command channel is valid and the ORT corresponding to the request to enter the QP queue reaches the storage threshold, the unexecuted WQE command channel is considered blocked, a blocking command is written, and the caching of the QP queue is stopped.

[0081] Further, see Figure 7 The structural diagram of the ORT (Outstanding Request Table) shown in FIG. 1 illustrates the writing and reading process of the ORT of this embodiment.

[0082] In an optional embodiment, the ORT writing process includes:

[0083] Step 1: The host driver dynamically configures the storage threshold of the ORT corresponding to each QP queue based on the activity level of the QP queue and the bandwidth-delay product of the current network topology. The ORT is equipped with a head pointer, tail pointer, and retransmission pointer indicating QP storage;

[0084] Step 2: After generating corresponding framing and sending frame requests for the acquired new WQE command or the unexecuted WQE command, write the data information of the WQE command into the ORT;

[0085] The data information of the WQE command is the WQE command information or the metadata information of the WQE; the WQE command information includes the WQE completion QP number, the complete WQE content and the WQE first sequence number; the WQE metadata information includes the WQE sequence number, the WQE first sequence number, the WQE total length and the WQE offset;

[0086] Optionally, when the RDMA network card needs to serve a large number (greater than 1k) of QP queues, only the metadata information of the WQE may be stored.

[0087] Step 3: After writing is completed, the head pointer of the QP queue corresponding to the ORT is increased by one.

[0088] In an optional embodiment, in normal transmission mode, the ORT reading process includes:

[0089] Step 1: In response to detecting an ACK response, generate a read request message based on the received QP number, message number, and sequence number;

[0090] Step 2: According to the read request information, index and read the data information of the WQE command stored in the ORT;

[0091] Step 3: After the read is completed, a read reply feedback message is generated, and the tail pointer of the QP queue corresponding to the ORT is increased by one.

[0092] In an optional embodiment, in the retransmission mode, the ORT reading process includes:

[0093] Step 1: In response to detecting a NACK response or a timeout pulse, read the ORT according to the QP number, message number, and sequence number corresponding to the NACK response or the timeout pulse;

[0094] Step 2: Move the retransmission pointer of the ORT corresponding to the QP queue that needs to be retransmitted to the current read pointer;

[0095] Step 3: Index and read the data information of the WQE command stored in the QRT from the retransmission pointer;

[0096] Step 4: The data information of the read WQE command is transmitted through the retransmission WQE command channel;

[0097] Step 5: Each time the ORT is read, the retransmission pointer of the QP queue corresponding to the ORT is increased by one until the retransmission pointer moves to the write pointer.

[0098] The embodiment of the present invention further provides a remote direct memory access data retransmission terminal, which serves as an RDMA network card request terminal to implement the above-mentioned remote direct memory access data retransmission method. Figure 6 The embodiment of the present invention provides a schematic diagram of the structure of a data retransmission terminal for remote direct memory access. As shown in the figure, the data retransmission terminal includes: a WQE acquisition module, a first WQE cache module, an ORT management module, a second WQE cache module, a command execution module, a timer module, a retransmission management module and a QP scheduler.

[0099] In an optional embodiment, the WQE acquisition module uses DMA to acquire WQE commands in the queue ring buffer from the host memory. The first WQE cache module is used to temporarily store the WQE commands acquired by the WQE acquisition module and perform order preservation, wherein the cache is implemented in the form of a first-in-first-out memory. The ORT management module is used to store the ORT and implement the reading and writing of the ORT and the sending of retransmission commands; the second WQE cache module is used to store unexecuted WQE commands when the ORT reaches the storage threshold to alleviate the head-of-line blocking problem of the first WQE cache module; wherein the cache is implemented in the form of a first-in-first-out memory.

[0100] In an optional embodiment, the command execution module is used to generate corresponding framing and framing requests for the acquired new WQE commands, retransmitted WQE commands and unexecuted WQE commands and send them to the sending side protocol stack, and also perform corresponding operations based on the received information of the receiving side protocol stack, wherein the received information includes NACK response, timeout pulse and ACK response.

[0101] In this embodiment, the command execution module is provided with a sending channel and a receiving channel. The sending channel is used to transmit the local command of the command execution module, including framing and frame request, and retransmission shield settings, to the sending-side protocol stack. The receiving channel is used to transmit the frame information of the reception response to the command execution module.

[0102] In an optional embodiment, the timer module is configured to time the response time of the WQE command request in each QP queue and count the number of retransmission operations in each QP queue. Optionally, the timer module uses a register file to set a corresponding time timer and retransmission counter for each QP queue.

[0103] In an optional implementation, the retransmission management module is used to control the WQE acquisition module, the first WQE cache module, the ORT management module, the command execution module, the second WQE cache module and the timer module to implement the remote direct memory access data retransmission method.

[0104] In this embodiment, the retransmission management module is responsible for managing the start, refresh and stop of the time timer; is responsible for refreshing the retransmission counter; is responsible for processing the NACK response from the receiving side protocol stack and the timeout pulse of the time timer, cooperating with the ORT to achieve a rapid response to the retransmission, cooperating with the sending side protocol stack to complete the retransmission queue interruption function, and resuming normal operation after the retransmission is completed.

[0105] In an optional implementation, the QP scheduler is configured to disable QP scheduling and stop acquiring new WQEs when the ORT storage reaches a threshold.

[0106] The specific execution steps of each module of the remote direct memory access data retransmission terminal in this embodiment when implementing the data retransmission method are similar to the steps of the above-mentioned remote direct memory access data retransmission method, and are not repeated here.

[0107] The remote direct memory access (RDMA) data retransmission method and terminal of the present invention design and optimize a remote direct memory access (RDMA) data retransmission system. The ORT (Optical Response Time) allows for rapid location of retransmitted packets and quick acquisition of metadata information during retransmission, reducing the requester's wait time for retransmitted commands. The three WQE (Warning Queuing Expectation) command channels ensure priority execution of retransmitted WQEs, while the retransmission queue mask eliminates duplicate execution issues on the requester side, reducing network latency for retransmitted packets and increasing effective network bandwidth.

[0108] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art will appreciate that the above-mentioned devices and methods can be implemented using computer-executable instructions and / or contained in processor control code, for example, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, or can be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0109] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations are intended to cover non-exclusive inclusion, such that an article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the article or device comprising the element.

[0110] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A data retransmission method for remote direct memory access, characterized in that: Applied to RDMA network card request terminal, including: normal transmission mode and retransmission mode, among which, In normal transmission mode, a new WQE command is obtained and cached in the QP queue of the first WQE cache module. The new WQE command is obtained from the QP queue in the cache order to generate corresponding framing and frame sending requests. The sending side protocol stack completes framing according to the framing request and sends it to the RDMA network card response terminal. The new WQE command in the QP queue is transmitted through the new WQE command channel. In retransmission mode, in response to detecting a NACK response or a timeout pulse, a retransmission command is generated, the QP queue corresponding to the retransmission command is marked as a retransmission QP queue, the new WQE command channel of the retransmission QP queue is blocked, its retransmission WQE command channel is opened, and the retransmission queue-jumping task of the retransmission QP queue is executed based on the stored ORT; after the retransmission queue-jumping task is completed, the retransmission WQE command channel of the retransmission QP queue is closed, its new WQE command channel is opened, and normal transmission mode operation is resumed; Each QP queue stores an ORT corresponding to the request but not received a response from the terminal. The ORT is used to store the data information of the WQE command in the QP queue, wherein the data information is used to indicate the metadata information of the data packet corresponding to the WQE command or the mapping relationship between the data packet corresponding to the WQE command and the metadata information. The process of the retransmission queue-jumping task of the retransmission QP queue includes: determining metadata information of the retransmitted data packet based on detecting a NACK response or a timeout pulse; According to the metadata information of the retransmitted data packet, searching the ORT of the retransmitted QP queue to determine the retransmitted WQE command corresponding to the metadata information of the retransmitted data packet; Blocking a new WQE command channel of the retransmission QP queue to shield the transmission of new WQE commands of the retransmission QP queue; The corresponding framing and sending frame request are generated according to the obtained retransmission WQE command, and the sending side protocol stack completes the framing according to the framing request and sends it to the RDMA network card response terminal; wherein, the retransmission WQE command is transmitted through the retransmission WQE command channel.

2. The remote direct memory access data retransmission method according to claim 1, characterized in that: In response to detecting a NACK response or a timeout pulse, generating a retransmission command, marking a QP queue corresponding to the retransmission command as a retransmission QP queue, including: In response to detecting a NACK response or a timeout pulse, a retransmission command is generated. The queue mask of the sending side protocol stack is configured to turn on the masking enable of the QP queue corresponding to the retransmission command and mark it as a retransmission QP queue.

3. The remote direct memory access data retransmission method according to claim 2, characterized in that: For the retransmission QP queue, during the execution of the retransmission queue interruption task, the sending side protocol stack silently discards the framing and sending frame requests of the new WQE command, and only executes the framing and sending frame requests corresponding to the retransmission WQE command to achieve retransmission queue interruption.

4. The remote direct memory access data retransmission method according to claim 2, characterized in that: After the retransmission queue-jumping task is completed, the retransmission WQE command channel of the retransmission QP queue is closed and its new WQE command channel is opened, the method further includes: configuring the queue shield of the sending side protocol stack, turning off the shielding enable of the QP queue corresponding to the retransmission command, and canceling the retransmission mark of the QP queue.

5. The remote direct memory access data retransmission method according to claim 1, wherein: When the ORT reaches the storage threshold, the new WQE command obtained from the QP queue of the first WQE cache module is stored in the second WQE cache module as an unexecuted WQE command of the QP queue; Correspondingly, after the retransmission queue-jumping task is completed, if there are unexecuted WQE commands stored in the second WQE cache module, all unexecuted WQE commands are obtained and sent, and then the sending task is executed for the new WQE command transmitted through the new WQE command channel; wherein, the unexecuted WQE command is transmitted through the unexecuted WQE command channel.

6. The remote direct memory access data retransmission method according to claim 5, characterized in that: The ORT writing process includes: The host driver dynamically configures the storage threshold of the ORT corresponding to each QP queue based on the activity level of the QP queue and the bandwidth-delay product of the current network topology. The ORT is set with a head pointer, tail pointer, and retransmission pointer indicating QP storage; After generating corresponding framing and sending frame requests for the acquired new WQE command or the unexecuted WQE command, the data information of the WQE command is written into the ORT; The data information of the WQE command is WQE command information or WQE metadata information; the WQE command information includes a WQE completion QP number, complete WQE content, and a WQE first sequence number; the WQE metadata information includes a WQE sequence number, a WQE first sequence number, a WQE total length, and a WQE offset; After writing is completed, the head pointer of the QP queue corresponding to the ORT is increased by one.

7. The remote direct memory access data retransmission method according to claim 6, characterized in that: In normal transmission mode, the ORT reading process includes: In response to detecting the ACK response, generating a read request message based on the received QP number, message number, and sequence number; According to the read request information, index and read the data information of the WQE command stored in the ORT; After the read is completed, a read reply feedback message is generated, and the tail pointer of the QP queue corresponding to the ORT is increased by one; In the retransmission mode, the ORT reading process includes: In response to detecting a NACK response or a timeout pulse, reading the ORT according to the QP number, message number, and sequence number corresponding to the NACK response or the timeout pulse; Move the retransmission pointer of the ORT corresponding to the QP queue that needs to be retransmitted to the current read pointer; Index and read the data information of the WQE command stored in QRT from the retransmission pointer; The data information of the read WQE command is transmitted through the retransmission WQE command channel; Each time the ORT is read, the retransmission pointer of the QP queue corresponding to the ORT is increased by one until the retransmission pointer moves to the write pointer.

8. A remote direct memory access data retransmission terminal, characterized in that: The data retransmission method for remote direct memory access according to any one of claims 1 to 7 is implemented as an RDMA network card request terminal, and the data retransmission terminal includes: The WQE acquisition module uses DMA to obtain WQE commands in the queue ring buffer from the host memory; A first WQE cache module, configured to temporarily store the WQE commands acquired by the WQE acquisition module and maintain order, wherein the cache is implemented in a first-in-first-out memory format; ORT management module, used to store ORT and implement ORT reading and writing and sending retransmission commands; A second WQE cache module, configured to store unexecuted WQE commands when the ORT reaches a storage threshold, to alleviate the head-of-queue blocking problem of the first WQE cache module; wherein the cache is implemented in a first-in-first-out memory format; a command execution module, configured to generate corresponding framing and frame sending requests for acquired new WQE commands, retransmitted WQE commands, and unexecuted WQE commands, and send them to a transmitting-side protocol stack, and to perform corresponding operations based on received information from a receiving-side protocol stack, wherein the received information includes a NACK response, a timeout pulse, and an ACK response; The timer module is used to time the response time of the WQE command request in each QP queue and count the number of retransmission operations in each QP queue; The retransmission management module is used to control the WQE acquisition module, the first WQE cache module, the ORT management module, the command execution module, the second WQE cache module and the timer module to implement the data retransmission method of remote direct memory access.

Images