Rdma read message processing method and rdma read message processing unit

Abstract

The application relates to an RDMA read message processing method, an RDMA read message processing unit, a network interface card, an RDMA communication system, a computer readable storage medium and a computer program product. According to the current number of read response messages to be received by a local RDMA end and the maximum read response message window threshold value allowed by a single QP, the number n of request messages that can be currently sent is determined; according to the receiving sequence of the RDMA read message and the starting position of the current RDMA read message to be processed, one or more RDMA read messages are selected from the RDMA read messages to be processed in sequence, and n request messages are generated according to the selected one or more RDMA read messages and are sent to a remote RDMA end. The effective bandwidth of the network is improved.

Description

Technical Field

[0001] This application relates to the field of chip technology, and in particular to an RDMA read message processing method, an RDMA read message processing unit, a network interface card, an RDMA communication system, a computer-readable storage medium, and a computer program product. Background Technology

[0002] In Remote Direct Memory Access (RDMA) communication, an RDMA read operation refers to the operation of actively reading data from remote memory from the local RDMA endpoint. During the execution of an RDMA read operation, after receiving an RDMA read request message, the remote RDMA endpoint reads data from memory and packages it into a read response message to return to the local RDMA endpoint. Throughout the entire execution process, the remote CPU does not need to participate and is unaware of the data being read from memory. In existing technology, the local RDMA endpoint sends the entire RDMA read message to the remote RDMA endpoint in one request message. After receiving the RDMA read request message, the remote RDMA endpoint reconstructs the entire RDMA read message, records it locally, and then, based on the working status of the remote RDMA endpoint, sequentially returns all the data that needs to be read in the RDMA read message.

[0003] According to existing RDMA protocols, a single RDMA read message can read up to 2GB of memory data at a time. When a remote RDMA endpoint processes an RDMA read request message, it needs to generate a maximum of 8 million RDMA read response messages to the local RDMA endpoint. In other words, the local RDMA endpoint only needs to send one RDMA read request message to the remote RDMA endpoint, and the remote RDMA endpoint needs to return 8 million RDMA read response messages to complete the processing of that request message. This leads to the following problems: 1. A severe imbalance in network communication traffic between the local and remote endpoints and between the remote and local endpoints. The local RDMA endpoint only needs to send a few RDMA read request messages to the remote RDMA endpoint consecutively to easily fill up all the bandwidth of the remote RDMA endpoint, crowding out its resources, affecting its own sending tasks, and forming a network traffic attack. 2. The remote RDMA endpoint continuously returns a large number of RDMA read response messages to the local RDMA endpoint, and the message window is not controlled by the remote RDMA endpoint, easily leading to network traffic congestion. Once packet loss occurs, all subsequent response messages following the discarded RDMA read response message become invalid. The remote RDMA end cannot detect the packet loss in time and cannot stop sending invalid response messages, resulting in a large number of invalid messages flooding the network and severely reducing the network's effective bandwidth. Summary of the Invention

[0004] Therefore, it is necessary to provide an RDMA read message processing method, an RDMA read message processing unit, a network interface card, an RDMA communication system, a computer-readable storage medium, and a computer program product that can improve the effective bandwidth of the network, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides an RDMA read message processing method, including:

[0006] Based on the number of read response messages to be received at the local RDMA terminal and the maximum read response message window threshold allowed by a single QP (Queue Pairs), determine the number of request messages n that can be sent at the moment.

[0007] Based on the receiving order of the RDMA read messages and the starting position of the current pending RDMA read messages, one or more RDMA read messages are selected sequentially from the pending RDMA read messages. Based on the selected one or more RDMA read messages, n request messages are generated and sent to the remote RDMA end.

[0008] Receive the read response message returned by the remote RDMA terminal, process the read response message, and update the current number of read response messages to be received.

[0009] Secondly, this application also provides an RDMA read message processing unit, comprising:

[0010] The message splitting module is used to determine the number n of request messages that can be sent at present based on the number of read response messages to be received at the local RDMA terminal and the maximum read response message window threshold allowed by a single QP; and to select one or more RDMA read messages from the pending RDMA read messages in sequence according to the receiving order of the RDMA read messages and the starting position of the pending RDMA read messages.

[0011] The first message sending and processing module is used to generate n request messages based on one or more selected RDMA read messages and send them to the remote RDMA end;

[0012] The first message receiving and processing module is used to receive the read response message returned by the remote RDMA terminal, process the read response message, and update the current number of read response messages to be received.

[0013] Thirdly, this application also provides a network interface card, including the aforementioned RDMA read message processing unit.

[0014] Fourthly, this application also provides an RDMA communication system, including the aforementioned RDMA read message processing unit.

[0015] Fifthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described RDMA read message processing method.

[0016] Sixthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described RDMA read message processing method.

[0017] The aforementioned RDMA read message processing method, RDMA read message processing unit, network interface card, RDMA communication system, computer-readable storage medium, and computer program product, wherein the local RDMA end determines the number n of request messages that can be sent at present based on the current number of read response messages to be received at the local RDMA end and the maximum read response message window threshold allowed by a single QP; according to the receiving order of RDMA read messages and the starting position of the currently pending RDMA read messages, one or more RDMA read messages are selected sequentially from the pending RDMA read messages, and n request messages are generated and sent to the remote RDMA end based on the selected one or more RDMA read messages; the read response messages returned by the remote RDMA end are received, the read response messages are processed, and the current number of read response messages to be received is updated. This method of controlling the sending of request messages based on the maximum read response message window threshold allowed by a single QP can reduce the imbalance of network communication traffic between the local end and the remote end and between the remote end and the local end, reduce network traffic congestion, and greatly improve the processing speed of RDMA read messages, thereby increasing the effective bandwidth of the network. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of a communication system in one embodiment;

[0020] Figure 2 This is a flowchart illustrating an RDMA message reading processing method in one embodiment;

[0021] Figure 3This is a block diagram of the RDMA read message processing unit in one embodiment;

[0022] Figure 4 This is an example diagram of an RDMA read message processing method in one embodiment. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0024] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0025] See Figure 1 As shown, the RDMA read message processing method provided in this application embodiment can be applied to... Figure 1 In the communication system shown, Figure 1 The communication system shown includes a local RDMA terminal and a remote RDMA terminal. The local RDMA terminal and the remote RDMA terminal can communicate with each other.

[0026] In some embodiments, the RDMA read message processing method provided in this application refers to... Figure 2 As shown, the steps performed on the local RDMA side include:

[0027] Step 202: Determine the number of request packets n that can be sent at present based on the number of read response packets to be received at the local RDMA terminal and the maximum read response packet window threshold allowed by a single QP.

[0028] In this embodiment, the request message is an RDMA read request message, and the read response message is an RDMA read response message.

[0029] Optionally, a scaling parameter N is configured on the local RDMA end. This scaling parameter N is referred to as the message splitting factor in this embodiment of the application. It is used to represent the maximum number of RDMA read response messages that can be generated by each RDMA read request message. The value of N can be flexibly set according to the actual situation. For example, N=2, which means that for one RDMA read request message, a maximum of two RDMA read response messages can be generated.

[0030] Optionally, the local RDMA terminal can monitor the number of read response messages to be received.

[0031] Optionally, the maximum read response message window threshold allowed per QP can be configured as the bandwidth delay product (BDP). This ensures the effective bandwidth of RDMA while reducing the overshoot of the RDMA read response message returned by the remote RDMA end.

[0032] In step 202, the number of read response messages that the local RDMA can still receive can be determined first based on the number of read response messages to be received at the local RDMA end and the maximum read response message window threshold allowed by a single QP. Then, the number of request messages n that can be sent can be determined based on the number of read response messages that can still be received and the message splitting factor.

[0033] Step 204: Based on the receiving order of the RDMA read messages and the starting position of the current pending RDMA read messages, select one or more RDMA read messages sequentially from the pending RDMA read messages, generate n request messages based on the selected one or more RDMA read messages, and send them to the remote RDMA end.

[0034] Optionally, the local RDMA end processes each RDMA read message sequentially according to the order in which they are received.

[0035] For each received RDMA read message, under certain conditions, the local RDMA end will split it into multiple sub-messages, generate corresponding request messages for each sub-message, and send all request messages to the remote RDMA end. Under other conditions, a certain number of sub-messages will be extracted from the multiple sub-messages, and corresponding request messages for these sub-messages will be generated and sent to the remote RDMA end. In this case, there may be remaining sub-messages. In other words, the RDMA read message currently pending processing at the local RDMA end may be a complete RDMA read message (i.e., an unsplit RDMA read message), or it may be the remaining unprocessed sub-messages of a split RDMA read message. Optionally, the starting position of the currently pending RDMA read message can be used to determine whether it is an unsplit RDMA read message or a remaining unprocessed sub-message of a split RDMA read message.

[0036] Optionally, the local RDMA end can sequentially select one or more RDMA read messages from the pending RDMA read messages, generate n request messages based on the selected one or more RDMA read messages, and send them to the remote RDMA end.

[0037] Step 206: Receive the read response message returned by the remote RDMA terminal, process the read response message, and update the current number of read response messages to be received.

[0038] The remote RDMA end processes the split request message as a normal RDMA read request message, without needing to know whether multiple adjacent request messages belong to the same RDMA read message. When processing each request message, the remote RDMA end only needs to read data of length N*PMTU from the starting address VA_n of the request message, generate N read response messages, and return them to the local RDMA end.

[0039] Optionally, after receiving a request message, the remote RDMA end generates a response message descriptor for the request message and sends the response message descriptor to the read response queue for scheduling. The remote RDMA end can schedule the response message descriptors in the read response queue according to traffic management and congestion management mechanisms. For the currently scheduled response message descriptor, data of length N*PMTU can be read starting from the starting address in the response message descriptor to generate N read response messages, and these N read response messages are returned to the local RDMA end. After receiving the read response messages, the local RDMA end processes the read response messages, updates the number of read response messages to be received, generates a CQE based on the response messages, and writes the CQE to the local host.

[0040] In the above embodiments, the local RDMA end determines the number n of request packets that can be sent at present based on the number of read response packets to be received at the local RDMA end and the maximum read response packet window threshold allowed by a single QP; according to the receiving order of RDMAread messages and the starting position of the RDMA read messages to be processed, it sequentially selects one or more RDMA read messages from the RDMA read messages to be processed, generates n request packets based on the selected one or more RDMA read messages, and sends them to the remote RDMA end; it receives the read response packets returned by the remote RDMA end, processes the read response packets, and updates the number of read response packets to be received. This method of controlling the sending of request packets based on the maximum read response packet window threshold allowed by a single QP can reduce the imbalance of network communication traffic between the local end and the remote end and between the remote end and the local end, reduce network traffic congestion, and greatly improve the processing speed of RDMA read messages, thereby increasing the effective bandwidth of the network.

[0041] In some embodiments, determining the number n of request packets that can be sent at present based on the current number of read response packets to be received at the local RDMA end and the maximum read response packet window threshold allowed by a single QP specifically includes:

[0042] The number of request messages n that can be sent can be calculated using the following formula:

[0043] n = (wnd - k) / N

[0044] Where wnd is the maximum read response message window threshold allowed by a single QP, and k is the number of read response messages to be received at present.

[0045] Specifically, as described above, the maximum read response message window threshold wnd allowed per QP can be configured as the bandwidth delay product (BDP). Subtracting the number of read response messages k to be received from wnd yields the number of read response messages that the local RDMA end can still receive. N is the message splitting factor, which refers to the maximum number of RDMA read response messages that each RDMA read request message can generate. Dividing wnd - k by N gives the number of request messages n that can be sent.

[0046] Since the number of read response messages corresponding to each RDMA read request message is fixed (N), the local RDMA end can calculate the number of RDMA read response messages waiting to be returned, i.e., the number of read response messages to be received, based on the number of RDMA read request messages that have been sent but have not yet returned.

[0047] Specifically, the number of currently pending read response messages can be calculated using the following formula:

[0048]

[0049] Where f is the number of request messages that have not received a corresponding read response message among all request messages sent by the local RDMA terminal, and N is a preset message splitting factor.

[0050] The above embodiments provide a specific method for calculating the number of read response messages to be received and a specific method for calculating the number n of request messages that can be sent. Subsequently, the sending of request messages is controlled based on n, which greatly improves the processing speed of RDMA read messages and increases the effective bandwidth of the network.

[0051] In some embodiments, the step of sequentially selecting one or more RDMA read messages from the pending RDMA read messages according to the receiving order of the RDMA read messages and the starting position of the currently pending RDMA read messages, and generating n request messages based on the selected one or more RDMA read messages and sending them to the remote RDMA end specifically includes:

[0052] For any currently pending RDMA read message, if the currently pending RDMA read message is determined to be an unsplit RDMA read message based on the starting position, then calculate the number of request messages that the currently pending RDMA read message can be split into according to the set rules, X1.

[0053] If X1 is greater than the number of request messages that can be sent at present, n, then the RDMA read message that is currently pending is split into multiple sub-messages according to the set rules, n sub-messages are obtained from the multiple sub-messages, n request messages are generated according to the n sub-messages and sent to the remote RDMA end, and the number of request messages that can be sent at present is updated to 0.

[0054] If X1 is less than or equal to the number of request messages n that can be sent, then X1 request messages are generated based on the current pending RDMA read message and sent to the remote RDMA end. The number of request messages that can be sent is updated to Y1, where Y1 = n - X1. Furthermore, the number of request messages X2 that the next pending RDMA read message can be split into according to the set rules is calculated. If X2 is less than or equal to Y1, then X2 request messages are generated based on the next pending RDMA read message and sent to the remote RDMA end. The number of request messages that can be sent is updated to Y2, where Y2 = Y1 - X2. Furthermore, the number of request messages X3 that the next pending RDMA read message can be split into is calculated. The relationship between X3 and Y2 is determined, and corresponding request messages are generated until n request messages are generated. The number of request messages that can be sent is then updated to 0. If X2 is greater than Y1, then the next pending RDMA read message is split according to the set rules. The read message selects Y1 sub-messages from the sub-messages split from the next pending RDMA read message, generates Y1 request messages based on the Y1 sub-messages, sends them to the remote RDMA end, and updates the number of request messages that can be sent to 0.

[0055] As described above, the RDMA read message currently pending processing on the local RDMA end may be a complete RDMA read message, i.e., an unsplit RDMA read message, or it may be the remaining unprocessed sub-message of a split RDMA read message. Whether it is an unsplit RDMA read message or a remaining unprocessed sub-message of a split RDMA read message can be determined based on the starting position of the currently pending RDMA read message.

[0056] Optionally, for any currently pending RDMA read message, if the starting position of the message is consistent with the starting address of the corresponding complete RDMA read message, then the currently pending RDMA read message is determined to be an unsplit RDMA read message; if they are inconsistent, then the currently pending RDMA read message is determined to belong to the remaining unprocessed sub-message part of the split RDMA read message.

[0057] Optionally, if the current RDMA read message to be processed is determined to be an unsplit RDMA read message based on the starting position, then calculate the number of request messages that the current RDMA read message to be processed can be split into according to the set rules, X1.

[0058] Specifically, the length of the currently pending RDMA read message can be obtained. Based on this length, the preset message splitting factor, and the preset size of each read response message PMTU, the number of request messages that can be split into, X1, can be calculated.

[0059] After calculating the number of request messages X1 that the current pending RDMA read message can be divided into according to the set rules, compare X1 with the number of request messages n that can be sent. If X1 is greater than n, then the current pending RDMA read message is divided into X1 sub-messages. The starting address of the i-th sub-message can be determined as follows:

[0060] ;

[0061] Where VA0 is the starting address of the RDMA read message obtained by splitting the X1 sub-messages, N is a preset message splitting factor, PMTU is a preset size of each read response message, and VA_i is the starting address of the i-th sub-message. Then, n sub-messages are sequentially obtained from the X1 sub-messages, a request message corresponding to each sub-message is generated, and the n request messages are sent to the remote RDMA end, and the number of request messages that can be sent is updated to 0. In this case, there are remaining sub-messages, which will be processed sequentially by the local RDMA end.

[0062] If X1 is less than or equal to n, the current pending RDMA read message is split into X1 sub-messages. The starting address of the i-th sub-message is also determined by the formula above. Then, a request message corresponding to each sub-message is generated, and X1 request messages are sent to the remote RDMA end. The number of request messages that can be sent is updated to Y1, where Y1 = n - X1. Furthermore, the number of request messages X2 that the next pending RDMA read message can be split into according to the set rules is calculated. The calculation method for X2 is similar to that for X1. X2 is compared with Y1. If X2 is less than or equal to Y1, the next pending RDMA read message is split into X2 sub-messages. Then, a request message corresponding to each sub-message is generated, and X2 request messages are sent to the remote RDMA end. The number of request messages that can be sent is updated to Y2, where Y2 = Y1 - X2. The process further calculates the number of request messages (X3) that the next pending RDMA read message can be split into. Following the same logic, X3 is compared with Y2. If X3 is less than or equal to Y2, the next pending RDMA read message is split into X3 sub-messages. Then, a request message corresponding to each sub-message is generated, and the X3 request messages are sent to the remote RDMA end. This process continues until the number of sent request messages reaches n. At this point, the number of request messages that can be sent is updated to 0. In the above process, if a comparison result is greater than, for example, if X2 is greater than Y1, the next pending RDMA read message is split into X2 sub-messages according to the message splitting method described above. Then, Y1 sub-messages are sequentially selected from the X2 sub-messages, a request message corresponding to each sub-message is generated, and the Y1 request messages are sent to the remote RDMA end. The number of request messages that can be sent is then updated to 0.

[0063] The above embodiments provide a specific implementation method for sending request messages when the current pending RDMA read message is an unsplit RDMA read message. This method of controlling the sending of request messages based on the maximum read response message window threshold allowed by a single QP can reduce network traffic congestion and greatly improve the processing speed of RDMA read messages, thereby increasing the effective bandwidth of the network.

[0064] In some embodiments, the RDMA read message processing method provided in this application further includes: for any currently pending RDMA read message, if it is determined from the starting position that the currently pending RDMA read message belongs to the remaining unprocessed sub-message portion of the split RDMA read message;

[0065] Calculate the number U of request messages that can be generated from the remaining unprocessed sub-message portion;

[0066] If U is greater than the number of request messages that can be sent at present, n sub-messages are selected from the remaining unprocessed sub-messages, n request messages are generated based on the n sub-messages and sent to the remote RDMA end, and the number of request messages that can be sent at present is updated to 0.

[0067] If U is less than or equal to the number of request messages n that can be sent, then U request messages are generated based on the remaining unprocessed sub-messages and sent to the remote RDMA end. The number of request messages that can be sent is updated to V1, where V1 = nU. Furthermore, the number of request messages U1 that the next pending RDMA read message can be split into according to the set rules is calculated. If U1 is greater than V1, then the next pending RDMA read message is split according to the set rules. V1 sub-messages are selected from the sub-messages after the next pending RDMA read message is split. V1 request messages are generated based on the V1 sub-messages and sent to the remote RDMA end. The number of request messages that can be sent is updated to 0.

[0068] As described above, the starting position of the currently pending RDMA read message can be used to determine whether it is an unsplit RDMA read message or a remaining unprocessed sub-message of a split RDMA read message.

[0069] If the current RDMA read message to be processed is determined to belong to the remaining unprocessed sub-message portion of the split RDMA read message based on the starting position, the number of sub-messages contained in the remaining unprocessed sub-message portion can be obtained, and this number is the number U of request messages that can be generated by the remaining unprocessed sub-message portion. Compare U with the number of request messages that can be sent currently, n. If U is greater than n, select n sub-messages sequentially from the remaining unprocessed sub-messages, generate a request message corresponding to each sub-message, send the n request messages to the remote RDMA end, and update the number of request messages that can be sent to 0. If U is less than or equal to n, generate U request messages according to the remaining unprocessed sub-messages and send them to the remote RDMA end, update the number of request messages that can be sent to V1, V1 = nU, and further calculate the number of request messages U1 that the next complete RDMA read message to be processed can be divided into according to the set rules. The following process is similar to the processing process after calculating X1 above. Specifically, compare U1 and V1. If U1 is greater than V1, then the RDMA to be processed... The read message is split into U1 sub-messages. V1 sub-messages are selected sequentially from these U1 sub-messages, and a request message corresponding to each sub-message is generated. These V1 request messages are then sent to the remote RDMA end, and the current number of sendable request messages is updated to 0. If U1 is less than or equal to V1, the pending RDMA read message is split into U1 sub-messages, and a request message corresponding to each sub-message is generated. These U1 request messages are then sent to the remote RDMA end, and the current number of sendable request messages is updated to V2, where V2 = V1 - U. This process continues until the number of sent request messages reaches n, at which point the current number of sendable request messages is updated to 0.

[0070] The above embodiments provide a specific implementation method for sending request messages when the currently pending RDMA read message belongs to the remaining unprocessed sub-message part of the split RDMA read message. This method of controlling the sending of request messages based on the maximum read response message window threshold allowed by a single QP can reduce network traffic congestion and greatly improve the processing speed of RDMA read messages, thereby increasing the effective bandwidth of the network.

[0071] In some embodiments, calculating the number of request packets X1 that the currently pending RDMA read message can be split into according to a set rule specifically includes:

[0072] X1 is calculated using the following formula:

[0073] ;

[0074] Where msg_len is the length of the current RDMA read message to be processed, N is the preset message splitting factor, and PMTU is the preset size of each read response message.

[0075] It should be noted that the above formula is not only applicable to calculating the number of request packets X1 that the current pending RDMA read message can be split into according to the set rules, but also applicable to calculating the number of request packets X1 corresponding to other RDMA read messages. For example, the number of request packets X2 that the next pending RDMA read message can be split into according to the set rules, and the number of request packets X3 that the next pending RDMA read message can be split into, can all be calculated using the above formula. The length of the corresponding RDMA read message can be substituted into msg_len in the above formula.

[0076] The above embodiments provide a specific method for calculating the number of request packets that an RDMA read message can be split into according to a set rule. The calculated number of request packets can be used to control the generation of request packets, thereby reducing network traffic congestion and improving the effective bandwidth of the network.

[0077] In some embodiments, obtaining n sub-messages from a plurality of sub-messages and generating n request messages based on the n sub-messages specifically includes:

[0078] Select n sub-messages sequentially from multiple sub-messages to generate n request messages, where the starting address of the i-th request message is determined as follows:

[0079] ;

[0080] Wherein, VA0 is the starting address of the RDMA read message from which the multiple sub-messages are split, N is the preset message splitting factor, PMTU is the preset size of each read response message, and VA_i is the starting address of the i-th query message.

[0081] Optionally, when splitting the currently pending RDMA read message into multiple sub-messages, the starting address of the i-th sub-message can be calculated according to the above formula. For the i-th sub-message, a corresponding request message can be generated according to preset rules, and the starting address of the request message is consistent with the starting address of the corresponding sub-message. Each sub-message corresponds to one request message.

[0082] It should be noted that, for any RDMA read message, when splitting it into a request message, the above formula can be used to determine the starting address of the request message.

[0083] Optionally, in the process of splitting the RDMA read message, all sub-messages can be split at once, and then the corresponding number of sub-messages can be obtained sequentially from all sub-messages as needed; or the message can be split as needed. For example, if the number of request messages that can be sent is n, then n sub-messages can be split first, and n request messages can be generated and sent to the remote RDMA end. This application embodiment does not limit this.

[0084] The above embodiments provide a specific method for determining the starting address of the request message. This method of splitting RDMA read messages to generate multiple request messages can reduce the imbalance of network communication traffic between the local end and the remote end and between the remote end and the local end.

[0085] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0086] In some embodiments, see Figure 3 As shown, an RDMA read message processing unit is provided, comprising:

[0087] The message splitting module is used to determine the number n of request messages that can be sent at present based on the number of read response messages to be received at the local RDMA terminal and the maximum read response message window threshold allowed by a single QP; and to select one or more RDMA read messages from the pending RDMA read messages in sequence according to the receiving order of the RDMA read messages and the starting position of the pending RDMA read messages.

[0088] The first message sending and processing module is used to generate n request messages based on one or more selected RDMA read messages and send them to the remote RDMA end;

[0089] The first message receiving and processing module is used to receive the read response message returned by the remote RDMA terminal, process the read response message, and update the current number of read response messages to be received.

[0090] In some embodiments, the message splitting module is specifically used for:

[0091] The number of request messages n that can be sent can be calculated using the following formula:

[0092] n = (wnd - k) / N

[0093] Where wnd is the maximum read response message window threshold allowed by a single QP, and k is the number of read response messages to be received at present;

[0094] The number of currently pending read response messages is calculated using the following formula:

[0095]

[0096] Where f is the number of request messages that have not received a corresponding read response message among all request messages sent by the local RDMA terminal, and N is a preset message splitting factor.

[0097] In some embodiments, the first message sending and processing module is specifically used for:

[0098] For any currently pending RDMA read message, if the currently pending RDMA read message is determined to be an unsplit RDMA read message based on the starting position, then calculate the number of request messages that the currently pending RDMA read message can be split into according to the set rules, X1.

[0099] If X1 is greater than the number of request messages that can be sent at present, n, then the RDMA read message that is currently pending is split into multiple sub-messages according to the set rules, n sub-messages are obtained from the multiple sub-messages, n request messages are generated according to the n sub-messages and sent to the remote RDMA end, and the number of request messages that can be sent at present is updated to 0.

[0100] If X1 is less than or equal to the number of request messages n that can be sent, then X1 request messages are generated based on the current pending RDMA read message and sent to the remote RDMA end. The number of request messages that can be sent is updated to Y1, where Y1 = n - X1. Furthermore, the number of request messages X2 that the next pending RDMA read message can be split into according to the set rules is calculated. If X2 is less than or equal to Y1, then X2 request messages are generated based on the next pending RDMA read message and sent to the remote RDMA end. The number of request messages that can be sent is updated to Y2, where Y2 = Y1 - X2. Furthermore, the number of request messages X3 that the next pending RDMA read message can be split into is calculated. The relationship between X3 and Y2 is determined, and corresponding request messages are generated until n request messages are generated. The number of request messages that can be sent is then updated to 0. If X2 is greater than Y1, then the next pending RDMA read message is split according to the set rules. The read message selects Y1 sub-messages from the sub-messages split from the next pending RDMA read message, generates Y1 request messages based on the Y1 sub-messages, sends them to the remote RDMA end, and updates the number of request messages that can be sent to 0.

[0101] In some embodiments, the first message sending and processing module is specifically used for:

[0102] For any currently pending RDMA read message, if it is determined from the starting position that the currently pending RDMA read message belongs to the remaining unprocessed sub-message portion of the split RDMA read message;

[0103] Calculate the number U of request messages that can be generated from the remaining unprocessed sub-message portion;

[0104] If U is greater than the number of request messages that can be sent at present, n sub-messages are selected from the remaining unprocessed sub-messages, n request messages are generated based on the n sub-messages and sent to the remote RDMA end, and the number of request messages that can be sent at present is updated to 0.

[0105] If U is less than or equal to the number of request messages n that can be sent, then U request messages are generated based on the remaining unprocessed sub-messages and sent to the remote RDMA end. The number of request messages that can be sent is updated to V1, where V1 = nU. Furthermore, the number of request messages U1 that the next pending RDMA read message can be split into according to the set rules is calculated. If U1 is greater than V1, then the next pending RDMA read message is split according to the set rules. V1 sub-messages are selected from the sub-messages after the next pending RDMA read message is split. V1 request messages are generated based on the V1 sub-messages and sent to the remote RDMA end. The number of request messages that can be sent is updated to 0.

[0106] In some embodiments, the first message sending and processing module is specifically used for:

[0107] X1 is calculated using the following formula:

[0108] ;

[0109] Where msg_len is the length of the current RDMA read message to be processed, N is the preset message splitting factor, and PMTU is the preset size of each read response message.

[0110] In some embodiments, the first message sending and processing module is specifically used for:

[0111] Select n sub-messages sequentially from multiple sub-messages to generate n request messages, where the starting address of the i-th request message is determined as follows:

[0112] ;

[0113] Wherein, VA0 is the starting address of the RDMA read message from which the multiple sub-messages are split, N is the preset message splitting factor, PMTU is the preset size of each read response message, and VA_i is the starting address of the i-th query message.

[0114] Each module in the aforementioned RDMA read message processing unit can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware within or independently of the processor in a computer device, or stored in software within the memory of the computer device, so that the processor can invoke and execute the operations corresponding to each module.

[0115] The following example illustrates this:

[0116] See Figure 4As shown, a local RDMA terminal is provided, including: a message splitting module, a first message sending processing module, and a first message receiving processing module, wherein the first message sending processing module is represented by RDMA TX1, and the first message receiving processing module is represented by RDMA RX1. A remote RDMA terminal is provided, including: a second message receiving processing module and a second message sending processing module, wherein the second message receiving processing module is represented by RDMA RX2, and the second message sending processing module is represented by RDMA TX2.

[0117] 1) The message splitting module is used to split an RDMA read message into multiple sub-messages. This example assumes a preset message splitting factor N=2 and a maximum read response window threshold of max_rd_rsp_pkt_wnd=4 per QP. When the message splitting module receives a new RDMA read message, assuming the length of this RDMA read message msg_len=8 PMTUs, it uses the formula:

[0118]

[0119] It can be determined that the RDMA read message needs to be split into 4 sub-messages, and each of the 4 sub-messages needs to generate a corresponding request message and send it to the remote RDMA end. Assuming that the number of read response messages to be received is rd_rsp_inflight_pkt_num=0 (i.e. there are no read response messages to be received and we need to wait for a return), the message splitting module decides to split the message into 2 sub-messages first according to max_rd_rsp_pkt_wnd / N=2.

[0120] 2) The message splitting module sends the two split sub-messages to RDMA TX1, and RDMA TX1 generates the corresponding request messages for each of these two sub-messages. Figure 4 (Request message 1 / 2 in the middle). RDMA TX1 sends two request messages to the remote RDMA end. At this time, rd_rsp_inflight_pkt_num is updated to 4, and rd_rsp_inflight_pkt_num is equal to max_rd_rsp_pkt_wnd.

[0121] 3) The remote RDMA end receives two request messages and sends them to RDMA RX2.

[0122] 4) RDMA RX2 processes the two request messages respectively, generates two independent response message descriptors, and sends them to the read response queue to wait for scheduling.

[0123] 5) The remote RDMA end schedules a certain read response descriptor according to the traffic management and congestion management mechanisms, and sends the read response descriptor to RDMA TX2. RDMA TX2 generates a corresponding read response packet based on the read response descriptor.

[0124] 6) According to two independent read response descriptors, RDMA TX2 returns a total of 4 read response packets (read response packets 1 / 2 / 3 / 4).

[0125] 7) After receiving the read response packet, RDMA RX1 in the local RDMA end starts to process it. After processing, RDMA RX1 writes the data carried by the read response packet into the local memory.

[0126] 8) Each time RDMA RX1 in the local RDMA end receives a read response packet, it decrements rd_rsp_inflight_pkt_num by 1. When rd_rsp_inflight_pkt_num < max_rd_rsp_pkt_wnd, RDMA RX1 notifies the message splitting module to restart splitting and sends the remaining two request packets (request packets 3 / 4) to the remote RDMA end.

[0127] In an exemplary embodiment, a network interface card is provided, which includes the RDMA read message processing unit in the foregoing embodiment.

[0128] In an exemplary embodiment, an RDMA communication system is provided, which includes the RDMA read message processing unit in the foregoing embodiment.

[0129] In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the RDMA read message processing method provided in the foregoing embodiment is implemented.

[0130] In an embodiment, a computer program product is provided, which includes a computer program. When the computer program is executed by a processor, the RDMA read message processing method provided in the foregoing embodiment is implemented.

[0131] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0132] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0133] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method of processing an RDMA Read message, the method comprising: The method applied to a local RDMA end comprises: determining the number n of request messages that can be currently sent according to the current number of read response messages to be received by the local RDMA end and the maximum read response message window threshold allowed by a single QP; selecting one or more RDMA read messages from the RDMA read messages to be processed in sequence according to the receiving sequence of the RDMA read messages and the starting position of the RDMA read messages to be processed, generating n request messages according to the selected one or more RDMA read messages, and sending the n request messages to a remote RDMA end; receiving read response messages returned by the remote RDMA end, processing the read response messages, and updating the current number of read response messages to be received.

2. The method of claim 1, wherein, The determination of the number n of request messages that can be currently sent according to the current number of read response messages to be received by the local RDMA end and the maximum read response message window threshold allowed by a single QP specifically comprises: The number n of request messages that can be currently sent is obtained by calculation using the following formula: n = (wnd-k) / N wherein wnd is the maximum read response message window threshold allowed by a single QP, and k is the current number of read response messages to be received. The current number of read response messages to be received is obtained by calculation using the following formula: wherein f is the number of request messages in all request messages sent by the local RDMA end that have not received corresponding read response messages, and N is a preset message splitting factor, used to represent the number of RDMA read response messages that can be generated by one RDMA request message.

3. The method of claim 1, wherein, The selection of one or more RDMA read messages from the RDMA read messages to be processed in sequence according to the receiving sequence of the RDMA read messages and the starting position of the RDMA read messages to be processed, the generation of n request messages according to the selected one or more RDMA read messages, and the sending of the n request messages to a remote RDMA end specifically comprise: For any current RDMA read message to be processed, if it is determined according to the starting position that the current RDMA read message to be processed is an undivided RDMA read message, then the number X1 of request messages that can be divided from the current RDMA read message to be processed according to a set rule is calculated; if X1 is greater than the number n of request messages that can be currently sent, then the current RDMA read message to be processed is divided according to a set rule to obtain a plurality of sub-messages, n sub-messages are obtained from the plurality of sub-messages, n request messages are generated according to the n sub-messages and sent to a remote RDMA end, and the number of request messages that can be currently sent is updated to 0. If the X1 is less than or equal to the number n of the request messages that can be currently sent, X1 request messages are generated according to the current to-be-processed RDMA read message and sent to the remote RDMA end, the number of the request messages that can be currently sent is updated as Y1, Y1=n-X1, and further, the number X2 of the request messages that the next to-be-processed RDMA read message can be split into according to the set rule is calculated, if the X2 is less than or equal to the Y1, X2 request messages are generated according to the next to-be-processed RDMA read message and sent to the remote RDMA end, the number of the request messages that can be currently sent is updated as Y2, Y2=Y1-X2, and further, the number X3 of the request messages that the next next to-be-processed RDMA read message can be split into is calculated, the relationship between the X3 and the Y2 is judged, and the corresponding request message is generated, until n request messages are generated, and the number of the request messages that can be currently sent is updated as 0; if the X2 is greater than the Y1, the next to-be-processed RDMA read message is split according to the set rule, Y1 sub-messages are selected from the sub-messages split from the next to-be-processed RDMA read message, Y1 request messages are generated according to the Y1 sub-messages and sent to the remote RDMA end, and the number of the request messages that can be currently sent is updated as 0.

4. The method of claim 1, wherein, The method further comprises: For any one of the current to-be-processed RDMA read messages, if the current to-be-processed RDMA read message belongs to the remaining to-be-processed sub-message part of the split RDMA read message according to the start position; The number U of the request messages that the remaining to-be-processed sub-message part can generate is calculated; If the U is greater than the number n of the request messages that can be currently sent, n sub-messages are selected from the remaining to-be-processed sub-message part, n request messages are generated according to the n sub-messages and sent to the remote RDMA end, and the number of the request messages that can be currently sent is updated as 0; If the U is less than or equal to the number n of the request messages that can be currently sent, U request messages are generated according to the remaining to-be-processed sub-message part and sent to the remote RDMA end, the number of the request messages that can be currently sent is updated as V1, V1=n-U, and further, the number U1 of the request messages that the next to-be-processed RDMA read message can be split into according to the set rule is calculated, if the U1 is greater than the V1, the next to-be-processed RDMA read message is split according to the set rule, V1 sub-messages are selected from the sub-messages split from the next to-be-processed RDMA read message, V1 request messages are generated according to the V1 sub-messages and sent to the remote RDMA end, and the number of the request messages that can be currently sent is updated as 0.

5. The method of claim 3, wherein, The calculating the number X1 of request packets that the current RDMA read message can be split into according to a set rule specifically includes: The number X1 of request packets is calculated by using the following formula: ； Where msg_len is the length of the current RDMA read message, N is a preset message splitting factor, and PMTU is a preset size of each read response packet.

6. The method of claim 3, wherein, The obtaining n sub-messages from the plurality of sub-messages and generating n request packets according to the n sub-messages specifically includes: n sub-messages are sequentially selected from the plurality of sub-messages to generate n request packets, and a start address of an i-th request packet is determined in the following manner: ； Where VA0 is a start address of the RDMA read message from which the plurality of sub-messages are split, N is a preset message splitting factor, PMTU is a preset size of each read response packet, and VA_i is the start address of the i-th request packet.

7. An RDMA Read message processing unit, comprising: The method comprises: A message splitting module is configured to determine the number n of request packets that can be currently sent according to a current number of read response packets to be received by a local RDMA end and a maximum read response packet window threshold allowed by a single QP; One or more RDMA read messages are sequentially selected from the RDMA read messages to be processed according to a receiving order of the RDMA read messages and a start position of the RDMA read message to be processed; A first packet sending processing module is configured to generate n request packets according to the selected one or more RDMA read messages and send the n request packets to a remote RDMA end; A first packet receiving processing module is configured to receive read response packets returned by the remote RDMA end, process the read response packets, and update the current number of read response packets to be received.

8. The RDMA Read message processing unit of claim 7, wherein, The message splitting module is specifically configured to: The number n of request packets that can be currently sent is calculated by using the following formula: n= (wnd-k) / N Where wnd is a maximum read response packet window threshold allowed by a single QP, and k is the current number of read response packets to be received; The current number of read response packets to be received is calculated by using the following formula: Where f is the number of request packets that have not received corresponding read response packets among all request packets sent by the local RDMA end, and N is a preset message splitting factor.

9. The RDMA Read message processing unit of claim 7, wherein, The first packet sending processing module is specifically configured to: For any current RDMA read message to be processed, if it is determined that the current RDMA read message to be processed is an undivided RDMA read message according to a start position, the number X1 of request packets that the current RDMA read message to be processed can be split into according to a set rule is calculated. If the X1 is greater than the current number of request messages that can be sent n, the current to-be-processed RDMA read message is split according to the set rule to obtain a plurality of sub-messages, n sub-messages are obtained from the plurality of sub-messages, n request messages are generated according to the n sub-messages and sent to the remote RDMA end, and the current number of request messages that can be sent is updated to 0; If the X1 is less than or equal to the current number of request messages that can be sent n, X1 request messages are generated according to the current to-be-processed RDMA read message and sent to the remote RDMA end, the current number of request messages that can be sent is updated to Y1, Y1=n-X1, and the number X2 of request messages that can be split from the next to-be-processed RDMA read message according to the set rule is further calculated, if the X2 is less than or equal to the Y1, X2 request messages are generated according to the next to-be-processed RDMA read message and sent to the remote RDMA end, the current number of request messages that can be sent is updated to Y2, Y2=Y1-X2, and the number X3 of request messages that can be split from the next to-be-processed RDMA read message is further calculated, the relationship between the X3 and the Y2 is judged, and the corresponding request message is generated, until n request messages are generated, and the current number of request messages that can be sent is updated to 0; If the X2 is greater than the Y1, the next to-be-processed RDMA read message is split according to the set rule, Y1 sub-messages are selected from the sub-messages split from the next to-be-processed RDMA read message, Y1 request messages are generated according to the Y1 sub-messages and sent to the remote RDMA end, and the current number of request messages that can be sent is updated to 0.

10. The RDMA Read message processing unit of claim 7, wherein, The first message sending processing module is specifically used for: For any current to-be-processed RDMA read message, if the current to-be-processed RDMA read message belongs to the remaining unprocessed sub-message part of the split RDMA read message according to the start position; The number U of request messages that can be generated from the remaining unprocessed sub-message part is calculated; If the U is greater than the current number of request messages that can be sent n, n sub-messages are selected from the remaining unprocessed sub-message part, n request messages are generated according to the n sub-messages and sent to the remote RDMA end, and the current number of request messages that can be sent is updated to 0; If the U is less than or equal to the number n of the current request messages that can be sent, U request messages are generated according to the remaining unprocessed sub-message parts and sent to the remote RDMA end, the number of the current request messages that can be sent is updated as V1, V1 = n - U, and the number U1 of request messages that the next to-be-processed RDMA read message can be split into according to the set rule is further calculated. If the U1 is greater than the V1, the next to-be-processed RDMA read message is split according to the set rule, V1 sub-messages are selected from the sub-messages split from the next to-be-processed RDMA read message, V1 request messages are generated according to the V1 sub-messages and sent to the remote RDMA end, and the number of the current request messages that can be sent is updated as 0.

11. The RDMA Read message processing unit of claim 9 or 10, wherein, The first message sending processing module is specifically configured to: X1 is calculated by using the following formula: ； Wherein, msg_len is the length of the current to-be-processed RDMA read message, N is a preset message splitting factor, and PMTU is a preset size of each read response message.

12. The RDMA Read message processing unit of claim 9, wherein, The first message sending processing module is specifically configured to: n sub-messages are sequentially selected from the plurality of sub-messages to generate n request messages, wherein a start address of an i-th request message is determined in the following manner: ； Wherein, VA0 is a start address of the RDMA read message split to obtain the plurality of sub-messages, N is a preset message splitting factor, PMTU is a preset size of each read response message, and VA_i is the start address of the i-th request message.

13. A network interface card, comprising: The RDMA read message processing unit as claimed in any one of claims 7-12.

14. An RDMA communication system, characterized by The RDMA read message processing unit as claimed in any one of claims 7-12.

15. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1-6.

16. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1-6.

Images