Rdma read request processing method and apparatus, chip, network interface card, device, medium and program product

Abstract

The application relates to an RDMA read request processing method and device, a chip, a network interface card, equipment, a medium and a program product. The method comprises the following steps: receiving and analyzing response information sent by a request response end to obtain coding information of an online QP quantity; obtaining a quantity threshold of an unfinished operation request sent by a request sending end according to the coding information of the online QP quantity, wherein the coding information is used for representing a quantity range of online QPs between the request sending end and the request response end; and controlling the sending of a current RDMA read request to the response end according to the number of RDMA read requests sent by the request sending end but not received by the response end and the quantity threshold. The method can improve the processing performance of READ read requests.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to an RDMA read request processing method, apparatus, chip, network interface card, device, medium, and program product. Background Technology

[0002] During connection establishment, RDMA (Remote Direct Memory Access) negotiates the receiving capacity of RDMA READ requests between the two communicating QPs (Queue Pairs) to prevent the receiving end from discarding RDMA READ requests that cannot be received by the sending end. Typically, the maximum RDMA READ receiving capacity is 128, meaning that a single sending QP can send a maximum of 128 incomplete RDMA READ requests. A single RDMA READ request can read up to 2GB of data.

[0003] Specifically, in combination Figure 1 As shown, Figure 1 This is a flowchart illustrating the RDMA READ processing flow between the data sending end and the request / response end in traditional technology, specifically including:

[0004] (1) The RDMA engine at the request and response end receives the RDMA READ request message from the network and sends it to the RXP (Receive Process, the receive processing module of the RDMA engine) for processing.

[0005] (2) The RXP reads the QPC (Queue Pair Context, QP context) based on the qpn information carried in the RDMA READ request message.

[0006] (3) RXP caches the RDMA READ request information in the cache.

[0007] (4) Since the RDMA READ request may not return READ RESPONSE immediately (for example, it may not be sent due to network congestion), an RDMA READ doorbell is generated and sent to SCH (Schedule, the scheduling processing module of the RDMA engine) to participate in scheduling.

[0008] (5) SCH schedules RDMA READ requests.

[0009] (6) SCH sends the RDMA READ doorbell obtained from the scheduling to TXP (Transmit Process, the transmit processing module of the RDMA engine) for processing.

[0010] (7) The RDMA engine reads the RDMA READ request information from the cache based on the RDMA READ doorbell information.

[0011] (8) The RDMA engine sends a READ RESPONSE message based on the RDMA READ request information.

[0012] In traditional technology, the receiving capacity of the receiving end of a single QP for RDMA READ requests between the two communicating parties is negotiated during the establishment of the connection and will not change during the service process. When the receiving end receives an RDMA READ request, it caches the request information in the cache.

[0013] However, when deploying RDMA on a large scale, for example, if a single QP can receive 128 RDMA READ requests, and a large-scale deployment involves 8K QPs, then the RDMA engine needs to cache a maximum of 128 * 8K = 1M RDMA READ requests. RDMA engines typically do not provide such a large cache.

[0014] Since the number of QPs is very large, if the sending end does not limit the ability of a single QP to send RDMA READ requests, it is easy for the cache to be unable to hold all these RDMA READ request information, causing a large number of cache replacements, thereby affecting performance. Summary of the Invention

[0015] Therefore, it is necessary to provide an RDMA read request processing method, apparatus, chip, network interface card, device, medium, and program product that can improve the processing performance of operation requests, addressing the aforementioned technical problems.

[0016] Firstly, this application provides an RDMA read request processing method, applied at the request sending end, the method comprising:

[0017] Receive and parse the response information sent by the requesting and responding end to obtain the encoded information of the number of online QPs;

[0018] Based on the encoding information of the number of online QPs, the threshold number of incomplete operation requests sent by the request sending end is obtained, and the encoding information is used to represent the range of the number of online QPs between the request sending end and the response end.

[0019] The current RDMA read request is sent to the responding end based on the number of RDMA read requests that the requesting end has sent but has not received a response to, and the quantity threshold.

[0020] In one embodiment, controlling the sending of the current RDMA read request to the request responding end based on the number of RDMA read requests that the request sending end has sent but has not received a response to and the number threshold includes:

[0021] If the number of RDMA read requests that have been sent but for which no response has been received is less than or equal to the number threshold, the current RDMA read request is sent.

[0022] If the number of RDMA read requests that the requesting end has sent but has not received a response to is greater than the number threshold, the process continues to obtain the number of requests that the requesting end has sent but has not received a response to.

[0023] In one embodiment, sending the current RDMA read request when the number of requests that the request sending end has sent but not received a response to is less than or equal to the quantity threshold includes:

[0024] Obtain the WQE information for sending the current RDMA read request;

[0025] Obtain the corresponding first QPC information from the QP's transmit QPC, wherein the QP's transmit QPC does not include second QPC information related to the receive QPC;

[0026] Based on the WQE information sent for the current RDMA read request and the first QPC information obtained, generate one or at least two sub-RDMA read requests corresponding to the current RDMA read request;

[0027] Send each of the sub-RDMA read requests.

[0028] In one embodiment, generating one or at least two sub-RDMA read requests corresponding to the current RDMA read request based on the WQE information of the current RDMA read request and the acquired QPC information includes:

[0029] The current RDMA read request is split according to the size of the maximum network packet to obtain sub-RDMA read requests. The size of the packet corresponding to each sub-RDMA read request is equal to the size of the maximum network packet.

[0030] Secondly, this application also provides an RDMA read request processing method, applied to a request-response end, the method comprising:

[0031] Receive the RDMA read request sent by the request sender and cache the received RDMA read request;

[0032] Schedule each of the cached RDMA read requests to determine the target RDMA read request for the current response to be sent;

[0033] Obtain the number of online QPs and generate encoding information based on the number of online QPs;

[0034] Based on the encoded information, a response information corresponding to the target RDMA read request is generated, and the response information is sent to the request sending end.

[0035] In one embodiment, reading the corresponding response data includes:

[0036] Obtain the second QPC information corresponding to the received RDMA read request from the received QPC of the QP, wherein the received QPC of the QP does not include the first QPC information related to the transmitted QPC;

[0037] The RDMA read request is cached based on the obtained second QPC information.

[0038] Thirdly, this application also provides an RDMA read request processing apparatus, applied at a request sending end, the apparatus comprising:

[0039] The first receiving module is used to receive and parse the response information sent by the request and response end to obtain the encoded information of the number of online QPs;

[0040] The threshold quantity acquisition module is used to obtain the threshold quantity of incomplete operation requests sent by the request sending end based on the encoding information of the online QP quantity, wherein the encoding information is used to represent the range of online QP quantity between the request sending end and the response end;

[0041] The first sending module is used to control the sending of the current RDMA read request to the responding end based on the number of RDMA read requests that the requesting end has sent but has not received a response to and the number threshold.

[0042] Fourthly, this application also provides an RDMA read request apparatus, applied at a request-response end, the apparatus comprising:

[0043] The second receiving module is used to receive the RDMA read request sent by the request sending end and read the corresponding response data;

[0044] The scheduling module is used to schedule each of the cached RDMA read requests to determine the target RDMA read request for the current response to be sent.

[0045] The encoding information generation module is used to obtain the number of online QPs and generate encoding information based on the number of online QPs;

[0046] The second sending module is used to generate response information corresponding to the target RDMA read request based on the encoding information and the response data, and send the response information to the request sending end.

[0047] Fifthly, this application also provides a chip including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method in any of the above embodiments.

[0048] Sixthly, this application also provides a network interface card, including the chip and multiple interfaces as described in any of the above embodiments.

[0049] The chip processes data or communicates with the outside world through the interface.

[0050] Seventhly, this application also provides a computer device including a network interface card in any of the above embodiments, the network interface card being used for processing data or external communication.

[0051] Eighthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the methods in any of the above embodiments.

[0052] Ninthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method in any of the above embodiments.

[0053] The aforementioned RDMA read request processing method, apparatus, chip, network interface card, device, medium, and program products receive and parse the response information sent by the request-response end to obtain encoded information on the number of online QPs; based on the encoded information on the number of online QPs, they obtain a threshold for the number of incomplete operation requests sent by the request sender, where the encoded information represents the range of the number of online QPs between the request sender and the response end; and based on the number of RDMA read requests that the request sender has sent but has not received a response to, and the threshold, they control the sending of the current RDMA read request to the response end, thereby reducing the number of operation requests sent by the request sender. This avoids cache replacement caused by insufficient cache at the request receiver, which would affect performance. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a flowchart illustrating the RDMA READ process between the data sending end and the request / response end in traditional technology.

[0056] Figure 2 This is an application environment diagram of an operation request processing method in RDMA in one embodiment;

[0057] Figure 3 This is a flowchart illustrating an operation request processing method in RDMA in one embodiment;

[0058] Figure 4 This is a schematic diagram of a QPC in one embodiment;

[0059] Figure 5 This is a flowchart illustrating the operation request processing method in RDMA in another embodiment;

[0060] Figure 6 This is a block diagram of an operation request processing device in an RDMA embodiment;

[0061] Figure 7 This is a block diagram of an operation request processing device in RDMA in another embodiment;

[0062] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0064] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0065] The operation request processing method in RDMA provided in this application embodiment can be applied to, for example... Figure 2In the application environment shown, the request sender 100 and the request responder 200 communicate via RDMA. The request sender 100 sends an RDMA read request to the request responder 200, and the request responder 200 processes the RDMA read request to return the corresponding response information.

[0066] The request-response end 200 can send encoded information about the number of online QPs to the request-sender end 100. Based on the encoded information about the number of online QPs, the request-sender end 100 obtains a threshold for the number of incomplete operation requests sent by the request-sender end. The encoded information represents the range of the number of online QPs between the request-sender end and the response end. Based on the number of RDMA read requests that the request-sender end has sent but has not received a response to and the threshold, the current RDMA read request is controlled to be sent to the response end. This can reduce the number of operation requests sent by the request-sender end 100, thereby avoiding cache replacement caused by insufficient cache at the request-response end, which would affect performance.

[0067] In one exemplary embodiment, such as Figure 3 As shown, an RDMA read request processing method is provided, which is applied to... Figure 2 Taking the request sending end as an example, the explanation includes the following steps S302 to S306. Wherein:

[0068] S302: Receive and parse the response information sent by the request and response end to obtain the encoded information of the number of online QPs.

[0069] The number of online QPs refers to the number of QPs with unprocessed operation requests in the request-response end. This number of online QPs can be obtained based on the number of doorbells in the scheduling processing module of the RDMA engine of the request-response end. It should be noted that a request-response end can communicate with multiple request senders simultaneously. In this case, the number of online QPs in the request-response end corresponds to multiple request senders, while the encoding information is specific to each QP in each request sender. Figure 1 Since the RDMA read request cannot immediately return a response message, a doorbell is generated to facilitate the scheduling of the receiving end RDMA engine's scheduling processing module. In order to ensure that each online QP is processed evenly, the receiving end RDMA engine's scheduling processing module will poll and process the RDMA read requests in each online QP. Thus, by counting the number of doorbells for operation requests in the receiving end RDMA engine's scheduling processing module, the number of online QPs can be obtained.

[0070] The encoding information is related to the number of online QPs. A mapping relationship between encoding information and the number of online QPs can be pre-set. For example, different ranges of online QPs correspond to different encoding information. Optionally, the more online QPs, the larger the encoding information. In some optional embodiments, the encoding information is 7 when the number of online QPs is greater than 4k; 6 when the number of online QPs is in the range of 2k+1 to 4k; 5 when the number of online QPs is in the range of 1k+1 to 2k; 4 when the number of online QPs is in the range of 512+1 to 1k; 3 when the number of online QPs is in the range of 256+1 to 512; 2 when the number of online QPs is in the range of 128+1 to 256; 1 when the number of online QPs is in the range of 64+1 to 128; and 0 when the number of online QPs is less than or equal to 64, which is the default encoding information. Under the default encoding information, the threshold for the number of incomplete operation requests sent by the request sender is the largest. In other embodiments, the encoding information may also be set to other values.

[0071] The encoded information is transmitted to the requesting end through ack / response messages, as shown in Table 1, where BTH.resv6 defined by the IB protocol is used to transmit the encoded information.

[0072] Table 1:

[0073]

[0074] S304: Based on the encoded information of the number of online QPs, obtain the threshold for the number of incomplete operation requests sent by the requesting end. The encoded information represents the range of the number of online QPs between the requesting end and the request-responding end.

[0075] The threshold for the number of incomplete operation requests sent by the requesting sender is used to limit the number of RDMA read requests sent by the requesting sender. The number of incomplete operation requests sent by the requesting sender must be less than this threshold. Specifically, if the requesting sender has only one QP, then this threshold is the threshold for the number of incomplete RDMA read requests sent corresponding to that one QP. If the requesting sender has multiple QPs, then this threshold is the threshold for the number of incomplete RDMA read requests sent corresponding to all multiple QPs; that is, the number of incomplete RDMA read requests sent corresponding to each QP must be less than this threshold.

[0076] The encoded information represents the range of online QPs between the request sender and the request responder. The mapping relationship between the encoded information and this threshold can be preset. In this mapping relationship, the more online QPs the encoded information represents at the request responder, the smaller the threshold. That is, the larger the encoded information, the smaller the threshold.

[0077] In some optional embodiments, the relationship between the number of online QPs, encoding information, and the number threshold can be referred to in Table 2:

[0078] Table 2:

[0079]

[0080] An incomplete RDMA read request can be determined based on whether the requesting end receives the corresponding response message. If a complete response message is received, the RDMA read request is complete; if a complete response message is not received, the RDMA read request is incomplete.

[0081] S306: Based on the number of RDMA read requests that the requesting end has sent but has not received a response to, and the quantity threshold, control the sending of the current RDMA read request to the responding end.

[0082] The requesting end can control the sending of RDMA read requests to the requesting response end based on the number of RDMA read requests that have been sent but for which no response has been received, and a threshold value. For example, if the number of incomplete RDMA read requests sent by the requesting end exceeds the threshold value, no further RDMA read requests can be sent to the requesting response end. In other embodiments, the speed at which RDMA read requests are sent to the requesting response end can also be controlled based on the relationship between the number of incomplete RDMA read requests sent by the requesting end and the threshold value. For example, if the number of incomplete RDMA read requests sent by the requesting end is less than or equal to the threshold value, the difference between the number of incomplete RDMA read requests sent by the requesting end and the threshold value can be calculated, and the range of the difference value can be determined. Each range corresponds to a different sending speed, and the larger the range, the faster the sending speed, etc., to avoid a large number of unprocessed RDMA read requests at the requesting response end, which would affect performance.

[0083] The aforementioned RDMA read request processing method receives and parses the response information sent by the requesting and responding end to obtain the encoded information of the number of online QPs. Based on the encoded information of the number of online QPs, it obtains the threshold for the number of incomplete operation requests sent by the requesting end. The encoded information is used to represent the range of the number of online QPs between the requesting end and the responding end. Based on the number of RDMA read requests that the requesting end has sent but has not received a response (i.e., the number of outstanding requests) and the threshold, it controls the sending of the current RDMA read request to the responding end, thereby reducing the number of operation requests sent by the requesting end. This avoids the requesting and responding end from replacing the cache due to insufficient cache, which would affect performance.

[0084] In some optional embodiments, controlling the sending of the current RDMA read request to the responding end based on the number of RDMA read requests that the requesting end has sent but has not received a response to and a quantity threshold includes: sending the current RDMA read request when the number of RDMA read requests that the requesting end has sent but has not received a response to is less than or equal to the quantity threshold; and continuing to obtain the number of requests that the requesting end has sent but has not received a response to when the number of RDMA read requests that the requesting end has sent but has not received a response to is greater than the quantity threshold.

[0085] If the number of RDMA read requests that the requesting end has sent but has not received a response to is less than or equal to the quantity threshold, that is, the requesting end is not congested, so the current RDMA read request can be sent. Otherwise, it means that the requesting end is congested, so the number of requests that the requesting end has sent but has not received a response to is obtained, and the judgment is made again.

[0086] Optionally, after receiving and parsing the response information sent by the request-response end to obtain the encoded information of the number of online QPs, the process also includes: pausing the sending operation request. This is because the request-response end can be configured to send encoded information to the request-sender end only when congestion is detected. For example, if the encoded information determined based on the number of online QPs changes, it indicates that congestion exists at the request-response end, and the request-response end will send new encoded information to the request-sender end. This allows the request-sender end to control the sending of RDMA read requests based on the encoded information. When the request-response end starts receiving data, the number of online QPs is small, so the encoded information is 0. When the request-sender end initializes, the quantity threshold is at its maximum, which is 128 according to Table 2. As RDMA read requests are sent, the number of online QPs at the request-response end increases, and therefore the encoded information also changes. Thus, after the request-response end sends the encoded information to the request-sender end, the request-sender end needs to pause the sending of RDMA read requests to adjust the quantity threshold in a timely manner and limit the number of incomplete RDMA read requests sent by the request-sender end.

[0087] The requesting end determines the relationship between the number of incomplete RDMA read requests sent by the requesting end and the number threshold. At this time, since the requesting response end is already congested, the number of RDMA read requests is generally greater than the number threshold, that is, the sending operation request is suspended. Subsequently, as the requesting response end processes the RDMA read requests, the requesting end receives more and more response messages, thereby reducing the number of RDMA read requests in the requesting end. This makes the current number of RDMA read requests less than or equal to the number threshold, so that the requesting end can be asked to restart the sending operation request.

[0088] To make it easier to understand, let's take the example of the RDMA engine providing 8K caches for RDMA read requests. When the sum of the number of REMA READ requests from all online QPs is no greater than 8K, it can avoid cache replacement due to insufficient cache, which would affect performance. Therefore, when the sum of the number of REMA READ requests from online QPs is greater than 8K, RDMA read requests will stop being sent.

[0089] Furthermore, it's important to note that the update of the quantity threshold and the sending of RDMA read requests occur in parallel. The quantity threshold is updated promptly based on the encoded information. The requesting end obtains the latest quantity threshold to determine whether to send an RDMA read request. For example, if the quantity threshold is updated after an encoding information update, the requesting end may not send an RDMA read request. However, if the encoding information is subsequently updated again, increasing the quantity threshold, and the current quantity of the RDMA read request is less than or equal to the quantity threshold, then the RDMA read request is resent. Alternatively, if the encoding information is not updated, but the current quantity of the RDMA read request is still less than or equal to the quantity threshold, then the RDMA read request also needs to be resent. In other words, the current quantity of the RDMA read request being less than or equal to the quantity threshold could be due to a change in the quantity threshold (e.g., an increase) or a decrease; no specific limitation is made here.

[0090] In the above embodiments, after receiving the encoded information, the sending of RDMA read requests is paused. Only when the current number of RDMA read requests is less than or equal to the number threshold will the RDMA read requests be resent, thus reducing the processing pressure on the request response end.

[0091] In one specific embodiment, when the number of requests sent by the requesting end that have not received a response is less than or equal to a quantity threshold, the present application sends a current RDMA read request, including: obtaining the transmission WQE information of the current RDMA read request; obtaining the corresponding first QPC information from the transmission QPC of the QP, wherein the transmission QPC of the QP does not include the second QPC information related to the reception QPC; generating one or at least two sub-RDMA read requests corresponding to the current RDMA read request based on the transmission WQE information of the current RDMA read request and the obtained first QPC information; and sending each sub-RDMA read request.

[0092] In many RDMA applications, a QP is often used only for sending requests or only for receiving requests. This application splits the QP context QPC into the context of the sending QP that is only used for sending requests, namely the first QPC information TX_QPC, and the context of the receiving QP that is only used for receiving requests, namely the second QPC information RX_QPC.

[0093] Taking sending an RDMA read request as an example, the request sender only includes the first QPC information related to the transmit QPC, but not the second QPC information related to the receive QPC. Therefore, it only needs to obtain the corresponding first QPC information from the transmit QPC, and generate one or more sub-RDMA read requests corresponding to the current RDMA read request based on the transmit WQE information corresponding to the current RDMA read request and the first QPC information. Finally, the sub-RDMA read request is sent. Figure 4 As shown, a QPC for a QP used to send and receive requests totals 256 bytes. However, in many business scenarios, a QP only needs to be used to send or receive requests, and does not need to support both simultaneously. Therefore, this application splits the QPC into a first QPC information TX_QPC for sending the QP (only 128 bytes, used only for sending requests) and a second QPC information RX_QPC for receiving the QP (only 128 bytes, used only for receiving requests). During business processing, only the first QPC information TX_QPC or the second QPC information RX_QPC needs to be cached. When the cache size provided by the RDMA engine is fixed, the number of QPCs for cached QPs can be nearly doubled. Figure 2 As shown, the request sending end only needs to cache the context of sending QP, i.e., the first QPC information TX_QPC, and the request response end only needs to cache the context of receiving QP, i.e., the second QPC information RX_QPC.

[0094] In the above embodiments, the QPC is split into a first QPC information TX_QPC used only for sending requests and a second QPC information RX_QPC used only for receiving requests. In this way, when sending, the corresponding first QPC information can be obtained only from the sending QPC of the QP.

[0095] In one specific embodiment, the method of generating one or at least two sub-RDMA read requests corresponding to the current RDMA read request based on the WQE information of the current RDMA read request and the obtained QPC information includes: splitting the current RDMA read request according to the size of the maximum network packet to obtain each sub-RDMA read request, wherein the size of the packet corresponding to each sub-RDMA read request is equal to the size of the maximum network packet.

[0096] One RDMA read request can read 2GB of data, which slows down the processing speed of the RDMA engine on the request response side. To improve the processing speed, the RDMA read request can be split into sub-RDMA read requests. Since the sub-RDMA read request reads less data, the processing speed of a sub-RDMA read request is faster, thereby reducing the accumulated RDMA read requests on the request response side and improving the processing speed.

[0097] In this application, the speed at which the RDMA engine processes a single RDMA read request is usually inversely proportional to the amount of data read by the RDMA read request. Taking a unidirectional 400Gbps network as an example, it takes 42949us for the request-response RDMA engine to complete a 2GB RDMA read request, while it only takes 82ns to complete a 4KB RDMA read request.

[0098] When the request sending end RDMA engine detects a large number of online QPs on the request response end, it splits the RDMA read requests when it can be resent. For example, it splits them according to PMTU granularity (maximum network packet size). For instance, an 8KB RDMA read request is split into two 4KB sub-RDMA read requests. This doubles the speed at which the request response end processes RDMA read requests, thereby reducing the accumulation of RDMA read requests on the request response end and reducing the occurrence of cache replacement due to insufficient cache.

[0099] In one exemplary embodiment, such as Figure 5 As shown, an RDMA read request processing method is provided, which is applied to... Figure 2 Taking the request and response end as an example, the explanation includes the following steps S502 to S508. Wherein:

[0100] S502: Receive the RDMA read request sent by the request sender and read the corresponding response data.

[0101] When the request-response end receives an RDMA read request from the network, the RDMA engine's receive processing module on the request-response end processes the RDMA read request. Since the response information cannot be returned immediately, the RDMA read request is cached first.

[0102] S504: Schedule each cached RDMA read request to determine the target RDMA read request for the current response to be sent.

[0103] The scheduling module of the RDMA engine at the subsequent request response end can schedule the RDMA read request to determine the target RDMA read request for the current response to be sent. Specifically, the receiving processing module of the RDMA engine periodically generates a doorbell for each QP, and the scheduling processing module of the RDMA engine schedules based on the doorbell to determine the doorbell corresponding to the target RDMA read request for the current response to be sent, and sends the scheduled doorbell to the sending processing module of the RDMA engine. Subsequently, the sending processing module of the RDMA engine determines the target RDMA read request for the current response to be sent in the buffer based on the received doorbell.

[0104] S506: Obtain the number of online QPs and generate encoding information based on the number of online QPs.

[0105] The number of online QPs is the number of QPs with unprocessed RDMA read requests in the request response end. In some optional embodiments, obtaining the number of online QPs includes: obtaining the number of doorbells of RDMA read requests in the scheduling processing module of the receiver's RDMA engine.

[0106] Combination Figure 1 Since the RDMA read request cannot immediately return a response message, the number of doorbells that generate RDMA read requests is used to facilitate the scheduling of the receiving end RDMA engine's scheduling processing module. In order to ensure that each online QP is processed evenly, the receiving end RDMA engine's scheduling processing module will poll and process the operation requests in each online QP. Therefore, by counting the number of doorbells for operation requests in the receiving end RDMA engine's scheduling processing module, the number of online QPs can be obtained.

[0107] The encoding information is related to the number of online QPs. A mapping relationship between encoding information and the number of online QPs can be pre-set. For example, different ranges of online QPs correspond to different encoding information. Optionally, the more online QPs, the larger the encoding information. In some optional embodiments, the encoding information is 7 when the number of online QPs is greater than 4k; 6 when the number of online QPs is in the range of 2k+1 to 4k; 5 when the number of online QPs is in the range of 1k+1 to 2k; 4 when the number of online QPs is in the range of 512+1 to 1k; 3 when the number of online QPs is in the range of 256+1 to 512; 2 when the number of online QPs is in the range of 128+1 to 256; 1 when the number of online QPs is in the range of 64+1 to 128; and 0 when the number of online QPs is less than or equal to 64, which is the default encoding information. Under the default encoding information, the threshold for the number of incomplete RDMA read requests sent by the requesting sender is the largest. In other embodiments, the encoding information may also be set to other values.

[0108] S508: Generate response information corresponding to the target RDMA read request based on the encoding information and the response data, and send the response information to the request sending end.

[0109] The encoded information is transmitted to the requesting end through ack / response messages, such as BTH.resv6 defined by the IB protocol in Table 1 above.

[0110] The system allows the request-response end to send encoded information to the request-sender end only when congestion is detected. For example, if the encoded information, determined based on the number of online QPs, changes, the request-response end will send new encoded information to the request-sender end. This allows the request-sender end to control the sending of RDMA read requests based on the encoded information. When the request-response end first receives RDMA read requests, the number of online QPs is low, so the encoded information is 0. At the initialization of the request-sender end, the quantity threshold is at its maximum, which is 128 according to Table 2. As RDMA read requests are sent, the number of online QPs on the request-response end increases, and the encoded information also changes. Therefore, after the request-response end sends the encoded information to the request-sender end, the request-sender end needs to pause sending RDMA read requests to adjust the quantity threshold accordingly.

[0111] The threshold for the number of incomplete operation requests sent by the requesting end is used to limit the number of RDMA read requests sent by the requesting end. The number of incomplete RDMA read requests sent by the requesting end is less than this threshold.

[0112] The mapping relationship between the encoded information and the quantity threshold can be preset. In this mapping relationship, the more online QPs there are on the request and response end, the smaller the quantity threshold will be. That is, the larger the encoded information is, the smaller the quantity threshold will be.

[0113] An incomplete RDMA read request can be determined based on whether the requesting end receives the corresponding response message. If a complete response message is received, the RDMA read request is complete; if a complete response message is not received, the RDMA read request is incomplete.

[0114] The request sender can control the number of operation requests sent to the request response end based on a threshold. For example, if the number of incomplete operation requests sent by the request sender exceeds the threshold, no further operation requests can be sent to the request response end. In other embodiments, the speed at which operation requests are sent to the request response end can also be controlled based on the relationship between the number of incomplete RDMA read requests sent by the request sender and the threshold. For example, if the number of incomplete RDMA read requests sent by the request sender is less than or equal to the threshold, the difference between the number of incomplete RDMA read requests sent by the request sender and the threshold can be calculated, and the range of this difference can be determined. Each range corresponds to a different sending speed, and the larger the range, the faster the sending speed, thus avoiding a large number of unprocessed RDMA read requests on the request response end, which would affect performance.

[0115] In the above embodiments, the encoding information indicates that the more online QPs there are on the request-response end, the smaller the threshold for the number of incomplete RDMA read requests sent by the request-sender end. By controlling the number of RDMA read requests sent to the request-response end based on the threshold, the number of RDMA read requests sent by the request-sender end can be reduced. This can prevent the request-response end from replacing the cache due to insufficient cache, thus affecting performance.

[0116] In some optional embodiments, reading the corresponding response data includes: obtaining second QPC information corresponding to the received RDMA read request from the QP's receive QPC, wherein the QP's receive QPC does not include first QPC information related to the send QPC; reading the corresponding response data according to the read request and the second QPC information, and caching the response data.

[0117] Among them, the combination Figure 4 As shown, in this application, QPC is split into the first QPC information TX_QPC (only 128B, used only for sending requests) for sending QP and the second QPC information RX_QPC (only 128B, used only for receiving requests) for receiving QP. During the business processing, only the first QPC information TX_QPC or the second QPC information RX_QPC needs to be cached. When the cache size provided by RDMAengine is fixed, the number of QPCs for cached QP can be increased by nearly double.

[0118] Taking receiving an RDMA read request as an example, the RDMA engine on the request-responding end only needs to access the context second QPC information RX_QPC of the receiving QP. The size of these QPCs is typically only slightly more than half the total number of QPCs. When deploying RDMA on a large scale and using RDMA READ, with a fixed cache size provided by the RDMA engine, the number of online QPs can be nearly doubled because the size of the QPC used by a single QP is reduced (only RX_QPC is used). Thus, when the RDMA engine processes an RDMA read request, it only needs to maintain the second QPC information RX_QPC. With a fixed cache size provided by the RDMA engine, the number of online QPs can be nearly doubled because the size of the QPC used by a single QP is reduced (only RX_QPC is used).

[0119] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, even a small portion of the steps in the flowcharts of the above embodiments may include multiple steps or stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or a small portion of steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations fall within the scope of protection of this application.

[0120] Based on the same inventive concept, this application also provides an RDMA read request processing apparatus for implementing the RDMA read request processing method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more RDMA read request processing apparatus embodiments provided below can be found in the limitations of the RDMA read request processing method described above, and will not be repeated here.

[0121] In one exemplary embodiment, such as Figure 6 As shown, an RDMA read request processing apparatus is provided, comprising: a first receiving module 601, a threshold quantity acquisition module 602, and a first sending module 603, wherein:

[0122] The first receiving module 601 is used to receive and parse the response information sent by the request-response end to obtain the encoded information of the number of online QPs;

[0123] The threshold quantity acquisition module 602 is used to obtain the threshold quantity of incomplete operation requests sent by the request sending end based on the encoding information of the online QP quantity. The encoding information is used to represent the range of the online QP quantity between the request sending end and the response end.

[0124] The first sending module 603 is used to control the sending of the current RDMA read request to the responding end based on the number of RDMA read requests that the requesting end has sent but has not received a response to and a quantity threshold.

[0125] In some optional embodiments, the first sending module 603 is specifically used to send the current RDMA read request when the number of RDMA read requests that the requesting sender has sent but has not received a response to is less than or equal to a quantity threshold; and to continue to obtain the number of requests that the requesting sender has sent but has not received a response to when the number of RDMA read requests that the requesting sender has sent but has not received a response to is greater than the quantity threshold.

[0126] In some optional embodiments, the first sending module 603 is further configured to obtain the sending WQE information of the current RDMA read request; obtain the corresponding first QPC information from the sending QPC of the QP, wherein the sending QPC of the QP does not include the second QPC information related to the receiving QPC; generate one or at least two sub-RDMA read requests corresponding to the current RDMA read request based on the sending WQE information of the current RDMA read request and the obtained first QPC information; and send each sub-RDMA read request.

[0127] In some optional embodiments, the first sending module 603 is further configured to split the current RDMA read request according to the size of the maximum network packet to obtain sub-RDMA read requests, wherein the size of the packet corresponding to each sub-RDMA read request is equal to the size of the maximum network packet.

[0128] In one exemplary embodiment, such as Figure 7 As shown, an RDMA read request processing apparatus is provided, comprising: a second receiving module 701, a scheduling module 702, an encoded information generation module 703, and a second sending module 704, wherein:

[0129] The second receiving module 701 is used to receive the RDMA read request sent by the request sending end and read the corresponding response data;

[0130] The scheduling module 702 is used to schedule each buffered RDMA read request in order to determine the target RDMA read request of the current response to be sent.

[0131] The encoding information generation module 703 is used to obtain the number of online QPs and generate encoding information based on the number of online QPs;

[0132] The second sending module 704 is used to generate response information corresponding to the target RDMA read request based on the encoding information and the response data, and send the response information to the request sending end.

[0133] In some optional embodiments, the second receiving module 701 is further configured to obtain second QPC information corresponding to the received RDMA read request from the received QPC of the QP, wherein the received QPC of the QP does not include first QPC information related to the transmitted QPC; read the corresponding response data according to the read request and the second QPC information, and cache the response data.

[0134] Each module in the aforementioned RDMA read request processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware within or independently of the processor in a computer device, or stored in software within the memory of the computer device, so that the processor can invoke and execute the operations corresponding to each module.

[0135] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements an operation request processing method in RDMA. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0136] Those skilled in the art will understand that Figure 8The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or smaller components than those shown in the figure, or combine certain components, or have different component arrangements.

[0137] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0138] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0139] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0140] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one of relational databases and non-relational databases. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0141] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0142] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for processing RDMA read requests, characterized in that, Applied to the request sending end, the method includes: Receive and parse the response information sent by the request and response end to obtain the encoded information of the number of online QPs with unprocessed operation requests in the request and response end; Based on the encoding information of the number of online QPs, a threshold for the number of incomplete operation requests sent by the request sending end is obtained, wherein the encoding information is used to represent the range of the number of online QPs between the request sending end and the request response end; The current RDMA read request is sent to the responding end based on the number of RDMA read requests that the requesting end has sent but has not received a response to, and the quantity threshold.

2. The method according to claim 1, characterized in that, The step of controlling the sending of the current RDMA read request to the responding end based on the number of RDMA read requests that the requesting end has sent but has not received a response to and the number threshold includes: If the number of RDMA read requests that have been sent but for which no response has been received is less than or equal to the number threshold, the current RDMA read request is sent. If the number of RDMA read requests that the requesting end has sent but has not received a response to is greater than the number threshold, the operation of obtaining the number of requests that the requesting end has sent but has not received a response to continue.

3. The method according to claim 2, characterized in that, When the number of RDMA read requests that have been sent but for which no response has been received at the request sending end is less than or equal to the quantity threshold, sending the current RDMA read request includes: Obtain the WQE information for sending the current RDMA read request; Obtain the corresponding first QPC information from the QP's transmit QPC, wherein the QP's transmit QPC does not include the second QPC information related to the receive QPC; Based on the WQE information sent for the current RDMA read request and the first QPC information obtained, generate one or at least two sub-RDMA read requests corresponding to the current RDMA read request; Send each of the sub-RDMA read requests.

4. The method according to claim 3, characterized in that, The step of generating one or at least two sub-RDMA read requests corresponding to the current RDMA read request based on the transmitted WQE information of the current RDMA read request and the acquired first QPC information includes: The current RDMA read request is split according to the size of the maximum network packet to obtain sub-RDMA read requests. The size of the packet corresponding to each sub-RDMA read request is equal to the size of the maximum network packet.

5. A method for processing RDMA read requests, characterized in that, Applied to the request-response end, the method includes: Receive the RDMA read request sent by the request sender and read the corresponding response data; Schedule each of the cached RDMA read requests to determine the target RDMA read request for the current response to be sent; Obtain the number of online QPs with unprocessed operation requests in the request response terminal, and generate encoding information based on the number of online QPs; Based on the encoded information and the response data, a response information corresponding to the target RDMA read request is generated, and the response information is sent to the request sending end.

6. The method according to claim 5, characterized in that, The process of reading the corresponding response data includes: Obtain the second QPC information corresponding to the received RDMA read request from the received QPC of the QP, wherein the received QPC of the QP does not include the first QPC information related to the transmitted QPC; Read the corresponding response data according to the RDMA read request and the second QPC information, and cache the response data.

7. An RDMA read request processing apparatus, characterized in that, The device, applied to the request sending end, includes: The first receiving module is used to receive and parse the response information sent by the request-response terminal to obtain the encoded information of the number of online QPs with unprocessed operation requests in the request-response terminal. The threshold quantity acquisition module is used to obtain the threshold quantity of incomplete operation requests sent by the request sending end based on the encoding information of the online QP quantity, wherein the encoding information is used to represent the range of online QP quantity between the request sending end and the response end; The first sending module is used to control the sending of the current RDMA read request to the responding end based on the number of RDMA read requests that the requesting end has sent but has not received a response to and the number threshold.

8. An RDMA read request device, characterized in that, The device, applied to a request-response end, includes: The second receiving module is used to receive the RDMA read request sent by the request sending end and read the corresponding response data; The scheduling module is used to schedule each of the cached RDMA read requests to determine the target RDMA read request for the current response to be sent. The encoding information generation module is used to obtain the number of online QPs with unprocessed operation requests in the request response terminal, and generate encoding information based on the number of online QPs; The second sending module is used to generate response information corresponding to the target RDMA read request based on the encoding information and the response data, and send the response information to the request sending end.

9. A chip comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

10. A network interface card, characterized in that, Includes the chip as described in claim 9 and multiple interfaces. The chip processes data or communicates with the outside world through the interface.

11. A computer device, characterized in that, Includes the network interface card as described in claim 10, wherein the network interface card is used for processing data or external communication.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images