Communication method and device

Abstract

The invention discloses a communication method and device, and relates to the technical field of communication, and the method is applied to first equipment supporting an RDMA function. The method comprises the following steps: acquiring a first RDMA connection establishment request, wherein the first RDMA connection establishment request indicates to establish a target QP connection between first equipment and second equipment; and determining a flow mode of the first flow corresponding to the first RDMA connection establishment request. And determining a target connection mode corresponding to the target QP connection according to the flow mode of the first stream, and transmitting data of the first stream through the target QP connection established based on the target connection mode. Through the method, the data streams in different flow modes can be transmitted by adopting the QP connection established based on different connection modes, so that the communication efficiency based on the RDMA can be improved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a communication method and device. Background Art

[0002] Remote direct memory access (RDMA) technology can bypass the operating system protocol stack and central processing unit (CPU), so RDMA technology can achieve end-to-end high-performance communication with extremely low latency and high bandwidth. This makes RDMA technology widely used in high-performance computing, cloud storage, big data and artificial intelligence (AI) training.

[0003] How to further improve the efficiency of RDMA-based communication has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present application provides a communication method and device, which can improve the efficiency of communication based on RDMA.

[0005] To achieve the above objectives, this application provides the following technical solutions:

[0006] In a first aspect, the present application provides a communication method, which is applied to a first device supporting an RDMA function. The method includes: obtaining a first RDMA connection establishment request, the first RDMA connection establishment request indicating the establishment of a target queue pair (queue pair, QP) connection between the first device and the second device. Determine the traffic pattern of the first stream corresponding to the first RDMA connection establishment request, and determine the target connection pattern corresponding to the target QP connection based on the determined traffic pattern. Transmit data of the first stream through the target QP connection established based on the target connection pattern. Wherein, the target QP connection corresponds to the first stream.

[0007] The method provided by the present application can achieve: for data streams with different traffic patterns, select QP connections established through different connection modes for transmission. That is, the scheme of the present application can select a connection mode suitable for the traffic pattern based on the traffic pattern of the data stream to establish a corresponding QP connection, and transmit the data stream with the traffic pattern through the QP connection. In this way, the communication delay caused when the traffic pattern of the data stream and the connection mode for establishing the QP connection do not match can be avoided. For example, when the connection mode is RC mode, when a data stream with a traffic pattern of a short stream mode is transmitted through a QP connection established based on the RC mode, QP connections will be frequently created based on the RC mode, and QP connections established based on the RC mode will be frequently deleted to release connection resources, wherein both the creation operation and the deletion operation take a certain amount of time, thereby causing the device to generate corresponding communication delays, thereby affecting the efficiency of the device to transmit data based on RDMA, and the scheme provided by the present application can avoid these problems.

[0008] In a possible design, when the traffic mode is a long-stable flow mode, the target connection mode is a reliable connection RC mode. A data flow that is stably transmitted over a long period of time can be referred to as a data flow whose traffic mode is a long-stable flow mode.

[0009] In this possible design, since the data flow with a long and steady flow mode has a stable transmission requirement, there is no need to frequently delete or create QP connections. Therefore, for the data flow with a long and steady flow mode, the RC mode QP connection can be used for transmission.

[0010] In another possible design, the first stream is a data stream initiated by the first application process / thread. The method further includes: obtaining data of a second stream, the second stream is a data stream initiated by the second application process / thread, the traffic mode of the second stream is a long-stable flow mode, and the destination device of the second stream is the second device. The data of the second stream is transmitted through a target QP connection established based on the target connection mode.

[0011] Through this possible design, for multiple data streams initiated by different application processes / threads, when the traffic modes of the multiple data streams are all long-stable flow modes, and the destination devices of the multiple data streams are all the same device, the data streams can be transmitted through a physical QP connection, which is equivalent to the multiple data streams transmitting data through the same physical QP connection in a time-division multiplexing manner. In other words, the present application scheme enables multiple application processes / threads to reuse the same physical QP connection. Therefore, through the present application scheme, a device that supports the RDMA function can serve application processes / threads that are several times the number of physical QP connections through a limited number of physical QP connections, thereby improving the RDMA expansion performance of the device.

[0012] In another possible design, the first device includes multiple groups of priority queues, the first group of priority queues in the multiple groups of priority queues corresponds to the above-mentioned target QP connection, the first group of priority queues includes at least one priority queue, and the priority of the at least one priority queue is used to indicate the priority of scheduling the RDMA data request of each priority queue in the first group of priority queues. The above method also includes: according to the priority of the first application process / thread, adding the first RDMA data request initiated by the first application process / thread to the first priority queue, and scheduling the first RDMA data request according to the priority of the first priority queue, and transmitting the data corresponding to the first RDMA data request through the target QP connection. The priority of the first application process / thread corresponds to the priority of the first priority queue. The priority of the application process / thread corresponds to the quality of service (QoS) level of the application process / thread, for example, a high-priority application process / thread has a high QoS level requirement.

[0013] Through this possible design, the first device maintains a set of priority queues for each QP connection, so that the first device can store the RDMA data requests initiated by multiple application processes / threads that reuse the QP connection in the priority queues of corresponding priorities according to the priority of each application process / thread that reuses the QP connection. The first device schedules the RDMA data requests according to the priority of each priority queue, so that RDMA data requests initiated by application processes / threads with high priorities can be prioritized. Therefore, this design meets the needs of application processes / threads with high QoS level requirements and improves user experience.

[0014] In another possible design, the plurality of priority queues are scheduled by a plurality of scheduling threads, wherein one priority queue corresponds to one scheduling thread.

[0015] Through this possible design, the first device can achieve the purpose of concurrently processing RDMA data requests in multiple priority queues by running multiple threads concurrently. This can improve the efficiency of the first device submitting RDMA data requests (RDMA data requests can also be understood as work requests (WR)) to a network card (RDMA network interface card, RNIC) that supports the RDMA function, and can avoid the problem of head-of-line blocking.

[0016] In another possible design, when the traffic mode is a short flow mode, the target connection mode is a dynamic connected transport (DCT) mode, wherein a data flow that bursts / transmits in a short time is called a data flow with a short flow mode.

[0017] In this possible design, since the data stream with a traffic mode of short flow mode has the characteristic of short transmission time, the QP connection for transmitting the short flow will be idle within a short time after creation and will be deleted, so that the released connection resources can be used to establish a QP connection with other devices. Therefore, for the data stream with a traffic mode of short flow mode, the present application adopts a QP connection in DCT mode for transmission. This is because the QP connection in DCT mode has low switching cost and low switching delay compared to the QP connection in RC mode.

[0018] In another possible design, the first device is configured with a dynamic connection pool, the dynamic connection pool includes multiple DCT connections, each DCT connection can be switched between multiple QP connections established based on the DCT mode, and one QP connection is maintained as an available connection. For a first DCT connection among the multiple DCT connections, when the available connection of the first DCT connection is the first QP connection, after the first device transmits data corresponding to a preset number of RDMA data requests through the first QP connection, the first DCT connection switches the available connection from the first QP connection to the second QP connection.

[0019] Through this possible implementation, after the available connection of each DCT connection transmits the data corresponding to the preset number of RDMA data requests, the DCT connection switches the current available connection to another QP connection to achieve communication with another device. This can avoid the destination device waiting too long for the RDMA data requests of different devices to be processed, thereby ensuring communication efficiency.

[0020] In another possible design, the first RDMA connection establishment request carries a flow mark, and the flow mark is used to indicate the traffic mode of the first flow corresponding to the first RDMA connection establishment request. The above-mentioned determination of the traffic mode of the first flow corresponding to the first RDMA connection establishment request includes: determining the traffic mode of the first flow according to the flow mark carried in the first RDMA connection establishment request.

[0021] In this possible design approach, the application process / thread specifies the traffic pattern of the first flow.

[0022] In another possible design, the determining of the traffic pattern of the first flow corresponding to the first RDMA connection establishment request includes: determining the traffic pattern of the first flow according to a traffic feature of the first flow.

[0023] In this possible design, when the traffic pattern of the first stream changes, the first device can timely determine the latest traffic pattern of the first stream according to the traffic characteristics of the first stream, and then select the QP connection established based on the appropriate connection mode to transmit the first stream according to the latest traffic pattern. In this way, it is possible to avoid wasting the QP connection resources of the RC mode on the basis of ensuring communication efficiency. For example, for the first stream whose traffic pattern is a long-stable flow mode, if a QP connection in DCT mode is used for transmission, since the QP connection in DCT mode in this application will be switched periodically, when the destination device of the switched QP connection is not the destination device of the first stream, the waiting processing time of the RDMA data request indicating the transmission of the first stream data will inevitably increase, resulting in low transmission efficiency of the first stream. For another example, for the first stream whose traffic pattern is a short-flow mode, if a QP connection in RC mode is used for transmission, the QP connection will be idle and wasted after a short burst of the first stream.

[0024] In another possible design, link establishment information of multiple devices is pre-stored in the first device, and the link establishment information includes a global identifier (global identifier, GID) of the RNIC in the device and a queue pair number (QP number, QPN) of the QP in the RNIC. The link establishment information is used by the first device to establish a QP connection with multiple devices.

[0025] Through this possible design, when the first device establishes a QP connection with another device, it is not necessary to obtain the link establishment information of the other device online, thereby improving the efficiency of the first device in establishing the QP connection, and further improving the communication efficiency of the first device.

[0026] In another possible design, the method is executed by an intermediate component of the first device, and the intermediate component pre-stores the driver context of the RNIC in the first device. Before obtaining the first RDMA connection establishment request, the method further includes: initializing the pre-stored driver context, and the initialized driver context is used to establish the QP connection.

[0027] Through this possible design, each application process / thread in the first device does not need to pre-store the RNIC driver context in its own user space, and thus does not need to initialize the RNIC driver context of its own user space when a QP connection needs to be established. The first device only needs to initialize the RNIC driver context once when the intermediate component is started. This can effectively improve the efficiency of the first device in establishing a QP connection for the application process / thread.

[0028] In another possible design, the first RDMA connection establishment request is an RDMA request initiated by an application process / thread in the first device by calling a first user-state library application programming interface (libibverbs application programming interface, verbs API).

[0029] In another possible design, the first verbs API call has a Unix domain socket (UDS) interface or a shared memory (SHM) interface. The obtaining of the first RDMA connection establishment request includes: receiving the first RDMA connection establishment request initiated by the application process / thread in the first device through the UDS interface or the SHM interface.

[0030] Through the above two possible design methods, communication between the application process in the first device and the intermediate component that executes the method of the present application in the first device can be achieved.

[0031] In another possible design, the above-mentioned transmission of data of the first stream through the target QP connection established based on the target connection mode includes: sending an RDMA data request indicating the transmission of data of the first stream to the RNIC of the first device by calling the second verbs API, so as to realize the transmission of data of the first stream through the target QP connection.

[0032] Through this possible design approach, communication between the intermediate component executing the method of the present application in the first device and the RNIC of the first device can be achieved.

[0033] In another possible design, the method further includes: obtaining a target completion queue entry (CQE) from the RNIC of the first device, and returning the target CQE to the application process / thread that initiates the first RDMA data request. The target CQE is generated after the RNIC in the first device completes the first RDMA data request, and the target CQE includes a first work request identifier for indicating the first RDMA data request.

[0034] In another possible design, the first device is configured with a mapping relationship set including a mapping relationship between an application process / thread initiating each RDMA data request and a work request identifier indicating each RDMA data request. Before returning the target CQE to the application process / thread initiating the first RDMA data request, the method further includes: querying the mapping relationship set according to the first work request identifier to determine the application process / thread initiating the first RDMA data request.

[0035] Through the above two possible implementations, the purpose of returning the CQE to the application process can be achieved.

[0036] In a second aspect, the present application provides a communication device.

[0037] In a possible design, the communication device is used to execute any one of the methods provided in the first aspect above. The present application may divide the communication device into functional modules according to any one of the methods provided in the first aspect above. For example, each functional module may be divided according to each function, or two or more functions may be integrated into one processing module. Exemplarily, the present application may divide the training device into an acquisition unit, a processing unit, and a transmission unit, etc. according to the function. The description of the possible technical solutions and beneficial effects executed by the above-mentioned divided functional modules can refer to the technical solutions provided by the above-mentioned first aspect or its corresponding possible design, and will not be repeated here.

[0038] In another possible design, the communication device includes: one or more processors and a memory, and the one or more processors are configured to call program instructions stored in the memory so that the communication device executes any method provided by the first aspect and any possible design thereof.

[0039] In a third aspect, the present application provides a computer-readable storage medium, which includes program instructions. When the program instructions are executed on a computer or a processor, the computer or the processor executes any method provided by any possible implementation in the first aspect.

[0040] In a fourth aspect, the present application provides a computer program product, which, when executed on a communication device, enables any method provided by any possible implementation of the first aspect to be executed.

[0041] In a fifth aspect, an embodiment of the present application provides a communication system, the communication system comprising a first device and a second device. The first device implements communication with the second device by executing any method provided in any possible implementation manner of the first aspect.

[0042] It can be understood that any of the above-mentioned devices, computer-readable storage media or computer program products can be applied to the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods and will not be repeated here.

[0043] In this application, the name of the above communication device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of this application, they fall within the scope of the claims of this application and their equivalent technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A flow chart of a communication method provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of establishing a QP connection provided in an embodiment of the present application;

[0046] Figure 3 A flowchart of another communication method provided in an embodiment of the present application;

[0047] Figure 4 A flowchart of another communication method provided in an embodiment of the present application;

[0048] Figure 5 A schematic diagram of the structure of a communication device provided in an embodiment of the present application;

[0049] Figure 6 A schematic diagram of the hardware structure of a device supporting RDMA function provided in an embodiment of the present application;

[0050] Figure 7 A schematic diagram of the logical structure of a device supporting RDMA function provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to more clearly understand the embodiments of the present application, some terms or technologies involved in the embodiments of the present application are explained below:

[0052] 1) RDMA

[0053] RDMA is a transport layer network protocol. RDMA uses queue pairs (QPs) to send and receive data from consumers (such as application processes) over the network.

[0054] QP contains a send queue (SQ) and a receive queue (RQ). SQ is used to store send tasks, and RQ is used to store receive tasks. QP is also associated with a completion queue (CQ). CQ is used to store completion queue entries (CQE) indicating that send tasks or receive tasks have been completed.

[0055] Specifically, when the device / host writes a work queue element (WQE) indicating the work request (WR) submitted by the consumer into the SQ or RQ of the corresponding QP, the network card of the device performs the RDMA operation according to the instruction of the WQE. When the device completes an RDMA operation indicated by the WQE, the device writes a CQE to the CQ associated with the QP to indicate that the data sending task or data receiving task corresponding to the RDMA operation has been completed. Furthermore, when the device processes the CQE in the CQ, it can notify the consumer that the WR submitted by the consumer has been processed.

[0056] RDMA operations include message-type operations, such as send operations, send with immediate operations, and send with invalidate operations. RDMA operations also include memory-type operations, such as read operations, write operations, and atomic operations. The consumer may be an application process running in a device, or a thread in an application process, but is not limited thereto. A process has independent memory resources, and a process may include multiple threads, and the multiple threads share the memory resources allocated by the device to the process.

[0057] 2) Reliable connection

[0058] The RDMA message is transmitted from the SQ of one QP to the RQ of another designated QP, and after the device where the other designated QP is located completes receiving the RDMA message, it returns an acknowledgement (ACK) message to the device that sent the RDMA message to inform that the RDMA message has been received by the RQ of the designated QP. In this case, the connection between the two QPs is called an RDMA reliable connection.

[0059] In the embodiment of the present application, the RDMA reliable connection may be a QP connection established based on a reliable connection (RC) mode, or may be a QP connection established based on a dynamic connected transport (DCT) mode.

[0060] Among them, the QP connection established based on the RC mode only supports connecting one to two devices. For example, for any QP in device 1, when the destination device connected to the QP connection 1 established based on the RC mode where the QP is currently located is device 2, when it is necessary to connect to device 3 through the QP, it is necessary to first dismantle QP connection 1, and then create a connection between the QP and the QP in device 3 based on the RC mode. This process will generate a large delay and consume a lot of resource costs.

[0061] The QP connection established based on the DCT mode allows fast switching of connections between a device and multiple destination devices, and the switching delay and switching cost are both low. The RC mode and the DCT mode are not described in detail in the present application embodiment.

[0062] 3) Other terms

[0063] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0064] In the embodiments of the present application, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0065] The term "at least one" in this application means one or more, and the term "plurality" in this application means two or more.

[0066] In some scenarios, in order to improve the scalability of RDMA, a connectionless or message-oriented transport layer can be used to transmit RDMA packets. This method does not require the establishment of a QP connection, but directly sends and receives RDMA packets, thereby avoiding the delay when establishing a QP connection and the overhead of allocating RDMA connection resources when establishing a QP connection. This not only improves communication efficiency, but also improves the scalability of RDMA. However, this method cannot guarantee the reliability of data transmission.

[0067] In other scenarios, RDMA connection resources (such as QP metadata) are managed by the kernel so that RDMA connection resources can be shared between different processes (or containers, virtual machines, etc.). In addition, the QP connection created for a process (or container, virtual machine, etc.) in this way can be reused by threads within the process. Based on this, the scalability performance of RDMA can be improved, thereby improving communication performance. However, when RDMA connection resources are managed by the kernel, when the application process applies for RDMA connection resources through the standard Verbs API interface to establish a QP connection, the kernel does not support the Verbs API interface.

[0068] In some other scenarios, RDMA resource sharing can be performed through the user state library to solve the scalability of RDMA and improve communication performance. In this way, the RDMA connection resources stored in the user space of the application process are managed by a dedicated thread in the application, so that the threads in the application process can share the RDMA connection resources. However, this solution can only improve the scalability of RDMA for threads in the process, and cannot support RDMA resource sharing between processes.

[0069] Based on this, an embodiment of the present application provides a communication method and apparatus, which is applied to a first device supporting an RDMA function. The method includes: after the first device obtains a first RDMA connection establishment request and determines the traffic pattern of the first stream corresponding to the first RDMA connection establishment request, the target connection pattern corresponding to the target QP connection is determined according to the traffic pattern of the first stream. The target QP connection is the QP connection established between the first device and the second device indicated by the first RDMA connection establishment request, and the target QP connection corresponds to the first stream. Then, the first device transmits data of the first stream through the target QP connection established based on the target connection pattern.

[0070] Through this method, it is possible to select QP connections established through different connection modes for transmission for data streams with different traffic patterns. That is, the solution provided by the embodiment of the present application can select a connection mode suitable for the traffic pattern based on the traffic pattern of the data stream to establish a corresponding QP connection, and transmit the data stream with the traffic pattern through the established QP connection. In this way, the communication delay caused by the mismatch between the traffic pattern of the data stream and the connection mode for establishing the QP connection can be avoided, thereby improving communication efficiency.

[0071] The embodiment of the present application also provides a communication device, which is used to execute the communication method provided in the embodiment of the present application. Optionally, the communication device can be any device supporting the RDMA function, or a functional module in a device supporting the RDMA function, which is not limited.

[0072] As an example, the above-mentioned communication device can be a general-purpose computer, laptop computer, tablet computer or other computing device configured with a network interface card / network interface card (RNIC) that supports RDMA function, or the above-mentioned communication device can be a network device such as a server, switch, router or the like configured with RNIC, but is not limited to this.

[0073] The communication method provided in the embodiment of the present application is described in detail below in conjunction with the accompanying drawings, taking the communication device as an example in which the communication device is a first device supporting the RDMA function.

[0074] refer to Figure 1 , Figure 1 A flow chart of a communication method provided in an embodiment of the present application is shown, and the method comprises the following steps:

[0075] S101: Obtain a first RDMA connection establishment request.

[0076] The first RDMA connection establishment request indicates that a target QP connection is established between the first device and the second device. For the first device, the second device can be referred to as a destination device of the target QP connection. Therefore, the first RDMA connection establishment request includes an identifier of the destination device of the target QP connection, that is, the first RDMA connection establishment request includes an identifier of the second device.

[0077] Among them, the second device is a device that needs to exchange data with the first application process / thread in the first device. The second device can have the same hardware structure as the first device, or it can have a different hardware structure, without limitation. The first application process is any process running in the first device that needs to exchange network data. The thread can be any thread in the first application process, without limitation. As an example, the first application process can be a process for implementing a certain business, or a process for implementing a virtual machine, or a process for implementing a container, without limitation.

[0078] The identifier of the second device may be an Internet protocol (IP) address of the second device, a media access control (MAC) address, an identification number (ID) of the second device, or a global identifier (GID) of an RNIC in the second device, etc., but is not limited thereto. The GID of the RNIC in the second device can uniquely identify the RNIC of the second device in the network system where the first device is located, or can uniquely identify the RNIC of the second device in all devices that perform network communication with the first device, but is not limited thereto.

[0079] Specifically, when the first application process / thread needs to exchange data with the second device, the first device may initiate (or be understood as "generating") a first RDMA connection establishment request through the first application process / thread.

[0080] As an example, when the first application process / thread in the first device needs to read data stored in the second device, or the first application process / thread in the first device needs to write data to the second device, the first application process / thread in the first device initiates a first RDMA connection establishment request to indicate that a target QP connection is established between the first device and the second device.

[0081] Optionally, the first device may obtain the first RDMA connection establishment request initiated by the first application process / thread through an intermediate component, wherein the intermediate component may be a process running in the first device for managing the RDMA resources of the first device.

[0082] Optionally, the first application process / thread in the first device may initiate a first RDMA connection establishment request by calling a first user state library application programming interface (libibverbs application programming interface, verbsAPI) from a front-end component of the first device. The front-end component is a component in the first device that implements communication between the application process and the intermediate component. The first verbs API interface calls a UDS interface or a shared memory SHM interface. Thus, the intermediate component of the first device may receive the first RDMA connection establishment request initiated by the first application process / thread in the first device through the UDS interface or the SHM interface.

[0083] In a possible implementation, a first interface library is pre-installed in the front-end component of the first device, and each verbs API interface in the first interface library is designed to call a verbs API interface of a UDS interface or a SHM interface. This verbs API interface is the verbs API interface provided by the embodiment of the present application as described above. Thus, when the first application process / thread in the first device needs to exchange data with the second device, it calls the first verbs API interface from the first interface library of the front-end component as needed, thereby achieving the purpose of sending the first RDMA connection establishment request from the first application process / thread to the first device intermediate component.

[0084] In another possible implementation, a second interface library is pre-installed in the front-end component of the first device, and the verbs API interfaces in the second interface library are all standard verbs API interfaces. Therefore, when the second application process / thread in the first device needs to exchange data with the second device, it calls the standard verbs API interface from the second interface library of the front-end component as needed, and modifies the called standard verbs API interface so that the standard verbs API interface further calls the verbs API interface of the UDS interface or the SHM interface, thereby achieving the purpose of sending the first RDMA connection establishment request from the first application process / thread to the first device intermediate component based on the modified standard verbs API interface (i.e., the first verbs API interface).

[0085] S102: Determine a traffic pattern of a first flow corresponding to a first RDMA connection establishment request.

[0086] The first flow corresponding to the first RDMA connection establishment request refers to: a data flow initiated by the first application process / thread that initiates the first RDMA connection establishment, or a data flow sent to the first application process / thread. Specifically, the first flow refers to the traffic that needs to transmit data through the target QP connection established based on the first RDMA connection establishment request.

[0087] Traffic modes include long-stable flow mode and short-stable flow mode. Traffic with a long-stable flow mode (i.e., long-stable flow) refers to traffic that is stably transmitted over a long period of time, while traffic with a short-stable flow mode (i.e., short-stable flow) refers to traffic that completes data transmission in a short period of time.

[0088] In a first possible implementation, when a first application process / thread in a first device initiates a first RDMA connection establishment request, a flow tag is added to the first RDMA connection establishment request, and the flow tag is used to indicate the traffic mode of the first flow corresponding to the first RDMA connection establishment request. Therefore, the intermediate component of the first device can determine the traffic mode of the first flow corresponding to the first RDMA connection establishment request according to the flow tag carried by the first RDMA connection establishment request.

[0089] As an example, when the flow mark "1" indicates a long steady flow mode and the flow mark "0" indicates a short flow mode, if the intermediate component of the first device determines that the flow mark carried in the first RDMA connection establishment request is "1", the flow mode of the first flow is determined to be a long steady flow mode. If the intermediate component of the first device determines that the flow mark carried in the first RDMA connection establishment request is "0", the flow mode of the first flow is determined to be a short flow mode.

[0090] Optionally, the first application process / thread in the first device may also separately send a traffic tag to the intermediate component of the first device, so that the intermediate component of the first device determines the traffic pattern of the first flow based on the traffic tag. The first application process / thread in the first device may separately send a traffic tag representing the traffic pattern of the first flow to the intermediate component of the first device before, at the same time, or after initiating the first RDMA connection establishment request, and this is not limited.

[0091] In a second possible implementation, the intermediate component of the first device may determine the flow pattern of the first flow according to the flow characteristics of the first flow.

[0092] Specifically, the intermediate component of the first device can record the traffic characteristics of the first flow, and the traffic characteristics include, for example, the size of each data packet in the first flow, the transmission time interval between two adjacent data packets, etc. In this way, the first device can analyze and determine the traffic pattern of the first flow according to the traffic characteristics recorded for the first flow.

[0093] For example, when the traffic characteristics of the first flow indicate that the first flow continuously and stably transmits data packets at intervals of 1 second, the intermediate component of the first device determines that the traffic mode of the first flow is a long-stable flow mode.

[0094] For another example, when the traffic characteristic of the first flow indicates that the first flow stops transmitting data packets after transmitting only one data packet, the intermediate component of the first device determines that the traffic mode of the first flow is the short flow mode.

[0095] It should be understood that in actual applications, when the intermediate component of the first device obtains the first RDMA connection establishment request, and the intermediate component of the first device does not obtain the traffic tag characterizing the traffic pattern of the first stream, the intermediate component of the first device defaults to transmitting the data of the first stream through the target QP connection established based on the DCT mode. This is because the QP connection established based on the DCT mode has a lower switching delay and switching cost. In addition, the intermediate component of the first device records the traffic characteristics of the first stream during the process of transmitting the first stream data through the target QP connection of the DCT mode. In this way, the intermediate component of the first device can determine the traffic pattern of the first stream based on the traffic characteristics of the first stream recorded within a period of time after obtaining the first RDMA connection establishment request, and execute the following S103 to S104 based on the determined traffic pattern.

[0096] Optionally, the intermediate component of the first device can periodically determine the traffic pattern of the first flow according to the traffic characteristics with a preset time period. In this way, the latest traffic pattern of the first flow can be obtained. Furthermore, the first device can perform the following S103 to S104 according to the latest traffic pattern of the first flow.

[0097] Optionally, after the intermediate component of the first device determines the traffic pattern of the first stream based on the above-mentioned first possible design method, and executes S103 to S104 based on the determined traffic pattern to transmit the data of the first stream through the target QP connection established based on the target connection mode, the intermediate component of the first device may also record the traffic characteristics of the first stream during the transmission of the first stream, and periodically determine the latest traffic pattern of the first stream, and execute the following S103 to S104 according to the latest determined traffic pattern, so as to realize that when the traffic pattern of the first stream data changes during the transmission process, the QP connection transmitting the first stream is promptly switched to a QP connection under a suitable connection mode.

[0098] Optionally, the intermediate component of the first device may execute the following S103 to S104 when the latest determined first stream traffic pattern is different from the previously determined first stream traffic pattern, so as to timely switch the QP connection transmitting the first stream to a QP connection under a suitable connection mode when the traffic pattern of the first stream data changes during transmission.

[0099] It should be understood that by timely adjusting the connection mode of the QP connection used to transmit the data stream when the traffic pattern of the data stream changes, the first device can avoid wasting the QP connection resources of the RC mode while ensuring communication efficiency. For example, for the first stream whose traffic pattern is the long-stable flow mode, if the QP connection established based on the DCT mode is used for transmission, since the QP connection established based on the DCT mode in the embodiment of the present application will be switched periodically (refer to the description of the dynamic connection pool below), when the destination device of the switched QP connection is not the destination device of the first stream (i.e., the second device), the waiting processing time of the RDMA data request indicating the transmission of the first stream data will inevitably increase, which will lead to a reduction in the transmission efficiency of the first stream, i.e., a reduction in communication efficiency. For another example, for the first stream whose traffic pattern is the short-stable flow mode, if the QP connection of the RC mode is used for transmission, the QP connection will be idle and wasted after a short burst of the first stream.

[0100] S103: Determine a target connection mode corresponding to the target QP connection according to the traffic mode of the first flow.

[0101] The target QP connection is used to transmit data of the first flow corresponding to the first RDMA connection establishment request, so the target QP connection corresponds to the first flow.

[0102] The target connection mode is a connection mode for establishing a target QP connection. As an example, the target connection mode may be an RC mode or a DCT mode, but is not limited thereto. When the target connection mode is an RC mode, the target QP connection is a QP connection established based on the RC mode. When the target connection mode is a DCT mode, the target QP connection is a QP connection established based on the DCT mode. The description of the DCT mode and the RC mode can refer to the above description and will not be repeated.

[0103] Optionally, when the traffic mode of the first stream is a long steady stream mode, the intermediate component of the first device determines that the target connection mode is an RC mode. Since the embodiment of the present application maintains the following first mapping relationship set for the QP connection established based on the RC mode, multiple application processes / threads can multiplex a QP connection for data transmission, thereby improving the scalability of RDMA.

[0104] The first mapping relationship set is configured and maintained by the first device, and the first mapping relationship set includes multiple mapping relationships, each mapping relationship includes a mapping relationship between at least one virtual connection and a QP connection. The QP connection is a QP connection established based on the RC mode. The relevant description of the virtual connection and the QP connection can refer to the relevant description above and will not be repeated.

[0105] In a specific implementation, each mapping relationship in the first mapping relationship set specifically includes at least one virtual connection identifier and one QP connection identifier. As an example, the QP connection identifier may be a number and / or a string set by an intermediate component of the first device for the QP connection, or the QP connection identifier may be a QPN of the QP used by the QP connection in the first device, or the QP connection identifier may be a GID of an RNIC in a destination device of the QP connection, without limitation thereto.

[0106] As an example, the first mapping relationship set can be specifically implemented as a first mapping relationship table maintained by the first device intermediate component. Referring to Table 1, Table 1 exemplarily shows a first mapping relationship table. As shown in Table 1, the first mapping relationship table includes: mapping relationships between virtual connection 1, virtual connection 2, and virtual connection 3 and QP connection 1, respectively, mapping relationship between virtual connection 4 and QP connection 2, and mapping relationship between virtual connection 5 and virtual connection 6 and QP connection 3, respectively.

[0107] Table 1

[0108]

[0109] Optionally, when the traffic mode of the first flow is the short flow mode, the intermediate component of the first device determines the target connection mode to be the DCT mode. This is because the QP connection established based on the DCT mode has the characteristics of low switching delay and low switching cost. Therefore, for short flows, the embodiment of the present application determines the DCT mode as the target connection mode.

[0110] In some possible situations, when the first device currently transmits data of the first stream in a QP connection in DCT mode, and the intermediate component of the first device determines that the latest traffic mode is a long steady flow mode based on the traffic characteristics of the first stream, the first device determines to determine the RC mode as the target connection mode, that is, the first device determines to switch the QP connection in the DCT mode that transmits the data of the first stream to a QP connection in RC mode. When the first device currently transmits data of the first stream in a QP connection in DCT mode, and the intermediate component of the first device determines that the latest traffic mode is a short flow mode based on the traffic characteristics of the first stream, the first device determines to continue to determine the DCT mode as the target connection mode, that is, the first device determines to continue to transmit data of the first stream through the QP connection in DCT mode.

[0111] In some other possible situations, when the first device currently transmits the data of the first stream in a QP connection in RC mode, and the intermediate component of the first device determines that the latest traffic mode is a long steady flow mode according to the traffic characteristics of the first stream, the first device determines to continue to determine the RC mode as the target connection mode, that is, the first device continues to determine to transmit the data of the first stream through the QP connection in RC mode. When the first device currently transmits the data of the first stream in a QP connection in RC mode, and the intermediate component of the first device determines that the latest traffic mode is a short flow mode according to the traffic characteristics of the first stream, the first device determines to determine the DCT mode as the target connection mode, that is, the first device determines to switch the QP connection in RC mode that transmits the data of the first stream to a QP connection in DCT mode.

[0112] S104: Transmit data of the first stream through a target QP connection established based on the target connection mode.

[0113] Specifically, after determining the target connection mode based on the traffic mode of the first flow, the intermediate component of the first device also records the correspondence between the first flow and the target connection mode. For simple description, the "correspondence between the first flow and the target connection mode" is referred to as the first correspondence below.

[0114] Exemplarily, when the intermediate component of the first device determines that the target connection mode is the RC mode according to the traffic mode of the first stream, the first correspondence recorded by the intermediate component of the first device is the correspondence between the first stream and the RC mode. Also exemplarily, when the intermediate component of the first device determines that the target connection mode is the DCT mode according to the traffic mode of the first stream, the first correspondence recorded by the intermediate component of the first device is the correspondence between the first stream and the DCT mode.

[0115] Optionally, since the first flow is a data flow initiated by the first application process / thread, or a data flow sent to the first application process / thread, in the first corresponding relationship recorded by the first device intermediate component, the first flow can be represented by the process / thread ID of the first application process / thread.

[0116] Furthermore, when the intermediate component of the first device receives an RDMA data request indicating the transmission of the first stream data, it can determine, based on the established first correspondence, that the data of the first stream is transmitted through the target QP connection established based on the target connection mode. Then, the intermediate component of the first device transmits the data of the first stream through the target QP connection established based on the target connection mode. The detailed process can be referred to the description of S201 to S204 below, which will not be repeated here.

[0117] Specifically, in one possible scenario, when the intermediate component of the first device determines that the target connection mode is the RC mode, the intermediate component of the first device, in addition to recording the first corresponding relationship, also creates a target QP connection in the RC mode according to the first RDMA connection establishment request. Further, when the intermediate component of the first device receives an RDMA data request indicating the transmission of the first stream data, the first device determines and transmits the data of the first stream through the target QP connection established based on the RC mode based on the established first corresponding relationship. The detailed description of the intermediate component of the first device creating the target QP connection in the RC mode according to the first RDMA connection establishment request can be referred to the description below and will not be repeated.

[0118] In another possible situation, after the intermediate component of the first device determines that the target connection mode is the DCT mode and records the first correspondence, when the intermediate component of the first device receives an RDMA data request indicating the transmission of the first stream data, the intermediate component of the first device determines and transmits the first stream data through the target QP connection established based on the DCT mode based on the established first correspondence. The target QP connection established based on the DCT mode is a QP connection in a dynamic connection pool preset in the first device. The detailed description of the dynamic connection pool can be referred to the description below and will not be repeated here.

[0119] Through the method described in S101 to S104, it is possible to select QP connections established through different connection modes for transmission for data streams with different traffic patterns. That is, the method can select a connection mode suitable for the traffic pattern based on the traffic pattern of the data stream to establish a corresponding QP connection, and transmit the data stream with the traffic pattern through the QP connection. This can avoid the communication delay caused when the traffic pattern of the data stream and the connection mode for establishing the QP connection do not match.

[0120] For example, when the connection mode is RC mode, when a data stream with a traffic mode of short flow mode is transmitted through a QP connection established based on the RC mode, QP connections based on the RC mode are frequently created, and QP connections established based on the RC mode are frequently deleted to release connection resources. Both the creation operation and the deletion operation take a certain amount of time, which causes the device to generate corresponding communication delays, thereby affecting the efficiency of the device in transmitting data based on RDMA. The solution provided in the present application can avoid these problems, thereby improving communication efficiency.

[0121] The following describes the process of "the intermediate component of the first device creates a target QP connection in RC mode according to the first RDMA connection establishment request".

[0122] The first RDMA connection establishment request includes, in addition to the identifier of the second device, an identifier of a virtual connection, where the virtual connection is a virtual QP connection (hereinafter referred to as virtual connection) for transmitting first stream data.

[0123] In an embodiment of the present application, for each application process running on the first device and needing to exchange data with other devices, the first device may set one or more virtual connections for each application process, and each virtual connection is represented by a preset identifier. Then, the first device establishes a mapping relationship between the virtual connection and the physical QP connection, so that each application process can transmit data through the physical QP connection that has a mapping relationship with the virtual connection. Among them, the identifier of the virtual connection can be, for example, any number or string, etc., which is not limited. In actual implementation, the identifier of the virtual connection can be represented by the process identification number (identity, ID) of the application process that initiates the first RDMA connection establishment request. The physical QP connection is an actual QP connection established by the RNIC of the first device based on the RC mode. For the sake of simple description, the physical QP connection is referred to as the QP connection in the embodiments of the present application below.

[0124] In an embodiment of the present application, multiple virtual connections set by the first device for multiple application processes can be mapped to one QP connection, that is, the same QP connection can be mapped with one or more virtual connections. In this way, the data requested to be transmitted by the RDMA data request carrying the virtual connection identifier can be transmitted through the QP connection that has a mapping relationship with the virtual connection. For example, for RDMA data request 1 carrying identifier 1, assuming that identifier 1 is the identifier of virtual connection 1, the data requested to be transmitted by RDMA data request 1 can be transmitted through the QP connection that has a mapping relationship with virtual connection 1.

[0125] Optionally, when an application process includes multiple threads, and the multiple threads need to exchange data with the same device, the first device can set a virtual connection for the multiple threads, that is, the multiple threads share the same virtual connection. Furthermore, when the multiple threads exchange data with the aforementioned same device, the first device transmits data through a QP connection that has a mapping relationship with the virtual connection.

[0126] Optionally, when an application process includes multiple threads, the first device can set a virtual connection for each of the multiple threads, that is, one thread corresponds to one virtual connection. In this case, the identifier of the virtual connection can be any number or string, etc., or the identifier of the virtual connection can be represented by the thread ID of the thread, which is not limited. Furthermore, for any thread among the multiple threads, such as the first thread, assuming that the virtual connection set by the first device for the first thread is the first virtual connection, when the first thread exchanges data with other devices, the first device transmits data through the QP connection that has a mapping relationship with the first virtual connection.

[0127] It should be understood that by mapping multiple virtual connections set for multiple application processes / threads to the same QP connection, the scalability performance of RDMA connection resources can be improved.

[0128] In the embodiment of the present application, the first device further sets a virtual queue for each virtual connection, and the virtual queue is used to store pending RDMA data requests, and these pending RDMA data requests indicate that the transmitted data needs to be transmitted through the QP connection in RC mode. Optionally, the virtual queue is set in the memory of the first device.

[0129] Among them, the RDMA data request to be processed is the RDMA data request obtained by the intermediate component of the first device from the application process / thread. The RDMA data request is specifically a work request (WR) instructing the application process / thread to transmit data, such as a request to send data or a request to receive data, etc., but is not limited to this.

[0130] Since each QP connection in the first mapping relationship set is mapped to at least one virtual connection, each QP connection in the first mapping relationship set corresponds to at least one virtual queue, and the at least one virtual queue corresponds to the at least one virtual connection. It should be understood that the data to be transmitted indicated by the RDMA data request to be processed stored in the at least one virtual queue are all transmitted through the QP connection having a mapping relationship with the at least one virtual connection, or are understood to be transmitted through the QP connection corresponding to the at least one virtual queue.

[0131] It can be understood that, in the embodiment of the present application, the first RDMA connection establishment request indicates the establishment of a target QP connection between the first device and the second device, which can be understood as indicating the process of establishing a mapping relationship between the virtual connection represented by the virtual connection identifier carried by the first RDMA connection establishment request and the target QP connection. Correspondingly, when the RDMA request is used to request the destruction of the target QP connection, the RDMA request is actually used to indicate the process of destroying the mapping relationship between the virtual connection represented by the virtual connection identifier carried by the RDMA request and the target QP connection.

[0132] Specifically, the intermediate component of the first device may first determine whether the RNIC of the first device has established a target QP connection between the first device and the second device based on the RC mode according to the identifier of the second device carried in the first RDMA connection establishment request.

[0133] In a possible implementation, metadata of all QP connections established by the RNIC of the first device based on the RC mode is pre-stored in the memory of the intermediate component of the first device. For any QP connection established by the RNIC of the first device based on the RC mode, assuming that the QP connection is a QP connection for connecting the first device and the third device, the metadata of the QP connection includes but is not limited to: connection status information of the QP connection, GID of the RNIC of the first device, QP number (QP number, QPN) of a QP in the RNIC of the first device, GID of the RNIC of the third device, QPN of a QP in the RNIC of the third device, etc.

[0134] Optionally, when the second device identifier carried by the first RDMA connection establishment request is the GID of the RNIC of the second device, the intermediate component of the first device can directly search in the QP connection metadata pre-stored in the memory of the intermediate component of the first device according to the GID of the RNIC of the second device carried by the first RDMA connection establishment request to determine whether the RNIC of the first device has established the target QP connection based on the RC mode. Optionally, when the second device identifier carried by the first RDMA connection establishment request is the address of the second device (such as an IP address or a MAC address), the intermediate component of the first device can first determine the GID of the RNIC of the second device according to the address of the second device. It should be understood that in this case, multiple corresponding relationships are pre-stored in the memory of the intermediate component of the first device, and each corresponding relationship is used to characterize the corresponding relationship between the address of a device and the GID of the RNIC in the device. In this way, the intermediate component of the first device can search in the QP connection metadata pre-stored in the memory of the intermediate component of the first device according to the determined GID of the RNIC of the second device to determine whether the RNIC of the first device has established the target QP connection based on the RC mode.

[0135] When the intermediate component of the first device determines that the GID of the RNIC of the second device exists in the pre-stored QP connection metadata, the intermediate component of the first device determines that the RNIC of the first device has established the target QP connection based on the RC mode. When the intermediate component of the first device determines that the GID of the destination device does not exist in the pre-stored QP connection metadata, the intermediate component of the first device determines that the RNIC of the first device has not established the target QP connection based on the RC mode.

[0136] In one possible scenario, when the intermediate component of the first device determines that the RNIC of the first device has established a target QP connection based on the RC mode, the intermediate component of the first device establishes a mapping relationship between the target QP connection and the virtual connection represented by the virtual connection identifier carried in the first RDMA connection establishment request, and adds the mapping relationship to the first mapping relationship set.

[0137] In combination with Table 1, when the intermediate component of the first device determines that the RNIC of the first device has established a QP connection 2 (i.e., a target QP connection) for connecting the first device and the second device based on the RC mode, and the virtual connection represented by the virtual connection identifier carried in the first RDMA connection establishment request is a virtual connection 7, the intermediate component of the first device establishes a mapping relationship between virtual connection 7 and QP connection 2, and adds the mapping relationship to the first mapping relationship set shown in Table 1, thereby obtaining the first mapping relationship set shown in Table 2.

[0138] Table 2

[0139]

[0140] In another possible situation, when the intermediate component of the first device determines that the RNIC of the first device has not established the target QP connection based on the RC mode, the intermediate component of the first device instructs the RNIC of the first device to establish the target QP connection based on the RC mode according to the first RDMA connection establishment request, and adds a mapping relationship between the newly created target QP connection and the virtual connection represented by the virtual connection identifier carried in the first RDMA connection establishment request in the first mapping relationship set.

[0141] Alternatively, when the intermediate component of the first device determines that the RNIC of the first device has established a target QP connection based on the RC mode, and the intermediate component of the first device also determines that the number of virtual connections in the first mapping relationship set that have a mapping relationship with the target QP connection is greater than a threshold, the intermediate component of the first device instructs the RNIC of the first device to establish a new target QP connection based on the RC mode according to the first RDMA connection establishment request, and adds a mapping relationship between the newly established target QP connection and the virtual connection represented by the virtual connection identifier carried in the first RDMA connection establishment request in the first mapping relationship set.

[0142] It can be understood that by controlling the number of virtual connections mapped to the same QP, that is, controlling the number of virtual connections that reuse the same QP connection, it is possible to prevent a large queuing delay when too many virtual connections reuse the same QP connection, thereby effectively ensuring the communication efficiency of the device.

[0143] Optionally, the process in which the intermediate component of the first device instructs the RNIC of the first device to establish a target QP connection based on the RC mode according to the first RDMA connection establishment request may be: the intermediate component of the first device sends a request to the second device to obtain the link establishment information of the RNIC of the second device according to the identifier of the second device carried in the first RDMA connection establishment request. The link establishment information of the RNIC of the second device includes but is not limited to: the GID of the RNIC of the second device and the QPN of a QP in the RNIC of the second device. In response, the first device receives the link establishment information returned by the second device. Further, the first device establishes the target QP connection according to the link establishment information of the RNIC of the first device and the link establishment information of the RNIC of the second device. The link establishment information of the RNIC of the first device includes but is not limited to: the GID of the RNIC of the first device and the QPN of a QP in the RNIC of the first device. Specifically, the first device can establish a mapping relationship between its own link establishment information and the link establishment information of the RNIC of the second device through the RNIC, thereby obtaining a QP connection, and the first device can also update the status information of the QP connection to an available connection (or a valid connection) through the RNIC, so that the first device completes the establishment of the target QP connection.

[0144] Optionally, when the first device sends a request to the second device to obtain the link establishment information of the second device, and receives the link establishment information returned by the second device, it can be implemented through the transmission control protocol (TCP) out-of-band connection communication between the first device and the second device, or through the RDMA communication manager (CM), which is not limited. Figure 2 As shown, the first device sends a request to obtain the link establishment information of the second device to the second device through the TCP out-of-band connection, and receives the link establishment information returned by the second device through the TCP out-of-band connection, so as to establish Figure 2 A QP connection between a first device and a second device is shown.

[0145] Furthermore, the intermediate component of the first device obtains metadata of the newly established target QP connection by the RNIC, establishes a mapping relationship between the target QP connection and the virtual connection represented by the virtual connection identifier carried in the first RDMA connection establishment request, and adds the mapping relationship to the first mapping relationship set.

[0146] Optionally, after updating the first mapping relationship set according to the first RDMA connection establishment request, the intermediate component of the first device also returns a response message indicating that the QP connection has been established to the application process.

[0147] Through the above process, the process of the intermediate component of the first device creating a target QP connection based on the first RDMA connection establishment request is implemented.

[0148] Optionally, when the intermediate component of the first device instructs the RNIC to establish a target QP connection based on the RC mode according to the first RDMA establishment connection request, in order to improve the efficiency of the first device in establishing the target QP connection, the first device's memory pre-stores link establishment information of multiple device RNICs, the link establishment information includes but is not limited to the GID of the device RNIC and the QPN of a QP in the device RNIC, and the link establishment information is used for the first device to establish a QP connection with the aforementioned multiple devices. The aforementioned multiple devices are devices that belong to the same communication system / network system as the first device, or are devices that can perform network communication with the first device, or are devices that may perform network communication with the first device, and there is no limitation on this.

[0149] In this way, when the above-mentioned multiple devices include the second device, it means that the link establishment information of the second device RNIC is pre-stored in the memory of the first device. Therefore, when the intermediate component of the first device instructs the RNIC to establish the target QP connection based on the RC mode according to the first RDMA connection establishment request, it does not need to obtain the link establishment information of the second device RNIC online (such as through TCP out-of-band connection communication or CM acquisition), but only needs to read the link establishment information of the second device RNIC from its own memory, and instruct the RNIC to establish the target QP connection based on the read link establishment information. In this way, the efficiency of the first device in establishing the QP connection can be greatly improved, thereby improving the communication efficiency of the first device.

[0150] In addition, the driver context of the RNIC of the first device is also pre-stored in the memory of the intermediate component of the first device. The memory of the intermediate component of the first device refers to a section of memory allocated by the first device to the intermediate component, and the intermediate component of the first device can access this section of memory. In this way, the driver context can be initialized when the intermediate component of the first device is started. Furthermore, when the intermediate component of the first device obtains the RDMA connection establishment request initiated by the application process / thread, it can instruct the RNIC to establish a QP connection for the application process / thread based on the initialized RNIC driver context. Compared with the current method of storing the RNIC driver context in the user space of each application process in the device, the solution provided in the embodiment of the present application can not only save the storage space for storing the RNIC driver context in the device, but also improve the efficiency of establishing a QP connection for the application process / thread. This is because: in the method in which the RNIC driver context is stored in the user space of each application process / thread in the device, each application process in the device must initialize the RNIC driver context pre-stored in its own user space when initiating an RDMA connection establishment request, and then further instruct the RNIC to establish a QP connection based on the initialized RNIC driver context. In the solution provided in the embodiment of the present application, the RNIC driver context only needs to be initialized once when the intermediate component of the device is started.

[0151] The "dynamic connection pool" is described in detail below.

[0152] A dynamic connection pool is configured in the memory of the first device intermediate component. The dynamic connection pool is configured with multiple waiting queues, and the waiting queues are used to store the pending RDMA data requests. Here, the detailed description of the pending RDMA data requests can refer to the above description, which will not be repeated here.

[0153] For multiple queues to be sent configured by the dynamic connection pool, different queues to be sent in the multiple queues to be sent are used to store pending RDMA data requests whose destination devices are different devices. In other words, the queues to be sent correspond one-to-one to the destination devices of the pending RDMA data requests. As an example, assume that the dynamic connection pool is configured with 10 queues to be sent, and the 10 queues to be sent are used to store pending RDMA data requests whose destination devices are 10 different devices. For example, queue 1 to be sent is used to store pending RDMA data requests whose destination device is device 1, queue 2 to be sent is used to store pending RDMA data requests whose destination device is device 2, ..., and queue 10 to be sent is used to store pending RDMA data requests whose destination device is device 10.

[0154] Optionally, the number of queues to be sent configured by the dynamic connection pool can generally be determined based on the number of devices that may communicate with the first device. Assuming that the number of devices that may communicate with the first device is s, the number of queues to be sent configured by the first device for the dynamic connection pool is greater than or equal to s.

[0155] The dynamic connection pool includes multiple DCT connections, and the multiple DCT connections are used to concurrently process the RDMA data requests stored in the multiple queues to be sent. For any one of the multiple DCT connections, such as the first DCT connection, the first DCT connection switches between multiple QP connections established based on the DCT mode, and keeps one QP connection as an available connection. Alternatively, the DCT connection can be understood as a fictitious connection concept, and the actual connection of the DCT connection is a QP connection whose status information is available (valid) among the multiple QP connections established based on the DCT mode.

[0156] Taking the case where the number of DCT connections in the dynamic connection pool is m (m is a positive integer) as an example, in a possible implementation, multiple QP connections established based on the DCT mode can be divided into m groups, so that each of the m DCT connections corresponds to a group of QP connections and can switch between the groups of QP connections.

[0157] As an example, assuming that the dynamic connection pool includes 10 DCT connections (respectively DCT connection 1 to DCT connection 10), and the intermediate component of the first device has established 200 QP connections based on the DCT mode, the intermediate component of the first device can divide the 200 QP connections into 10 groups, each group including 20 QP connections. Furthermore, each DCT connection in the dynamic connection pool corresponds to a group of QP connections, and switches between the groups of QP connections. For example, DCT connection 1 corresponds to the first group of QP connections, and switches between the first group of QP connections. DCT connection 3 corresponds to the third group of QP connections, and switches between the third group of QP connections, and so on.

[0158] Still taking the example that the number of DCT connections in the dynamic connection pool is m, in another possible implementation, the m DCT connections in the dynamic connection pool are switched between multiple QP connections in a polling manner, where each DCT connection can switch between j (j is a positive integer) QP connections.

[0159] As an example, assume that the dynamic connection pool includes 10 DCT connections (respectively DCT connection 1 to DCT connection 10), 200 QP connections are established in the intermediate component of the first device based on the DCT mode, and each DCT connection can switch between the 10 QP connections. When the 10 DCT connections in the dynamic connection pool switch between the 200 QP connections in a polling manner, the specific process is as follows: first, DCT connection 1 switches between the 1st to 10th QP connections in the 200 QP connections, DCT connection 2 switches between the 11th to 20th QP connections in the 200 QP connections, DCT connection 3 switches between the 21st to 30th QP connections in the 200 QP connections, ..., DCT connection 10 switches between the 91st to 100th QP connections in the 200 QP connections. After a period of time, DCT connection 1 switches between the 101st to 110th QP connections among the 200 QP connections, DCT connection 2 switches between the 111th to 120th QP connections among the 200 QP connections, DCT connection 3 switches between the 121st to 130th QP connections among the 200 QP connections, ..., DCT connection 10 switches between the 191st to 200th QP connections among the 200 QP connections. This process is called the polling method.

[0160] It should be understood that the intermediate component of the first device pre-establishes multiple QP connections based on the DCT mode and pre-stores metadata of the multiple QP connections. Thus, each DCT connection in the above dynamic connection can quickly switch between the multiple QP connections, which can improve the efficiency of the first device in creating QP connections, thereby improving the communication efficiency.

[0161] Optionally, for multiple QP connections pre-established by the intermediate component of the first device based on the DCT mode, the destination devices of the multiple QP connections may include the same device or different devices, without limitation. For any DCT connection in the dynamic connection pool, such as the first DCT connection, in the multiple QP connections switched by the first DCT connection, the destination devices of the multiple QP connections may include the same device or different devices, without limitation. For example, the 10 QP connections switched by the first DCT connection include QP connection 1 and QP connection 2, and QP connection 1 and QP connection 2 are both used to connect the first device and the second device. Among them, QP connection 1 can be a QP connection between QP 11 in the RNIC of the first device and QP 21 in the RNIC of the second device, and QP connection 2 is a QP connection between QP 12 in the RNIC of the first device and QP 22 in the RNIC of the second device, without limitation.

[0162] In practice, among multiple QP connections switched through a DCT connection, the destination devices of the multiple QP connections are generally set to different devices, so that the devices connected to the DCT connection are different each time the QP connection is switched.

[0163] Optionally, for the first DCT connection in the dynamic connection pool, when the current available connection (i.e., valid connection) of the first DCT connection is the first QP connection, after the first device transmits data requested by a preset number of RDMA data requests through the first QP connection, the first DCT connection is switched from the currently available first QP connection to the second QP connection, that is, the first DCT connection sets the second QP connection as an available connection. The destination devices of the first QP connection and the second QP connection may be the same or different, and this is not limited.

[0164] Optionally, each DCT connection in the dynamic connection pool may be periodically switched between multiple QP connections established based on the DCT mode with a preset duration as a period.

[0165] It should be understood that since the number of destination devices for which the application process / thread initiates RDMA data requests is generally greater than the number of DCT connections in the dynamic connection pool, the number of queues to be sent storing RDMA data requests to be processed will also be greater than the number of DCT connections in the dynamic connection pool. In this case, when all DCT connections in the dynamic connection pool respond to RDMA data requests in the queue to be sent, a portion of the queues to be sent storing RDMA data requests will be in a waiting state. Since the DCT connections in the dynamic connection pool in the embodiment of the present application can be switched among multiple QP connections, and when the destination devices of the multiple QP connections are different, the first device can process the RDMA data requests in the queues to be sent corresponding to different destination devices through the QP connections between different destination devices, which can avoid the first device from continuously processing the RDMA data requests in the corresponding queues to be sent through the QP connection corresponding to a certain destination device, which makes the RDMA data requests in the queues to be sent corresponding to other destination devices wait for too long. Therefore, after processing a preset number of RDMA data requests through the currently available connection of the DCT connection, the DCT connection switches the currently available connection to a QP connection whose destination device is different from the destination device of the currently available connection, thereby achieving the effect of balanced processing of RDMA data requests whose destination devices are different devices.

[0166] In addition, when multiple DCT connections in the dynamic connection pool concurrently process RDMA data requests, when the number of QP connections pre-established based on the DCT mode is large, such as greater than 256, the embodiment of the present application can achieve an overall throughput improvement of more than 30-40% in processing RDMA data requests.

[0167] Based on the above description, when the intermediate component of the first device obtains the RDMA data request indicating the transmission of the first stream data, refer to Figure 3 , Figure 3 A flow chart of another communication method provided by an embodiment of the present application is shown. After establishing the first corresponding relationship and establishing the target QP connection in the above step S104, the method further includes the following steps:

[0168] S201. Obtain a first RDMA data request.

[0169] The first RDMA data request indicates transmission of the data to be transmitted, and the data to be transmitted is data of the first stream. The data to be transmitted may also be referred to as data corresponding to the first RDMA data request.

[0170] The first RDMA data request may be a request for executing a data sending task, or a request for executing a data receiving task, without limitation. Wherein, "a request for executing a data sending task" and "a request for executing a data receiving task" may be understood as a work request (WR). Exemplarily, the first RDMA data request is an RDMA data request for executing an RDMAsend / write / read operation, without limitation thereto.

[0171] Optionally, after acquiring the first RDMA data request, the intermediate component of the first host may determine, according to the process / thread ID carried in the first RDMA data request, that the data corresponding to the first RDMA data request is the data of the first stream.

[0172] Optionally, the first device may obtain the first RDMA data request initiated by the first application process / thread in the first device through an intermediate component. The first application process / thread is the application process / thread that initiates the first RDMA connection establishment request mentioned above.

[0173] Optionally, the first application process / thread in the first device may initiate a first RDMA data request by calling a verbs API interface from a front-end component. The verbs API interface calls a UDS interface or a shared memory SHM interface. Thus, the intermediate component of the first device may receive the first RDMA data request initiated by the first application process / thread in the first device through the UDS interface or the SHM interface. The verbs API interface may carry the first RDMA data request.

[0174] In a possible implementation, a first interface library is pre-installed in the front-end component of the first device. The detailed description of the first interface library can be referred to the above description and will not be repeated here. Therefore, when the first application process / thread of the first device needs to exchange data with other devices, it calls the verbs API interface from the first interface library of the front-end component as needed, thereby achieving the purpose of sending the first RDMA data request from the first application process / thread to the middle component of the first device.

[0175] In another possible implementation, a second interface library is pre-installed in the front-end component of the first device. The detailed description of the second interface library can be referred to the above description and will not be repeated here. Therefore, when the application process of the first device needs to exchange data with other devices, it calls the standard verbs API interface from the second interface library of the front-end component as needed, and modifies the called standard verbs API interface so that the standard verbs API interface further calls the verbs API interface of the UDS interface or the SHM interface, thereby achieving the purpose of sending the first RDMA data request from the first application process / thread to the first device intermediate component based on the modified standard verbs API interface.

[0176] As an example, the first application process / thread in the first device initiates an RDMA data request by calling the ibv_post_send() interface (or ibv_post_recv() interface, etc.) from the front-end component. The ibv_post_send() interface is a verbs API interface for requesting to perform a send operation, and the ibv_post_recv() interface is a verbs API interface for requesting to perform a receive operation. Correspondingly, the ibv_post_send() interface is used to carry the RDMA data request for requesting to perform a send operation, and ibv_post_recv() is used to carry the RDMA data request for requesting to perform a receive operation. The ibv_post_send() interface (or ibv_post_recv() interface, etc.) also calls a UDS interface or a SHM interface, so that the intermediate component of the first device can receive the RDMA data request for requesting to perform a send operation (or requesting to perform a receive operation) initiated by the first application process / thread in the first device through the UDS interface or the SMH interface.

[0177] In addition, the first RDMA data request carries an identifier representing a first virtual connection, where the first virtual connection is a virtual connection set by the first device for the first application process / thread.

[0178] S202: Determine, according to the first RDMA data request, a target connection mode of a target QP connection for transmitting data to be transmitted.

[0179] Since the first RDMA data request indicates that the data to be transmitted is data of the first stream, the intermediate component of the first host may first determine the target connection mode of the target QP connection according to the pre-recorded first corresponding relationship.

[0180] In a possible case, when the first corresponding relationship is a corresponding relationship between the first stream and the RC mode, the intermediate component of the first host may determine that the target connection mode of the target QP connection is the RC mode.

[0181] In another possible case, when the first corresponding relationship is a corresponding relationship between the first stream and the DCT mode, the intermediate component of the first host may determine that the target connection mode of the target QP connection is the DCT mode.

[0182] S203: Transmit the data to be transmitted indicated by the first RDMA data request through the target QP connection established based on the target connection mode.

[0183] In one possible scenario, after the intermediate component of the first host determines that the target connection mode of the target QP connection is the RC mode, the intermediate component of the first host writes the first RDMA data request into the first virtual queue, wherein the first virtual queue is a virtual queue set for the first virtual connection, and the first virtual connection is a virtual connection represented by the virtual connection identifier carried by the first RDMA data request.

[0184] In this way, when the intermediate component of the first device schedules the first RDMA data request from the first virtual queue, the data to be transmitted indicated by the first RDMA data request is transmitted through the target QP connection having a mapping relationship with the first virtual connection in the first mapping relationship set. The detailed description of transmitting the data to be transmitted indicated by the first RDMA data request through the target QP connection can be referred to the following description, which will not be repeated here.

[0185] In order to improve the quality of service (QoS) of the business, the first device can give priority to RDMA data requests initiated by application processes / threads with high QoS levels, and post-process RDAM data requests initiated by application processes / threads with low QoS levels. Since the embodiment of the present application is provided with a virtual connection for each application process / thread that initiates an RDAM data request, and each virtual connection is provided with a virtual queue. Therefore, the application process / thread corresponds to the virtual queue one-to-one. Therefore, the intermediate component of the first device can set a priority for the virtual queue corresponding to the application process / thread based on the priority of the application process / thread, thereby dividing the virtual queue into different priority queues.

[0186] In this way, for any QP connection in the first mapping relationship set, such as the target QP connection mentioned above, after setting the priority for each virtual queue in the multiple virtual queues corresponding to the target QP connection, the multiple virtual queues are divided into at least one priority queue. If the at least one priority queue is referred to as the first group of priority queues, the target QP connection corresponds to the first group of priority queues. In this case, the multiple QP connections in the first mapping relationship set correspond to multiple groups of priority queues, and the multiple groups of priority queues include the first group of priority queues. Or it can be understood that the first device includes the aforementioned multiple groups of priority queues. Among them, the multiple virtual queues corresponding to the target QP connection refer to multiple virtual queues set for multiple virtual connections that have a mapping relationship with the target QP connection. The priority of each priority queue in the first group of priority queues is used to indicate the priority of the RDMA data request for scheduling each priority queue in the first group of priority queues. It should be understood that in the first group of priority queues, a priority queue can include at least one virtual queue, such as including two virtual queues. In this case, it means that the priorities of the two virtual queues are the same and are both the priorities of the priority queue, and there is no limitation on this.

[0187] In this case, the intermediate component of the first device can add the first RDAM data request initiated by the first application process / thread to the first priority queue according to the priority of the first application process / thread after determining that the target connection mode of the target QP connection is the RC mode. The first priority queue is one of the first group of priority queues mentioned above, and the priority of the first application process / thread corresponds to the priority of the first priority queue. It can be understood that adding the first RDAM data request to the first priority queue is actually adding the first RDAM data request to the first virtual queue whose priority is set to the priority of the first priority queue.

[0188] Furthermore, the intermediate component of the first device schedules the first RDMA data request according to the priority of the first priority queue, and when the first RDMA data request is scheduled, transmits the to-be-transmitted data of the first RDMA data request through the target QP connection.

[0189] In another possible situation, when the intermediate component of the first host determines that the target connection mode of the target QP connection is the DCT mode, the intermediate component of the first host writes the first RDMA data request into the queue to be sent corresponding to the second device (i.e., the queue to be sent for storing the destination device being the second device). The detailed description of the queue to be sent can be referred to above and will not be repeated here.

[0190] In this way, when the available connection of any DCT connection in the dynamic connection pool is switched to the QP connection connecting the first device and the second device, the RDMA data requests in the to-be-sent queue corresponding to the second device are processed in sequence by the QP connection, so as to transmit the to-be-transmitted data indicated by the first RDMA data request through the QP connection. The detailed description of "transmitting the to-be-transmitted data indicated by the first RDMA data request through the QP connection" can refer to the description of "transmitting the to-be-transmitted data indicated by the first RDMA data request through the target QP connection" below, which will not be repeated here.

[0191] The process of "transmitting the data to be transmitted indicated by the first RDMA data request through the target QP connection" is briefly introduced below.

[0192] Specifically, the intermediate component of the first device sends a first RDMA data request to the RNIC of the first device by calling the second verbs API interface, so that the RNIC of the first device transmits the data to be transmitted indicated by the first RDMA data request through the target QP connection. The second verbs API interface is a standard verbs API interface for calling QP connection, which is not described in detail here. It should be understood that when the second verbs API interface calls the target QP connection, the second verbs API interface includes an identifier of the target QP connection, which is, for example, the QPN of the QP used by the target QP connection in the first device, but is not limited to this. It should be understood that the target QP connection is a connection between the QP in the first device and the QP in the second device, and the QP in the first device is the QP used by the target QP connection in the first device.

[0193] In response, the RNIC of the first device writes a target WQE to the QP used by the target QP connection in the first device according to the identifier of the target QP connection carried in the second verbs API interface, and the target WQE is the WQE whose work request (WR) is the first RDMA data request. Then, the RNIC of the first device processes the WQEs in the QP in sequence, and when processing the target WQE, performs a corresponding RDMA operation based on the indication of the target WQE to realize the transmission of the first stream data indicated by the first RDMA data request.

[0194] Exemplarily, assuming that the target QP connection is a connection between a first QP in the first device and a second QP in the second device, when the first RDMA data request is an RDMA data request for requesting to perform a send operation, the RNIC of the first device responds to the operation of "the intermediate component of the first device sends the first RDMA data request to the RNIC of the first device by calling the second verbs API interface", and writes a work request (WR) to the first QP as the first WQE for requesting to perform a send operation according to the identifier of the target QP connection carried in the second verbs API interface. The RNIC of the first device processes the WQEs in the first QP in sequence, and when processing the first WQE, performs a send operation based on the instruction of the first WQE. Specifically, the RNIC of the first device obtains the data to be sent as indicated by the send operation, and sends the obtained data through the target QP connection, thereby realizing the transmission of the data to be sent as indicated by the send operation.

[0195] Through the communication method described in S101 to S104 and S201 to S203, the purpose of transmitting the data stream by selecting a QP connection established based on a suitable connection mode based on the traffic mode of the data stream can be achieved. In this way, the communication delay caused by the mismatch between the traffic mode of the data stream and the connection mode of establishing the QP connection can be avoided, thereby improving the communication efficiency.

[0196] In addition, it can be seen from the above that when the first RDMA data request indicates that the data to be transmitted is the data of the first stream, the first RDMA data request carries an identifier representing the first virtual connection, and the first corresponding relationship is the corresponding relationship between the first stream and the RC mode, the intermediate component of the first device transmits the data to be transmitted indicated by the first RDMA data request through the target QP connection that has a mapping relationship with the first virtual connection in the first mapping relationship set. Since in the first mapping relationship set, in addition to the first virtual connection, other virtual connections may also be included that have a mapping relationship with the target QP connection, for example, the first mapping relationship set also includes a mapping relationship between the second virtual connection and the target QP connection, and the second virtual connection is a virtual connection set by the first device for the second application process / thread. In this case, the data transmitted indicated by the RDMA data request initiated by the second application process / thread is also transmitted through the target QP connection. That is, the first application process / thread and the second application process / thread reuse the target QP connection for data transmission, and it can be seen that the method provided in the embodiment of the present application can improve the scalability performance of RDMA.

[0197] Specifically, when the traffic mode of the second stream initiated by the second application process / thread is a long-stable flow mode, and the destination device of the second stream is the second device, the first device can establish a mapping relationship between the second virtual connection and the target QP connection in the first mapping relationship set based on the method described in S101 to S104. In this way, when the intermediate component of the first device obtains the second RDMA data request initiated by the second application process / thread and the data to be transmitted is the second stream data, it can determine to transmit the data of the second stream through the target QP connection established in the RC mode. In this way, the intermediate component of the first device can obtain the data of the second stream transmitted as indicated by the second RDMA data request, and transmit the data of the second stream through the target QP connection established in the RC mode. For detailed descriptions, please refer to the description of determining and transmitting the data to be transmitted through the target QP connection established in the RC mode according to the first RDMA data request in S202 to S203, which will not be repeated.

[0198] In other embodiments, when the number of RDMA data requests is large (for example, the number of concurrent RDMA data requests in the first device is large, and / or the number of application processes / threads of concurrent RDMA data requests is also large), and a large number of RDMA data requests indicate that the data to be transmitted are all transmitted by default through the QP connection established based on the RC mode, or a large number of RDMA data requests indicate that the data streams to be transmitted are all long steady streams, therefore, these RDMA data requests indicate that the data to be transmitted are all required to be transmitted through the QP connection established based on the RC mode.

[0199] In this case, for each RDMA data request in the large number of RDMA data requests, in step S203, the intermediate component of the first device needs to write each RDMA data request into the corresponding virtual queue to wait for scheduling processing. The virtual queue corresponding to each RDMA data request refers to the virtual queue set for the virtual connection corresponding to each RDMA data request, and the virtual connection corresponding to each RDMA data request refers to the virtual connection represented by the virtual connection identifier carried in each RDMA data request.

[0200] For example, there are currently 10 concurrent RDMA data requests in the first device, namely, request 1, request 2, ..., and request 10. Request 1 carries an identifier representing virtual connection 1, request 2 carries an identifier representing virtual connection 2, ..., and request 10 carries an identifier representing virtual connection 10. In this case, the intermediate component of the first device can write request 1 into the virtual queue set for virtual connection 1, write request 2 into the virtual queue set for virtual connection 2, ..., and write request 10 into the virtual queue set for virtual connection 10.

[0201] Then, the scheduling thread in the intermediate component of the first device schedules the RDMA data requests stored in multiple virtual queues, and when scheduled to a certain virtual queue, such as an RDMA data request in a virtual queue set for the third virtual connection, the intermediate component of the first device transmits the data indicated by the RDMA data request through the QP connection that has a mapping relationship with the third virtual connection in the first mapping relationship set.

[0202] In a possible implementation manner, the first device may set a scheduling thread for each QP connection in the first mapping relationship set.

[0203] Specifically, since each QP connection in the first mapping relationship set is mapped to at least one virtual connection, and each virtual connection is provided with a virtual queue, each QP connection in the first mapping relationship set corresponds to at least one virtual queue, and the at least one virtual queue corresponds to the aforementioned at least one virtual connection in a one-to-one manner. For the sake of simple description, at least one virtual queue corresponding to a QP connection in the first mapping relationship set is referred to as a group of virtual queues corresponding to the QP connection.

[0204] In this case, the RDMA data requests stored in a group of virtual queues corresponding to any QP connection in the first mapping relationship set can be scheduled by one scheduling thread. Furthermore, the RDMA data requests stored in multiple groups of virtual queues corresponding to multiple QP connections in the first mapping relationship set can be scheduled by multiple scheduling threads. In this way, when the multiple scheduling threads concurrently schedule the RDMA data requests stored in multiple groups of virtual queues, the efficiency of the first device intermediate component in scheduling and processing the RDMA data requests can be improved, thereby avoiding the problem of head-of-line blocking.

[0205] As an example, when the first mapping relationship set maintained by the first device intermediate component includes mapping relationships between multiple virtual connections and three QP connections, and the three QP connections are QP connection 1, QP connection 2, and QP connection 3, the first device intermediate component sets three scheduling threads for the three QP connections, and the three scheduling threads are scheduling thread 1, scheduling thread 2, and scheduling thread 3. Among them, scheduling thread 1 is used to schedule RDMA data requests stored in a group of virtual queues corresponding to QP connection 1, scheduling thread 2 is used to schedule RDMA data requests stored in a group of virtual queues corresponding to QP connection 2, and scheduling thread 3 is used to schedule RDMA data requests stored in a group of virtual connection queues corresponding to QP connection 3. In this way, when scheduling thread 1, scheduling thread 2, and scheduling thread 3 run concurrently, concurrent scheduling processing of RDMA data requests in the aforementioned three groups of virtual queues can be realized, thereby improving the scheduling processing efficiency of RDMA data requests.

[0206] Optionally, for any QP connection, such as the third QP connection, when a group of virtual queues corresponding to the third QP connection is the first group of virtual queues, and the scheduling thread set by the first device intermediate component for the third QP connection is the first scheduling thread, in one possible case, if each virtual queue in the first group of virtual queues is preset with a priority, the first scheduling thread schedules the RDMA data requests stored in the first group of virtual queues according to the priority of each virtual queue in the first group of virtual queues. For at least two virtual queues in the first group of virtual queues with the same priority, the first scheduling thread can schedule the RDMA data requests stored in the at least two virtual queues based on a preset policy. In another possible case, if each virtual queue in the first group of virtual queues is not set with a priority, the first scheduling thread can schedule the RDMA data requests stored in the first group of virtual queues based on a preset policy.

[0207] Among them, the preset strategy is, for example, round-robin scheduling, but is certainly not limited thereto.

[0208] In another possible implementation, the first device may set a scheduling thread for multiple QP connections in the first mapping relationship set. The multiple QP connections may be any multiple QP connections in the first mapping relationship set, or all QP connections in the first mapping relationship set, without limitation. In this way, RDMA data requests stored in multiple groups of virtual queues corresponding to the multiple QP connections may be scheduled and processed by one scheduling thread.

[0209] Optionally, for multiple groups of virtual queues corresponding to multiple QP connections, the intermediate component of the first device may schedule the RDMA data requests stored in the multiple groups of virtual queues in a sequential polling manner through a scheduling thread.

[0210] For example, for the three groups of virtual queues corresponding to the three QP connections in the first mapping relationship set, assuming that the three groups of virtual queues are virtual queue group 1, virtual queue group 2 and virtual queue group 3, the intermediate component of the first device can use a scheduling thread to first schedule the RDMA data requests stored in virtual queue group 1, then schedule the RDMA data requests stored in virtual queue group 2, and finally schedule the RDMA data requests stored in virtual queue group 3, and execute the scheduling process in a loop.

[0211] Optionally, for multiple groups of virtual queues corresponding to multiple QP connections, the intermediate component of the first device may schedule RDMA data requests stored in the multiple groups of virtual queues according to the priorities of the multiple QP connections through a scheduling thread.

[0212] The priority of the QP connection may be preset.

[0213] Alternatively, the priority of the QP connection can be determined based on the priorities of multiple virtual queues corresponding to the QP connection. Here, the detailed description of the priority of the virtual queue can refer to the above description and will not be repeated. As an example, assume that the priorities of the three virtual queues corresponding to QP connection 1 are priority 1, priority 2, and priority 2, respectively, and the priorities of the two virtual queues corresponding to QP connection 2 are priority 2 and priority 3, respectively. Among them, priority 1 is higher than priority 2, and priority 2 is higher than priority 3. It can be seen that the priorities of the three virtual queues corresponding to QP connection 1 are higher than the priorities of the two virtual queues corresponding to QP connection 2 as a whole, so the priority of QP connection 1 is higher than the priority of QP connection 2.

[0214] Alternatively, for the QP connections in the first mapping relationship set, the intermediate component of the first device determines the priority of the first device RNIC when scheduling these QP connections as the priority of the QP connections in the first mapping relationship set. For example, the first mapping relationship set includes 3 QP connections, namely QP connection 1, QP connection 2, and QP connection 3. When the RNIC of the first device schedules these 3 QP connections, the priority of QP connection 3>the priority of QP connection 2>the priority of QP connection 1. Therefore, the intermediate component of the first device can determine the priorities of the 3 QP connections as follows: the priority of QP connection 3>the priority of QP connection 2>the priority of QP connection 1.

[0215] In this case, based on the priorities of the above-mentioned multiple QP connections, the intermediate component of the first device preferentially schedules the RDMA data requests stored in the virtual queue corresponding to the high-priority QP connection, and then schedules the RDMA data requests stored in the virtual queue corresponding to the low-priority QP connection.

[0216] As an example, when the first mapping relationship set includes three QP connections, namely QP connection 1, QP connection 2 and QP connection 3, and the intermediate component of the first device determines the priorities of the three QP connections as follows: the priority of QP connection 3> the priority of QP connection 2> the priority of QP connection 1. Then, when the intermediate component of the first host schedules the three groups of virtual connections corresponding to the three QP connections through a scheduling thread, it first schedules a group of virtual queues corresponding to QP connection 3, then schedules a group of virtual queues corresponding to QP connection 2, and finally schedules a group of virtual queues corresponding to QP connection 1, and repeats the schedule.

[0217] It should be understood that when the RDMA data requests stored in multiple groups of virtual queues corresponding to multiple QP connections are scheduled by a scheduling thread, such as a first scheduling thread, for any QP connection among the multiple QP connections, such as a third QP connection, when a group of virtual queues corresponding to the third QP connection is a first group of virtual queues, when the first scheduling thread schedules the RDMA data requests stored in the first group of virtual queues, the RDMA data requests stored in each virtual queue in the first group of virtual queues can be scheduled in a round-robin manner, or, according to the priority of each virtual queue in the first group of virtual queues, the RDMA data requests stored in each virtual queue in the first group of virtual queues can be scheduled, without limitation thereto. The priority of the virtual queue can refer to the above description and will not be repeated.

[0218] In this way, by scheduling multiple groups of virtual queues corresponding to multiple QP connections through one scheduling thread, resource consumption of the first device can be saved.

[0219] In some other embodiments, reference Figure 4 , Figure 4 A flow chart of another communication method provided in an embodiment of the present application is shown.

[0220] like Figure 4 As shown, after the first device completes S203 to transmit the data of the operation requested by the first RDMA data request through the target QP connection, the first device further performs the following steps.

[0221] S301: A first device obtains a target CQE.

[0222] It should be understood that one or more CQs are established and maintained in the RNIC of the first device. When a target CQ in the one or more CQs is associated with a target QP, the RNIC of the first device will write a CQE indicating that the WQE has been processed into the target CQ each time it processes a WQE in the target QP. The target QP is a QP used in the first device in a QP connection (such as the target QP connection described above) for transmitting data corresponding to the first RDMA data request.

[0223] Therefore, after the RNIC of the first device completes processing the target WQE corresponding to the first RDMA data request in the target QP, that is, after the first device completes processing the first RDMA data request, a target CQE indicating that the target WQE or the first RDMA data request has been processed is generated, and the target CQE is written into the target CQ. The target WQE corresponding to the first RDMA data request refers to the WQE whose work request (WR) is the first RDMA data request.

[0224] As an example, assuming that the first RDMA data request is an RDMA data request for requesting to perform a send operation, when the RNIC of the first device sends the data to be transmitted to the destination device of the first RDMA data request, that is, the second device, according to the instruction of the WQE corresponding to the first RDMA data request, and receives an ACK message returned by the second device indicating that the data to be transmitted has been received, the RNIC of the first device can determine that the WQE corresponding to the first RDMA data request has been processed, that is, the first RDMA data request has been processed. At this time, the RNIC of the first device generates a target CQE and writes the target CQE to the target CQ.

[0225] The target WQE includes a first work request identifier (WR-ID) indicating the first RDMA data request. Optionally, the first WR-ID may be added to the target WQE when the RNIC of the first device writes the target WQE to the target QP. For the relevant description of the RNIC of the first device writing the target WQE to the target QP, reference may be made to the relevant description of "writing the target WQE to the QP connected to the target QP used in the first device" above, and the detailed description of the first device generating the first WR-ID may refer to the relevant description of S302 below, which will not be repeated here. The WR-ID may be any number and / or character string, etc., or the WR-ID may be represented by the process / thread ID of the application process / thread corresponding to the WR-ID, which is not limited thereto. The application process / thread corresponding to the WR-ID refers to: the application process / thread that initiates the RDMA data request indicated by the WR-ID. Exemplarily, the first WR-ID indicating the first RDMA data request is represented by the process / thread ID of the first application process / thread that initiates the first RDMA data request.

[0226] In this case, after the RNIC of the first device completes processing the target WQE in the target QP, the generated target CQE carries the first WR-ID indicating the first RDMA data request.

[0227] Furthermore, the intermediate component of the first device may read the target CQE from the target CQ of the RNIC.

[0228] S302: The first device returns a target CQE to the first application process / thread that initiates the first RDMA data request.

[0229] Specifically, the intermediate component of the first device first determines the first application process / thread that initiates the first RDMA data request according to the acquired target CQE.

[0230] In a possible implementation, the first WR-ID is represented by the process / thread ID of the first application process / thread that initiates the first RDMA data request. In this case, after the intermediate component of the first device parses the first WR-ID from the acquired target CQE, it determines that the application process / thread identified by the first WR-ID is the first application process / thread that initiates the first RDMA data request.

[0231] In another possible implementation, a second mapping relationship set is configured in the first device, and the second mapping relationship set includes a mapping relationship between an application process / thread that initiates each RDMA data request and a WR-ID indicating each RDMA data request. Optionally, an application process / thread can initiate multiple RDMA data requests, each RDMA data request is indicated by a WR-ID, so an application process / thread can correspond to multiple WR-IDs indicating multiple RDMA data requests.

[0232] As an example, the second mapping relationship set can be specifically implemented as a second mapping relationship table maintained by the first device intermediate component. Referring to Table 3, Table 3 exemplarily shows a second mapping relationship table. When the RDMA data request initiated by process 1 includes request 1 and request 2, the RDMA data request initiated by process 2 includes request 3 and request 4, and the RDMA data request initiated by process 3 includes request 5, and the WR-ID indicating request 1 is ID 1, the WR-ID indicating request 2 is ID 2, the WR-ID indicating request 3 is ID 3, the WR-ID indicating request 4 is ID 4, and the WR-ID indicating request 5 is ID5, then the second mapping relationship table is as shown in Table 3:

[0233] Table 3

[0234]

[0235] It should be understood that the second mapping relationship set is maintained and updated by the intermediate component of the first device. Specifically, each time the intermediate component of the first device obtains an RDMA data request from an application process / thread, it generates a WR-ID indicating the RDMA data request for the RDMA data request, and adds the mapping relationship between the application process / thread and the WR-ID to the second mapping relationship set.

[0236] It should be understood that in S301, the first WR-ID added to the target WQE when the RNIC of the first device writes the target WQE to the target QP may be generated by the intermediate component of the first device when updating the second mapping relationship set according to the first RDMA data request, or may be generated by the RNIC of the first device when the RNIC of the first device writes the target WQE to the target QP, and this is not limited. It should be noted that when the first WR-ID added to the target WQE when the RNIC of the first device writes the target WQE to the target QP is generated by the RNIC of the first device, then the strategy of generating the first WR-ID when the RNIC of the first device writes the target WQE to the target QP is the same as the strategy of generating the first WR-ID indicating the first RDMA data request when the intermediate component of the first device updates the second mapping relationship set, so that based on the same strategy, the first WR-ID indicating the first RDMA data request generated by the RNIC of the first device and the intermediate component of the first device are the same. The embodiment of the present application does not specifically limit this strategy.

[0237] Furthermore, after the intermediate component of the first device parses the first WR-ID from the acquired target CQE, it can query the second mapping relationship set based on the first WR-ID, so as to determine the application process / thread that has a mapping relationship with the first WR-ID, and the application process / thread is the first application process / thread that initiates the first RDMA data request.

[0238] Next, after determining the first application process / thread that initiated the first RDMA data request, the intermediate component of the first device returns the target CQE to the first application process / thread. In this way, the first application process / thread can know that the first RDMA data request has been processed according to the target CQE.

[0239] In a possible implementation, the intermediate component of the first device sends the acquired target CQE to the initiating first application process / thread through the SHM communication method, thereby achieving the purpose of returning the target CQE to the first application process / thread. Specifically, the intermediate component of the first device writes the target CQE into the shared memory of the first application process / thread. In this way, the first application process / thread can read the target CQE from the shared memory, that is, the purpose of returning the target CQE to the first application process / thread is achieved.

[0240] In another possible implementation, the intermediate component of the first device maintains one or more CQE queues for each application process / thread. Taking the CQE queue maintained by the intermediate component of the first device for the first application process / thread that initiates the first RDMA data request as the first CQE queue as an example, after the intermediate component of the first device determines the first application process / thread that initiates the first RDMA data request according to the target CQE, it adds the target CQE obtained from the RNIC to the first CQE queue. In this way, the first application process / thread can read the target CQE from the first CQE queue by calling the standard verbs API interface of CQ (such as the ibv_poll_cq interface), thereby achieving the purpose of returning the target CQE to the first application process / thread.

[0241] pass Figure 4 The method described above can achieve the purpose of returning the CQE indicating that the RDMA data request has been processed to the application process / thread that initiated the RDMA data request.

[0242] In some other embodiments, through the logical structure of the first device provided by the embodiments of the present application and the verbs API interface provided by the embodiments of the present application, the first device can also perform corresponding processing for other RDMA control class requests except RDMA connection establishment requests, such as RDMA requests for requesting to establish QP / CQ, or RDMA requests for destroying QP connections.

[0243] Optionally, the application process / thread in the first device may initiate the above other RDMA control class requests by calling the first verbs API interface from the front-end component. The relevant description of the first verbs API interface can refer to the above description and will not be repeated here.

[0244] As an example, the application process in the first device initiates an RDMA control class request to establish a PD (or CQ, QP, etc.) by calling the ibv_alloc_pd interface (or ibv_create_cq interface, ibv_create_qp interface, etc.) from the front-end component. Among them, the ibv_alloc_pd interface is a verbsAPI interface for requesting to allocate a memory protection domain (PD), the ibv_create_cq interface is a verbs API interface for requesting to create a CQ, and the ibv_create_qp interface is a verbs API interface for requesting to create a QP. Accordingly, the ibv_alloc_pd interface is used to carry the RMDA control class request for requesting to allocate a PD, the ibv_create_cq interface is used to carry the RDMA control class request for requesting to create a CQ, and the ibv_create_qp interface is used to carry the RDMA control class request for requesting to create a QP. Among them, the UDS interface is also called in the ibv_alloc_pd interface (or ibv_create_cq interface, ibv_create_qp interface, etc.), so that the intermediate component of the first device can receive the RDMA control class request for requesting allocation of PD (or requesting to create CQ, requesting to create QP) initiated by the application process in the first device through the UDS interface.

[0245] In response, the intermediate component of the first device may receive the other RDMA control class request initiated by the application process / thread in the first device through the UDS interface or the SHM interface, and perform corresponding actions according to the instructions of the other RDMA control class request.

[0246] In one example, when the intermediate component of the first device obtains an RDMA control class request requesting the creation of a CQ, the intermediate component of the first device sends the RDMA control class request to the RNIC of the first device by calling a standard verbs API interface, so that the RNIC of the first device responds to the RDMA control class request and creates a CQ.

[0247] In another example, when the intermediate component of the first device obtains an RDMA control class request requesting the creation of a QP, the intermediate component of the first device sends the RDMA control class request to the RNIC of the first device by calling a standard verbs API interface, so that the RNIC of the first device creates a QP in response to the RDMA control class request.

[0248] In another example, when the intermediate component of the first device obtains a control class request indicating the destruction of a QP connection, the intermediate component of the first device can destroy the mapping relationship corresponding to the RDMA control class request in the first mapping relationship set according to the RDMA control class request. The mapping relationship corresponding to the RDMA control class request is specifically: the first mapping relationship set includes a mapping relationship of an identifier of a virtual connection carried by the RDMA control class request. Optionally, the intermediate component of the first device can delete the identifier of the virtual connection carried by the RDMA control class request in the first mapping relationship set according to the RDMA control class request, thereby destroying the mapping relationship corresponding to the RDMA control class request in the first mapping relationship set.

[0249] In this way, the embodiment of the present application manages the RDMA resources of the first device through the intermediate component of the first device. Since it supports the use of the verbs API interface to operate the RDMA resources of the first device, there is no need to make adaptive modifications to the application process / thread, that is, the embodiment of the present application has strong usability.

[0250] In summary, through the method described in the embodiment of the present application, it is possible to use a QP connection with a suitable connection mode for transmission for data streams with different traffic modes, thereby improving the communication efficiency on the basis of improving the scalability performance of RDMA connection resources. For example, for long and stable flows, the embodiment of the present application improves the scalability performance of RDMA by multiple application processes / threads reusing the same QP connection to transmit data, and concurrently schedules RDMA data requests stored in multiple groups of virtual queues corresponding to multiple QP connections through multiple scheduling threads, thereby improving the processing efficiency of RDMA data requests and improving communication performance. For another example, for short flows, the embodiment of the present application transmits data through a QP connection pre-established based on the DCT mode in a dynamic connection pool. Since the QP connection established based on the DCT mode has the characteristics of low switching delay and low switching cost, and the dynamic connection pool is pre-configured, the transmission of short flow data through the QP connection established based on the DCT mode in the dynamic connection pool can not only improve the scalability performance of RDMA connection resources, but also improve communication efficiency.

[0251] The above mainly introduces the solution provided in the embodiment of the present application from the perspective of method.

[0252] In order to achieve the above functions, Figure 5 As shown, Figure 5 FIG. 5 is a schematic diagram showing the structure of a communication device 50 provided in an embodiment of the present application. The communication device 50 is applied to a first device supporting the RDMA function and is used to execute the above communication method, for example, to execute Figure 1 , Figure 3 as well as Figure 4The communication device 50 may include an acquisition unit 51 , a processing unit 52 and a transmission unit 53 .

[0253] The acquisition unit 51 is used to acquire a first RDMA connection establishment request, where the first RDMA connection establishment request indicates to establish a target QP connection between the first device and the second device. The processing unit 52 is used to determine a traffic pattern of the first flow corresponding to the first RDMA connection establishment request, and to determine a target connection pattern corresponding to the target QP connection according to the traffic pattern. The target QP connection corresponds to the first flow. The transmission unit 53 is used to transmit data of the first flow through the target QP connection established based on the target connection pattern.

[0254] As an example, combining Figure 1 The acquisition unit 51 may be used to execute S101, the processing unit 52 may be used to execute S102 to S103, and the transmission unit 53 may be used to execute S104.

[0255] Optionally, when the above traffic mode is a long steady flow mode, the target connection mode is an RC mode.

[0256] Optionally, the first stream is a data stream initiated by the first application process / thread. The acquisition unit 51 is further used to acquire data of the second stream, where the second stream is a data stream initiated by the second application process / thread, the traffic mode of the second stream is a long-stable flow mode, and the destination device of the second stream is the second device. The transmission unit 53 is further used to transmit the data of the second stream through the target QP connection established based on the target connection mode.

[0257] Optionally, the first device includes multiple groups of priority queues, a first group of priority queues in the multiple groups of priority queues corresponds to a target QP connection, and the first group of priority queues includes at least one priority queue, the priority of the at least one priority queue is used to indicate the priority of scheduling the RDMA data request of each priority queue in the first group of priority queues. The processing unit 52 is also used to add the first RDMA data request initiated by the first application process / thread to the first priority queue according to the priority of the first application process / thread, and to schedule the first RDMA data request according to the priority of the first priority queue. The transmission unit 53 is specifically used to transmit the data corresponding to the first RDMA data request through the target QP connection when the first RDMA data request is scheduled. The priority of the first application process / thread corresponds to the priority of the first priority queue.

[0258] Optionally, when the above traffic mode is the short flow mode, the target connection mode is the DCT mode.

[0259] Optionally, the first RDMA connection establishment request carries a flow tag, which is used to indicate the flow mode of the first flow corresponding to the first RDMA connection establishment request. The processing unit 52 is specifically configured to determine the flow mode of the first flow according to the flow tag carried in the first RDMA connection establishment request.

[0260] As an example, combining Figure 1 , the processing unit 52 can be used to execute S102.

[0261] Optionally, the processing unit 52 is specifically configured to determine a flow pattern of the first flow according to flow characteristics of the first flow.

[0262] As an example, combining Figure 1 , the processing unit 52 can be used to execute S102.

[0263] Optionally, link establishment information of multiple devices is pre-stored in the first device, the link establishment information includes the GID of the RNIC in the device and the QPN of the QP in the RNIC, and the link establishment information is used by the first device to establish QP connections with the multiple devices.

[0264] Optionally, the intermediate component of the communication device 50 pre-stores the driver context of the RNIC in the first device. The processing unit 52 is configured to initialize the driver context before obtaining the first RDMA connection establishment request, and the initialized driver context is used to establish the QP connection.

[0265] Optionally, the first RDMA connection establishment request is an RDMA request initiated by an application process / thread in the first device by calling a first verbs API interface.

[0266] Optionally, the first verbs API calls a UDS interface or a SHM interface, and the acquiring unit 51 is specifically configured to receive, through the UDS interface or the SHM interface, a first RDMA connection establishment request initiated by an application process / thread in the first device.

[0267] As an example, combining Figure 1 , the acquisition unit 51 can be used to execute S101.

[0268] Optionally, the transmission unit 53 is specifically configured to send an RDMA data request indicating transmission of the data of the first stream to the RNIC of the first device by calling a second verbs API, so as to transmit the data of the first stream through the target QP connection.

[0269] As an example, combining Figure 3 , the transmission unit 53 can be used to execute S203.

[0270] For the detailed description of the above optional manner, please refer to the above method embodiment, which will not be repeated here. In addition, the explanation of any communication device 50 provided above and the description of the beneficial effects can refer to the above corresponding method embodiment, which will not be repeated here.

[0271] Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0272] It should be noted that Figure 5 The division of modules / units in the above is schematic and is only a logical function division. There may be other division methods in actual implementation. For example, two or more functions may be integrated into one processing module. The above integrated modules may be implemented in the form of hardware or software function modules.

[0273] The following describes the hardware structure and logical structure of the communication device by taking the communication device provided in the embodiment of the present application as a device supporting the RDMA function as an example.

[0274] refer to Figure 6 , Figure 6 FIG. 1 is a schematic diagram showing the hardware structure of a device supporting RDMA function provided by an embodiment of the present application. Figure 6 As shown, the device 60 includes a processor 601, a memory 602, a network interface 603, and a bus 604. The processor 601, the memory 602, and the network interface 603 are connected via the bus 604.

[0275] Processor 601 is the control center of device 60, which can be a general-purpose central processing unit (CPU). Processor 601 may also be other general-purpose processors, digital signal processors (digital signal processing, DSP), application-specific integrated circuits (application-specific integrated circuits, ASIC), field-programmable gate arrays (field-programmable gate arrays, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, graphics processing units (graphics processing units, GPU), neural processing units (neural processing units, NPU), tensor processing units (tensor processing units, TPU) or artificial intelligence (artificial intelligent) chips, data processors (data processing units, DPU), etc.

[0276] As an example, the processor 601 includes one or more CPUs, such as Figure 6 In addition, the present application does not limit the number of processor cores in each processor.

[0277] The memory 602 is used to store program instructions or data to be accessed by the application process. The processor 601 can implement the communication method provided in the embodiment of the present application by executing the program instructions in the memory 602.

[0278] The memory 602 includes a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM) and direct rambus RAM (DR RAM). Non-volatile memory can be storage class memory (SCM), solid state drive (SSD), hard disk drive (HDD), etc. Among them, storage class memory can be, for example, non-volatile memory (NVM), phase-change memory (PCM), persistent memory, etc.

[0279] In a possible implementation, the memory 602 exists independently of the processor 601. The memory 602 is connected to the processor 601 via a bus 604 and is used to store data, instructions or program codes. When the processor 601 calls and executes the instructions or program codes stored in the memory 602, the communication method provided in the embodiment of the present application can be implemented.

[0280] In another possible implementation, the memory 602 and the processor 601 are integrated together.

[0281] The network interface 603 is used for connecting the device 60 to other devices (such as the second device, etc.) through a communication network, and the communication network may be Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc. The network interface 603 includes a receiving unit for receiving data / messages and a sending unit for sending data / messages.

[0282] In the embodiment of the present application, the network interface 603 includes an RNIC. The RNIC may be an independent Peripheral Component Interconnect Express (PCIe) network card, or may be a network interface unit integrated into a system on chip (SoC), which is not limited.

[0283] RNIC is an external device of device 60. Generally, the processor 601 of device 60 can allocate a segment of memory for RNIC, and establish at least one QP on this segment of memory. That is, RNIC includes at least one QP. As an example, an RNIC may include 655360 QPs, but is not limited thereto. Taking any QP on RNIC, such as QP a, as an example, when QPa establishes a QP connection with the peer device of device 60, such as QP b in the second device, network communication between device 60 and the second device can be achieved. Among them, the QP connection includes a reliable connection and an unreliable connection. When the QP connection is a reliable connection, the QP connection can be established through the RC mode or through the DCT mode, and the comparison is not limited.

[0284] In addition, the RNIC can also provide RDMA services for application processes / threads in the device 60 by performing RDMA operations, and support one application process / thread to communicate with a process / thread in the opposite device using one QP connection.

[0285] The bus 604 may be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, a high-speed serial computer expansion bus (PCIe), a compute express link (CXL) or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0286] It should be pointed out that Figure 6 The structure shown in the figure does not constitute a limitation on the device 60, except Figure 6 In addition to the components shown, the device 60 includes Figure 6 More or fewer components may be shown, or certain components may be combined, or the components may be arranged differently.

[0287] As an example, combining Figure 6 The functions implemented by the acquisition unit 51 and the processing unit 52 in the communication device 50 described above can be realized by Figure 6 Processor 601 in the Figure 6 The functions implemented by the transmission unit 63 can be realized by Figure 6 Processor 601 in the Figure 6 The program code in the memory 602 and the network interface 603 are implemented.

[0288] refer to Figure 7 , Figure 7 FIG. 1 is a schematic diagram showing a logical structure of a device supporting RDMA function provided by an embodiment of the present application. Figure 7 As shown, application process 1, application process 2, application process 3, ..., and application process n, a total of n application processes, are running in device 70, where n is a positive integer. Device 70 also includes a front-end component 71, an intermediate component 72, and an RNIC 73.

[0289] Among them, the front-end component 71 is configured with a user state library application programming interface (libibverbs application programming interface, verbs API) provided by an embodiment of the present application, and then the application process in the device 70 communicates with the intermediate component 72 by calling the verbs API interface in the front-end component 71. Alternatively, the front-end component 71 is configured with a standard verbs API interface, and then the application process in the device 70 calls the standard verbs API interface in the front-end component 71, and modifies the called standard verbs API interface to achieve communication between the application process and the intermediate component 72. Optionally, the communication between the application process and the intermediate component 72 achieved by the front-end component 71 can be a Unix domain socket (unix domain socket, UDS) communication method, or a shared memory (share memory, SHM) communication method, but is not limited thereto. For detailed descriptions, please refer to the method description below, which will not be repeated here.

[0290] The intermediate component 72 can be implemented by a process running in the device 70, and the device 70 allocates memory to the intermediate component 72. In an embodiment of the present application, a first mapping relationship set and a dynamic connection pool are configured in the memory of the intermediate component 72. The first mapping relationship set includes multiple mapping relationships, each mapping relationship includes a mapping relationship between at least one virtual connection and a QP connection, and the QP connection is a QP connection established based on the RC mode. The dynamic connection pool includes multiple DCT connections, and the multiple DCT connections are used to concurrently process multiple RDMA data requests. For the first DCT connection in the multiple DCT connections, the first DCT connection switches between multiple QP connections and keeps one QP connection as an available connection (or called an effective connection). Here, the QP connection is a QP connection established based on the DCT mode. Furthermore, the device 70 can select the first mapping relationship set or the dynamic connection pool for corresponding processing for RDMA data requests of data streams with different traffic modes (such as long and stable streams, short streams, etc.). For detailed descriptions, please refer to the method description below, which will not be repeated here.

[0291] Optional, combined Figure 6 , the front-end component 71 and the middle component 72 in the device 70 can be operated by Figure 6 The program instructions of the processor 601 and the memory 602 in the device 60 are implemented together, and this is not limited.

[0292] The embodiment of the present application further provides a communication system, the communication system comprising a first device and a second device. The first device can achieve communication with the second device through the communication method described above.

[0293] The present application also provides a computer program product and a computer-readable storage medium for storing the computer program product. The computer program product may include one or more program instructions, which, when executed by one or more processors, can provide the above Figure 1 , Figure 3 or Figure 4 Thus, for example, reference to Figure 1 One or more features of S101 to S104 may be undertaken by one or more instructions in the computer program product.

[0294] In some examples, such as for executing Figure 1 , Figure 3 or Figure 4 The communication device of the method may be configured to provide various operations, functions, or actions in response to one or more program instructions stored in a computer-readable storage medium.

[0295] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When executing instructions on a computer and executing a computer, a process or function according to an embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. that contains one or more servers that can be integrated with a medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0296] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A communication method, characterized in that: Applied to a first device supporting a remote direct memory access (RDMA) function, the method comprising: Obtaining a first RDMA connection establishment request, where the first RDMA connection establishment request indicates establishing a target queue pair QP connection between the first device and the second device; Determine a traffic pattern of a first flow corresponding to the first RDMA connection establishment request; Determine, according to the traffic pattern, a target connection pattern corresponding to the target QP connection, wherein the target QP connection corresponds to the first flow; The data of the first stream is transmitted through the target QP connection established based on the target connection mode.

2. The method according to claim 1, characterized in that When the traffic mode is a long steady flow mode, the target connection mode is a reliable connection RC mode.

3. The method according to claim 2, characterized in that The first stream is a data stream initiated by a first application process / thread; the method further comprises: Acquire data of a second flow, where the second flow is a data flow initiated by a second application process / thread, a traffic mode of the second flow is the long-stable flow mode, and a destination device of the second flow is the second device; The data of the second stream is transmitted through the target QP connection established based on the target connection mode.

4. The method according to claim 2 or 3, characterized in that: The first device includes multiple groups of priority queues, a first group of priority queues in the multiple groups of priority queues corresponds to the target QP connection, the first group of priority queues includes at least one priority queue, and the priority of the at least one priority queue is used to indicate the priority of scheduling the RDMA data request of each priority queue in the first group of priority queues; The method further comprises: According to the priority of the first application process / thread, adding the first RDMA data request initiated by the first application process / thread to the first priority queue, wherein the priority of the first application process / thread corresponds to the priority of the first priority queue; The first RDMA data request is scheduled according to the priority of the first priority queue, and data corresponding to the first RDMA data request is transmitted through the target QP connection.

5. The method according to claim 1, characterized in that When the traffic mode is the short flow mode, the target connection mode is the dynamic connection transmission DCT mode.

6. The method according to any one of claims 1 to 5, characterized in that The first RDMA connection establishment request carries a flow tag, where the flow tag is used to indicate a traffic mode of the first flow corresponding to the first RDMA connection establishment request; Determining the traffic pattern of the first flow corresponding to the first RDMA connection establishment request includes: The traffic pattern of the first flow is determined according to the flow tag carried in the first RDMA connection establishment request.

7. The method according to any one of claims 1 to 5, characterized in that Determining the traffic pattern of the first flow corresponding to the first RDMA connection establishment request includes: A flow pattern of the first flow is determined according to the flow characteristics of the first flow.

8. The method according to any one of claims 1 to 7, characterized in that: The first device pre-stores link establishment information of multiple devices, the link establishment information including the global identifier GID of the network card RNIC supporting the RDMA function in the device and the queue pair number QPN of the QP in the RNIC, and the link establishment information is used by the first device to establish a QP connection with the multiple devices.

9. The method according to any one of claims 1 to 8, characterized in that: The method is performed by an intermediate component of the first device, wherein the intermediate component pre-stores a driver context of the RNIC in the first device; Before obtaining the first RDMA connection establishment request, the method further includes: The driver context is initialized, and the initialized driver context is used to establish a QP connection.

10. The method according to any one of claims 1 to 9, characterized in that The first RDMA connection establishment request is an RDMA request initiated by an application process / thread in the first device by calling a first user state library application programming interface verbs API.

11. The method according to claim 10, characterized in that The first verbs API interface calls a Unix domain socket UDS interface or a shared memory SHM interface; The obtaining of the first RDMA connection establishment request comprises: The first RDMA connection establishment request initiated by the application process / thread in the first device is received through the UDS interface or the SHM interface.

12. The method according to any one of claims 1 to 11, characterized in that The transmitting the data of the first stream through the target QP connection established based on the target connection mode includes: By calling the second verbs API interface, an RDMA data request indicating transmission of the data of the first stream is sent to the RNIC of the first device, so as to realize transmission of the data of the first stream through the target QP connection.

13. A communication device, characterized in that: A first device for supporting a remote direct memory access (RDMA) function, the device comprising: An acquiring unit, configured to acquire a first RDMA connection establishment request, where the first RDMA connection establishment request indicates establishing a target queue pair QP connection between the first device and the second device; a processing unit, configured to determine a traffic pattern of a first flow corresponding to the first RDMA connection establishment request, and determine a target connection pattern corresponding to the target QP connection according to the traffic pattern, wherein the target QP connection corresponds to the first flow; A transmission unit is configured to transmit data of the first stream through the target QP connection established based on the target connection mode.

14. The device according to claim 13, characterized in that When the traffic mode is a long steady flow mode, the target connection mode is a reliable connection RC mode.

15. The device according to claim 14, characterized in that The first stream is a data stream initiated by a first application process / thread; The acquisition unit is further used to acquire data of a second flow, where the second flow is a data flow initiated by a second application process / thread, the traffic mode of the second flow is the long-stable flow mode, and the destination device of the second flow is the second device; The transmission unit is further configured to transmit data of the second stream through the target QP connection established based on the target connection mode.

16. The device according to claim 14 or 15, characterized in that The first device includes multiple groups of priority queues, a first group of priority queues in the multiple groups of priority queues corresponds to the target QP connection, the first group of priority queues includes at least one priority queue, and the priority of the at least one priority queue is used to indicate the priority of scheduling the RDMA data request of each priority queue in the first group of priority queues; The processing unit is further configured to add a first RDMA data request initiated by the first application process / thread to a first priority queue according to a priority of the first application process / thread, wherein the priority of the first application process / thread corresponds to the priority of the first priority queue; and schedule the first RDMA data request according to the priority of the first priority queue; The transmission unit is specifically configured to transmit the data corresponding to the first RDMA data request through the target QP connection.

17. The device according to claim 13, characterized in that When the traffic mode is the short flow mode, the target connection mode is the dynamic connection transmission DCT mode.

18. The device according to any one of claims 13 to 17, characterized in that The first RDMA connection establishment request carries a flow tag, where the flow tag is used to indicate a traffic mode of the first flow corresponding to the first RDMA connection establishment request; The processing unit is specifically configured to determine a traffic pattern of the first flow according to the flow tag carried in the first RDMA connection establishment request.

19. The device according to any one of claims 13 to 17, characterized in that The processing unit is specifically configured to determine a flow pattern of the first flow according to a flow feature of the first flow.

20. The device according to any one of claims 13 to 19, characterized in that The first device pre-stores link establishment information of multiple devices, the link establishment information including the global identifier GID of the network card RNIC supporting the RDMA function in the device and the queue pair number QPN of the QP in the RNIC, and the link establishment information is used by the first device to establish a QP connection with the multiple devices.

21. The device according to any one of claims 13 to 20, characterized in that The intermediate component of the device pre-stores the driver context of the RNIC in the first device; The processing unit is used to initialize the driver context before obtaining the first RDMA connection establishment request, and the initialized driver context is used to establish the QP connection.

22. The device according to any one of claims 13 to 21, characterized in that The first RDMA connection establishment request is an RDMA request initiated by an application process / thread in the first device by calling a first user state library application programming interface verbs API.

23. The device according to claim 22, characterized in that The first verbs API call has a Unix domain socket UDS interface or a shared memory SHM interface; The acquisition unit is specifically configured to receive, through the UDS interface or the SHM interface, the first RDMA connection establishment request initiated by the application process / thread in the first device.

24. The device according to any one of claims 13 to 23, characterized in that The transmission unit is specifically configured to send an RDMA data request indicating transmission of the data of the first stream to the RNIC of the first device by calling a second verbs API, so as to transmit the data of the first stream through the target QP connection.

25. A communication device, characterized in that: include: One or more processors and a memory, wherein the one or more processors are configured to call program instructions stored in the memory to execute the method according to any one of claims 1 to 12.

26. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes program instructions. When the program instructions are executed on a computer or a processor, the computer or the processor is caused to perform the method according to any one of claims 1 to 12.

Images