RDMA-based data transmission

Abstract

Provided in the present disclosure are an RDMA-based data transmission method, and a device, a storage medium and a program product. During an RDMA connection establishment process, a local work queue used for same-host RDMA communication and a remote work queue used for cross-host RDMA communication are created at the same time, and a same-host recognition function is added. When same-host communication is recognized, the local work queue is used to directly execute a data copy operation in registered memories of two applications to replace a traditional RDMA transmission operation. Therefore, the compatibility with an RDMA programming interface and transparency or unawareness with regard to applications are achieved, the transmission delay between different RDMA applications on the same host can also be reduced, thereby reducing the consumption of a PCIe bandwidth, and ensuring the fairness in resource contention between local traffic and Internet traffic.

Description

Data transmission technology based on RDMA

[0001] The present disclosure relates to the field of cloud computing technology, and in particular to data transmission based on RDMA.

[0002] Remote Direct Memory Access (RDMA) is a technology that bypasses the kernel of a remote host's operating system to access data in its memory. This technology allows data to be transferred directly between user mode and network card hardware, effectively avoiding the context switching overhead associated with user and kernel mode, as well as the kernel's network protocol processing overhead. This improves communication performance while reducing CPU (central processing unit) overhead. Consequently, RDMA has been widely used in scenarios such as cloud storage, high-performance computing (HPC), and distributed systems.

[0003] With the rapid adoption of cloud-native containerization, different RDMA applications can be deployed in different containers on the same host. However, communication between RDMA applications in different containers on the same host has exposed the shortcomings of RDMA. RDMA network cards are typically connected to the CPU via the high-speed Peripheral Component Interconnect Express (PCIe) bus. Therefore, when RDMA applications in different containers on the same host communicate, data must also pass through two PCIe operations, which not only consumes PCIe bandwidth but also increases transmission latency.

[0004] Various aspects of the present disclosure provide an RDMA-based data transmission method, device, storage medium, and program product for reducing transmission latency between different RDMA applications on the same host and saving bandwidth resources while being compatible with an RDMA programming interface.

[0005] An embodiment of the present disclosure provides an RDMA-based data transmission method, which is applied to a physical machine on which a first application is deployed. The method includes: when the first application initiates an RDMA communication request to a second application, creating a first RDMA work queue for the first application and registering it in memory, where the first RDMA work queue includes a local work queue and a remote work queue; obtaining, based on a control plane connection between the first application and the second application, description information of a second RDMA work queue corresponding to the second application as context information of the first RDMA work queue; identifying, based on the description information of the second RDMA work queue, whether same-host RDMA communication is between the first application and the second application, where the second RDMA work queue also includes a local work queue and a remote work queue; and, if same-host RDMA communication is identified, performing a data copy operation in the memory registered by the two applications based on the local work queue of at least one of the two applications, to replace the RDMA data transmission operation between the two applications.

[0006] An embodiment of the present disclosure further provides a physical machine, comprising an RDMA network card, and having a first application and an RDMA management program deployed on the physical machine, wherein the RDMA management program comprises: a user-state interface management module and a kernel-state driver management module; the driver management module, in cooperation with the interface management module, is configured to: when a first application initiates an RDMA communication request to a second application, create a first RDMA work queue for the first application and register it in memory; obtain, based on a control plane connection between the first application and the second application, description information of a second RDMA work queue corresponding to the second application as context information of the first RDMA work queue; the first and second RDMA work queues both comprise a local work queue and a remote work queue; identify, based on the description information of the second RDMA work queue, whether same-host RDMA communication is between the first application and the second application; and, if same-host RDMA communication is identified, perform a data copy operation in the memory registered by the two applications based on the local work queue of at least one of the two applications, to replace the RDMA data transfer operation between the two applications.

[0007] An embodiment of the present disclosure further provides a physical machine, comprising: a memory and a processor; the memory is used to store a computer program, and the processor is coupled to the memory and is used to execute the computer program to implement the steps in the above method.

[0008] An embodiment of the present disclosure further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is enabled to implement the steps in the above method.

[0009] An embodiment of the present disclosure further provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the processor is enabled to implement the steps in the above method.

[0010] In the disclosed embodiments, improvements are made to the user-mode interface management module and kernel-mode driver management module within the RDMA hypervisor deployed on a physical machine. With the cooperation of the driver management module and the interface management module, a local work queue and a remote work queue are simultaneously created during the RDMA connection establishment process. Same-host identification is also added. When same-host communication is detected, the local work queue is used to directly perform data copy operations in the registered memory of the two applications, replacing traditional RDMA transfer operations. For applications, RDMA communication requests can be directly initiated, just like traditional RDMA communication. While being compatible with the RDMA programming interface and transparent or unaware to applications, this reduces transmission latency between different RDMA applications on the same host, reduces PCIe bandwidth consumption, and ensures fairness between local and network traffic during resource competition. Furthermore, when cross-host communication is detected, the remote work queue is used to perform traditional RDMA transfer operations, without affecting the cross-host RDMA communication process. (Drawing Description)

[0011] The drawings described herein are intended to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are intended to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:

[0012] FIG1 is a schematic diagram of an interconnection structure of physical machines and a data transmission process provided by an exemplary embodiment of the present disclosure;

[13] Figures 2a and 2b are schematic diagrams of the internal architecture of a physical machine provided by another exemplary embodiment of the present disclosure;

[0014] FIG3a is a schematic structural diagram of an interface management module provided by another exemplary embodiment of the present disclosure;

[0015] FIG3 b is a flow chart of a method for creating a CQ queue provided by another exemplary embodiment of the present disclosure;

[0016] FIG3c is a flowchart of a method for creating a QP queue provided by another exemplary embodiment of the present disclosure;

[0017] FIG3 d is a flow chart of a method for same-host identification provided by another exemplary embodiment of the present disclosure;

[0018] FIG3e is a flow chart of a memory registration method provided by another exemplary embodiment of the present disclosure;

[0019] FIG4 is a schematic structural diagram of a driver management module provided by another exemplary embodiment of the present disclosure;

[20] FIG5 is a flowchart of a method executed by a transmitting end when the type of a data transmission task is a bilateral operation, provided by another exemplary embodiment of the present disclosure;

[21] FIG6 is a flowchart of a method executed by a receiving end when the type of a data transmission task is a bilateral operation, provided by another exemplary embodiment of the present disclosure;

[22] FIG7 is a flowchart of a method for an RDMA application to check a CQ queue provided by another exemplary embodiment of the present disclosure;

[23] FIG8 is a flowchart of an RDMA-based data transmission method provided by an exemplary embodiment of the present disclosure;

[24] FIG9 is a schematic diagram of the structure of an RDMA-based data transmission device provided by an exemplary embodiment of the present disclosure;

[0025] FIG10 is a schematic diagram of the structure of a physical machine provided by an exemplary embodiment of the present disclosure.

[0026] To further clarify the objectives, technical solutions, and advantages of this disclosure, the technical solutions of this disclosure will be described clearly and completely below in conjunction with specific embodiments of the disclosure and the corresponding drawings. It should be understood that the described embodiments represent only a portion of the embodiments of this disclosure, and are not exhaustive. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of this disclosure without inventive effort are intended to fall within the scope of protection of this disclosure.

[0027] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0028] RDMA is a technology for directly accessing peer memory at high speed, leveraging a fully optimized network card device and software architecture. The present disclosure does not limit the implementation of RDMA technology at the protocol level; for example, it may include but is not limited to the following.

[0029] Infiniband protocol: RDMA technology based on InfiniBand architecture was proposed by IBTA (InfiniBand Trade Association, literally translated as InfiniBand Trade Association), referred to as IB. Infiniband networks require dedicated IB network cards and IB switches. In terms of performance, Infiniband networks offer better performance, but the network cards and switches are relatively expensive.

[0030] iWARP (Internet Wide Area RDMA Protocol), short for Internet Wide Area RDMA Protocol, is an RDMA technology based on the Transmission Control Protocol / Internet Protocol (TCP / IP) and defined by the Internet Engineering Task Force (IETF). iWARP supports RDMA over standard Ethernet infrastructure, eliminating the need for switches supporting lossless Ethernet transmission. However, servers must use iWARP-capable network cards.

[0031] RoCE (RoCE), short for RDMA over Converged Ethernet, was also proposed by the IBTA. RoCE supports RDMA technology over standard Ethernet infrastructure, but requires switches that support lossless Ethernet transmission and servers equipped with RoCE network adapters. Its performance is comparable to IB.

[0032] RDMA technologies using different protocols will have different hardware implementations, specifically different physical layers and link layers. However, regardless of the protocol or hardware implementation used by RDMA, the application can use the same set of application programming interfaces. The Application Programming Interface (API) provided by RDMA for applications can also be called the software transport interface (STI) between the application and the network card device, or the Verbs interface. In other words, RDMA provides a series of Verbs interfaces in the application's user space. Applications use these Verbs interfaces to operate the RDMA hardware, allowing RDMA to bypass the kernel and directly access the application's user space. Applications in the embodiments of this disclosure refer to applications that can directly read and write remote memory using RDMA technology. These applications are implemented using the RDMA API and can be referred to as RDMA applications or simply applications. From a functional perspective, the applications in the embodiments of this disclosure can include video processing applications, instant messaging applications, e-commerce applications, and other applications implemented using the RDMA API.

[0033] Traditional verbs interfaces include two types: memory verbs and messaging verbs. Based on these two verbs interfaces, RDMA provides two basic operations for applications: unilateral operations and bilateral operations.

[0034] Memory verbs: These include RDMA read, write, and atomic operations. These operations are unilateral, requiring only the local end to clearly identify the source and destination addresses of the data being transmitted. The peer application does not need to be aware of the communication. Data reading or writing is accomplished between the peer network card device and the application's buffer using Direct Memory Access (DMA) technology, without requiring the peer application's awareness or participation.

[0035] RDMA Read: The local application uses RDMA technology to read partial memory data from the peer. Before executing an RDMA read operation, the peer must grant the local application the appropriate permissions to access the peer's memory. Once permissions are set, the RDMA read operation can be performed without notifying the peer. The local application specifies a virtual address on the peer. The local network card then uses RDMA technology to read data directly from the peer's memory at the specified virtual address, just as it would read data from local memory, without the peer application's knowledge.

[0036] RDMA Write: Similar to RDMA Read, this operation writes data to the peer's memory. Before executing an RDMA write operation, the peer must grant the local client the appropriate permissions to access the peer's memory. Once permissions are set, the RDMA write operation can be performed without notifying the peer. The local application specifies a virtual address on the peer. The local network card then uses RDMA technology, just like writing data to local memory, to write data directly to the peer's memory at the specified virtual address without the peer application's knowledge.

[0037] RDMA Atomic: This includes atomic loads, atomic adds, atomic comparisons, and atomic swaps, and is an extension of RDMA atomic operations. This disclosure does not focus on this.

[0038] Messaging verbs: Includes RDMA send (Send) operations and RDMA receive (Receive) operations. These operations are bilateral operations, that is, data sending and receiving operations that require the perception and participation of the application on the other end to complete.

[0039] RDMA Send: This is the process of sending data from the local memory to the peer memory using RDMA technology. Specifically, the local application needs to place a Work Queue Element (WQE) representing a send task into the send queue (SQ) of a queue pair (QP). Similarly, the peer application needs to send a WQE representing a receive task to the gateway device (i.e., the RDMA hardware) to the receive queue (RQ) of the peer QP. This way, the RDMA hardware knows where to place the received data in memory. Then, the local RDMA The hardware obtains a send task from the SQ. Based on the send task, it retrieves data from its own memory and transmits it over the network to the peer RDMA hardware. The peer RDMA hardware receives the data, verifies it, and sends an acknowledgment (ACK) to the sender. It then places the data in the location specified in the WQE. It then generates a Completion Queue Element (CQE) for the receive task and places it in the Completion Queue (CQ). The peer application then receives the task completion information. After receiving the ACK, the local end also generates a CQE for the send task and places it in the CQ. The local application then receives the task completion information. The entire process requires the awareness and participation of both the local and peer applications.

[0040] RDMA Receive: This is the process of receiving data from the peer's memory into the local memory using RDMA technology. Specifically, the peer application sends a WQE (Warning Query Entity) representing a send task to the gateway device (i.e., the RDMA hardware) and places it into the peer's SQ. The local application places the WQE representing a receive task into the QP's RQ. This allows the RDMA hardware to know where to place the received data in memory. The peer's RDMA hardware then retrieves the send task from the SQ, retrieves the data from the peer's memory based on the send task, and transmits it over the network to the local RDMA hardware. The local RDMA hardware receives the data, verifies it, and sends an acknowledgment (ACK) to the peer. It then places the data into the location specified in the WQE, generates a CQE for the receive task, and places it into the CQ. The local application then receives the task completion information. After receiving the ACK, the peer also generates a CQE for the send task and places it into the CQ. The peer application then receives the task completion information. The entire process requires the awareness and participation of both the local and peer applications.

[0041] Optionally, the local application and the peer application may be instant messaging applications, or the local application may be a client of a database storage system and the peer application may be a server of the database storage system, and so on.

[0042] In actual applications, the local application and the peer application may reside on different physical machines or on the same physical machine. In particular, with the development of virtualization technology, a virtualization manager can run on a physical machine. The virtualization manager is responsible for creating and maintaining virtualization instances, which provide independent operating environments for hosting various applications. A virtualization manager can be a virtual machine monitor (VMM), also known as a hypervisor. A virtualization instance can be a cloud host, Elastic Compute Service (ECS), a virtual machine (VM), or a container. Applications in different virtualization instances can communicate using RDMA technology. For example, in a cloud-native containerization scenario, multiple containers run on the same physical machine, with RDMA applications deployed in different containers. These RDMA applications in different containers communicate using RDMA technology. In the embodiments of the present disclosure, improvements are made to the RDMA communication process between different applications deployed on the same physical machine, providing an improved RDMA communication method. The following describes the technical problems faced by RDMA communication between different applications on the same physical machine, the improvement ideas proposed to address these technical problems, and the improved RDMA communication process.

[0043] As shown in Figure 1, the hardware resources of a physical machine include at least a CPU, memory, and an RDMA network card. The RDMA network card is interconnected with the CPU and memory via the PCIe bus. In this embodiment, the implementation form of the physical machine is not limited. The physical machine can be implemented as a terminal device such as a desktop computer, laptop computer, smartphone, or IoT device, or as a server-side device such as a conventional server, cloud server, or server array. Similarly, in this disclosed embodiment, the implementation form of the RDMA network card is not limited. It can be any network card device that supports the RDMA protocol, such as a Cloud Infrastructure Processing Unit (CIPU), Smart NIC (Smart NIC), Network Interface Card (NIC), or Infrastructure Processing Unit (IPU) chip or module that integrates an RDMA-supported network card module. Furthermore, PCIe is used as an example of the interconnection bus between RDMA and the CPU and memory, but is not limited to this.

[0044] For different applications on the same physical machine, for example, the first and second applications each have their own memory spaces in memory, storing their own data. For example, let's say data A is stored in the first memory space of the first application and needs to be sent to the second application. In traditional RDMA communication, data A is transferred from the first memory space to the RDMA network card via the PCIe bus (line 1 in Figure 1), and then from the RDMA network card to the second memory space of the second application via the PCIe bus (line 2 in Figure 1). The entire data transfer process requires two PCIe operations, consuming two PCIe bandwidth cycles. This results in poor transmission performance for data transfers on the same physical machine and increases transmission latency.

[0045] Secondly, RDMA communication on the same physical machine includes both local traffic and network traffic. Local traffic refers to the traffic generated by communication between two RDMA applications on the same physical machine, while network traffic refers to the traffic generated by communication between two RDMA applications on different physical machines. Considering that the RDMA network card has different processing logic for local traffic and network traffic, and that local traffic is not affected by the congestion control algorithm and does not experience packet loss, resource allocation will allocate more cache resources to network traffic and less cache resources to local traffic. In the case of originally unbalanced resource allocation, In this case, when local traffic and network traffic compete for resources on the same RDMA network card, although the local traffic and network traffic themselves are equivalent, the network traffic stores more information. During processing and scheduling, the network traffic that occupies more resources will be processed first, which will trigger resource preemption, squeeze out the local traffic, and cause unfairness.

[0046] To address the above technical issues, in the embodiments of the present disclosure, based on the premise of compatibility with RDMA programming interfaces and transparency or imperceptibility to applications, improvements are made to RDMA communication between different applications on the same physical machine. This reduces the PCIe bandwidth consumed by RDMA communication between different RDMA applications on the same physical machine, reduces data transmission latency, and ensures fairness between local traffic and network traffic when competing for resources.

[0047] To ensure compatibility with the RDMA programming interface and transparency or unawareness to applications, the disclosed embodiments simultaneously create a local work queue and a remote work queue during the RDMA connection establishment process. Same-host identification is also added. When same-host RDMA communication is detected, the local work queue is used to directly copy data between the two applications' registered memories, replacing traditional RDMA transfer operations. For applications, they simply initiate RDMA communication requests, just like traditional RDMA communication. This compatibility with the RDMA programming interface and transparency or unawareness to applications can reduce transmission latency between different RDMA applications on the same host, minimize PCIe bandwidth consumption, and ensure fairness between local and network traffic during resource contention. Furthermore, when cross-host RDMA communication is detected, the remote work queue is used to perform traditional RDMA transfer operations, without affecting the cross-host RDMA communication process.

[0048] The following describes in detail the technical solutions provided by various embodiments of the present disclosure in conjunction with the accompanying drawings.

[0049] The present embodiment provides a distributed system. This distributed system can be a distributed storage system, such as various distributed database systems, or a distributed application system, such as a microservice system, without limitation. The distributed system of this embodiment includes multiple physical machines, each of which runs a virtualization manager. The virtualization manager creates and manages virtualization instances, and RDMA applications are deployed in the virtualization instances. The RDMA applications are implemented based on traditional RDMA programming interfaces, and different RDMA applications can communicate with each other using RDMA technology. The following describes the RDMA communication process between different RDMA applications provided by the present embodiment, with reference to the hardware and software architecture within the physical machine shown in Figures 2a and 2b.

[0050] Figure 2a is a schematic diagram of the structure of a physical machine provided in an exemplary embodiment of the present disclosure. As shown in Figure 2a, the internal hardware and software architecture of the physical machine includes an application layer, a kernel layer, and a hardware layer. The hardware layer includes the various hardware resources of the physical machine. The hardware resources of the physical machine include at least a CPU, memory, and an RDMA network card. In addition, the hardware resources of the physical machine also include communication components, a display, a power supply, audio components, and various external devices. Figure 2a illustrates an RDMA network card 11, but is not limited thereto. In this embodiment, a kernel layer runs on the hardware resources of the physical machine. The kernel layer includes at least an operating system. Above the operating system is the application layer. The application layer includes various virtualized instances running on the physical machine, each of which has an application deployed in it. Figure 2a illustrates the virtualized instances using containers A and B as examples. Container A has a first application deployed, and container B has a second application deployed. In contrast to application programs, an operating system is a set of interrelated system software programs that manage and control various physical machine operations, utilize and run hardware and software resources, and provide public services to facilitate user interaction. In the disclosed embodiments, the operating system serves as a bridge between application programs and hardware resources. The operating system includes drivers for various hardware resources.

[0051] In the disclosed embodiment, an RDMA hypervisor is also deployed on the physical machine. This RDMA hypervisor includes an interface management module 13 in user mode and a driver management module 12 in kernel mode. User mode and kernel mode are two different privilege levels in a computer system, representing the execution state of programs at different privilege levels. Programs in user mode have lower privileges and are subject to restrictions, generally referring to the running state of program code in the application layer. Programs in kernel mode, on the other hand, have higher privileges and can directly access and operate the core functions and hardware resources of the operating system. This generally refers to the running state of program code in the kernel layer. User mode executes in user space, while kernel mode executes in kernel space. Switching between the two modes can be achieved through system calls.

[0052] The distributed system described above includes communication between different applications on the same host and communication between applications across hosts. In this embodiment, with reference to the internal architecture of the physical machine shown in FIG. 2a and FIG. 2b , the communication process between a first application and a second application is taken as an example to illustrate communication between different applications on the same host and communication between applications across hosts.

[0053] In this embodiment, the driver management module 12 and the interface management module 13 can cooperate to complete RDMA communication between the first application and the second application. As a user-state library for RDMA technology, the interface management module can provide RDMA programming interfaces for RDMA applications (e.g., the first application and the second application) so that RDMA applications can initiate RDMA communication. These RDMA programming interfaces include, but are not limited to, ibv_modify_qp (for modifying QP), ibv_reg_mr (for memory registration), and ibv_create_qp (used to create a QP queue), ibv_post_send (used to send a work request), ibv_post_recv (used to send a receive operation These RDMA programming interfaces can be divided into two categories: one is the programming interface required for RDMA connection establishment, such as ibv_create_cq, ibv_create_qp, ibv_modify_qp, and ibv_reg_mr. These programming interfaces are also called control plane programming interfaces and are used for preparatory work before data transfer, such as establishing internal registration, resource allocation and management between applications. The other is the programming interface for RDMA data transmission, such as ibv_post_recv, ibv_post_send, and ibv_poll_cq. These programming interfaces are also called data plane programming interfaces and are used for RDMA data transmission operations, such as sending and receiving data and monitoring the completion status of operations. Some of the above interfaces are prefixed with ibv (user mode). These interfaces are part of the InfiniBand architecture. The InfiniBand architecture provides a set of user space (user mode) programming interfaces for developing high-efficiency, low-latency network communication applications.

[0054] The first application can initiate RDMA communication with the second RDMA application through the RDMA programming interface provided by the interface management module 13. A complete RDMA communication process includes both control and data plane processes. Specifically, the first application can call the programming interface for RDMA connection establishment to request an RDMA connection with the second application. After the RDMA connection is established, the first application can call the programming interface for RDMA data transfer to perform RDMA data transfer with the second application. In the disclosed embodiments, improvements have been made to the RDMA connection establishment and RDMA data transfer processes, but these improvements are imperceptible to the first and second applications. Both the RDMA connection establishment and RDMA data transfer operations are identical to traditional RDMA communication processes.

[0055] In this embodiment, the interface management module 13 can respond to the first application's calls to various RDMA programming interfaces and, based on these calls, cooperate with the driver management module 12 to complete the RDMA communication process between the first application and the second application. Specifically, the interface management module 13 can respond to the first application's calls to the programming interface for RDMA connection establishment and, in turn, call the driver management module 12 to establish an RDMA connection between the first and second applications. The RDMA connection establishment process in this embodiment differs from the traditional RDMA connection establishment process. In this embodiment, the driver management module 12 provides a system call interface, such as an Application Binary Interface (ABI), to the interface management module 13. The interface management module 13 can issue call requests to the driver management module 12 through the system call interface provided by the driver management module 12. The driver management module 12 is responsible for processing user-mode system call requests through the system call interface (e.g., the ABI).

[0056] Specifically, when a first application initiates an RDMA communication request to a second application, the driver management module 12 responds to the call of the interface management module 13, creates a first RDMA work queue for the first application, and performs memory registration. Memory registration generates memory index information, including an LKey (Local Key) and an RKey (Remote Key), which are used to guarantee access rights to local and remote memory areas, respectively. Memory registration obtains the first application's memory region (MR). The MR refers to a memory area allocated by the driver management module 12 for the first application in the memory of the physical machine to which it belongs, used to store and receive data.

[0057] The driver management module 12 is further configured to respond to a call from the interface management module 13 and, based on the control plane connection between the first and second applications, obtain description information of a second RDMA work queue corresponding to the second application as context information for the first RDMA work queue. Both the first and second RDMA work queues include a local work queue and a remote work queue. The local work queue is used for intra-host RDMA communication, and the remote work queue is used for inter-host RDMA communication. Optionally, the local work queue includes a local CQ queue (Completion Queue) and a local QP queue (Queue Pair), and the remote work queue includes a remote CQ queue and a remote QP queue. It is explained here that a control plane link can be established between the first application and the second application. Compared to the RDMA connection, the control plane connection between the first application and the second application can be regarded as an out-of-band communication method. Through the control plane link, description information of the first RDMA work queue and description information of the second RDMA work queue can be exchanged between the first application and the second application. In this process, the driver management module 12 on the physical machine where the first application is located can obtain the description information of the second RDMA work queue as the context information of the first RDMA work queue. Similarly, the driver management module 12 on the physical machine where the second application is located can obtain the description information of the first RDMA work queue as the context information of the second RDMA work queue.

[0058] In this embodiment, the implementation method of the control plane connection between the first application and the second application is not limited. For example, it can be a TCP connection or a UDP connection, and this is not limited. Furthermore, the control plane connection between the first application and the second application can be established based on a socket API or a communication manager API (CM API), and this disclosure does not impose any restrictions on this. In this embodiment, establishing a connection generally refers to establishing a connection between a remote QP queue in the first RDMA work queue and a remote QP queue in the second RDMA work queue. Based on this connection, description information of the first and second RDMA work queues can be exchanged. The description information includes, but is not limited to, a queue pair number (QPN) and a global identifier (GID).

[0059] After exchanging descriptive information based on the control plane connection at both ends, the first and second applications can conduct RDMA communication based on their registered memory areas. At this point, the interface management module 13 responds to the first application's call to the programming interface for RDMA data transmission and performs different processing depending on whether it is intra-host RDMA communication. If it is intra-host RDMA communication, the interface management module 13 triggers the driver management module 12 to enter kernel mode, which converts the RDMA data transfer process into a local memory copy operation. If it is inter-host RDMA communication, the interface management module 13 directly performs traditional RDMA operations in user mode, sending the data through the RDMA network card.

[0060] In this embodiment, the driver management module 12 is further configured to identify, based on the description information of the second RDMA work queue, whether the communication between the first application and the second application is intra-host RDMA communication. Furthermore, if intra-host RDMA communication is identified, the driver management module 12 responds to a call from the interface management module 13 and, based on the local work queue of at least one of the two applications, performs a data copy operation in the memory registered by the two applications, replacing the RDMA data transfer operation between the two applications.

[0061] Furthermore, the driver management module 12 is further configured to provide the interface management module 13 with an identification result of whether intra-host RDMA communication is occurring. The interface management module 13 is further configured to: if the identification result is intra-host RDMA communication, in response to the first application's call to a programming interface for RDMA data transfer, trigger the driver management module 12 to enter kernel mode, whereupon the driver management module 12 converts the RDMA data transfer process into a local memory copy operation; and if the identification result is inter-host RDMA communication, perform the RDMA data transfer operation between the registered memories of the two applications via the RDMA network card based on the remote work queue of at least one of the two applications. It should be noted that, if the identification result is inter-host RDMA communication, the RDMA-based communication process between the two applications is the same as the traditional RDMA communication process.

[0062] In the embodiment of the present disclosure, when the identification result is RDMA communication with the host, the implementation method of the interface management module 13 triggering the driver management module 12 to enter the kernel state is not limited. In an optional embodiment, the interface management module 13 may issue a call request to the driver management module 12 through a system call interface provided by the driver management module 12, thereby instructing the driver management module 12 to perform a data copy operation in kernel mode in the memory registered by the two applications based on the local work queue of at least one of the two applications, thereby replacing the RDMA data transfer operation between the two applications. In another optional embodiment, during the process of creating the work queue, the driver management module 12 creates a command queue and shares the command queue with the interface management queue. Specifically, when the interface management module 13 identifies intra-host RDMA communication, in response to the first application's call to the programming interface for RDMA data transfer, it writes a local memory copy command to the command queue. The driver management module 12 periodically polls the command queue. When the local memory copy command written by the interface management module 13 appears in the command queue, it may be determined that the communication between the first application and the second application is intra-host RDMA communication. Therefore, based on the local work queue of at least one of the two applications, the driver management module 12 performs a data copy operation in the memory registered by the two applications, thereby replacing the RDMA data transfer operation between the two applications.

[0063] In an alternative embodiment, the command queue can be a local SQ queue in the local work queue, and the local memory copy command is the SQE. In another alternative embodiment, the command queue can also be an independent queue. A separate command queue can be created at the same time as the first RDMA work queue is created. It should be noted that in this embodiment, the first application is used as the sender, so the command queue is created at the same time as the first RDMA queue is created. In actual situations, if the second application acts as the sender, the command queue should be created at the same time as the second RDMA work queue is created.

[0064] In an optional embodiment, when intra-host RDMA communication is identified, the first application and the second application are deployed in different virtualized instances on the same physical machine. That is, although the two applications are deployed on the same physical machine, they are deployed in different independent runtime environments (i.e., virtualized instances) using virtualization technology, such as virtual machines or containers. In this case, the second application and the first application are deployed on the same physical machine. As shown in Figure 2a, the RDMA hypervisor includes an interface management module 13 corresponding to the first application and an interface management module 13 corresponding to the second application. The interface management module 13 of the second application and the interface management module 13 of the first application are functionally identical, performing both interface management functions when acting as a sender and interface management functions when acting as a receiver. In actual applications, if the corresponding application acts as a sender, the interface management function is performed as a sender; if the corresponding application acts as a receiver, the interface management function is performed as a receiver. In this embodiment, since the second application acts as the receiving end, the interface management module 13 corresponding to the second application is configured to respond to the second application's request to create a second RDMA work queue and register the work in memory, and to initiate a call to the driver management module 12. Accordingly, the driver management module 12 is further configured to: when the first application initiates an RDMA communication request to the second application, respond to the call from the second application's interface management module 13, create a second RDMA work queue for the second application, register the work in memory, and save the index information of the registered memory; and, based on the control plane connection between the first and second applications, obtain the description information of the first RDMA work queue as context information for the second RDMA work queue.

[0065] In the embodiment of the present disclosure, the internal implementation structure of the interface management module 13 is not limited. The various internal implementation structures of the functions implemented by the interface management module 13 are applicable to the embodiments of the present disclosure. Similarly, the embodiments of the present disclosure do not limit the internal implementation structure of the driver management module 12. Any internal implementation structure that can accomplish the functions implemented by the driver management module 12 described above is applicable to the embodiments of the present disclosure. In the following embodiment, as shown in FIG2b , an internal implementation structure of the interface management module 13 and the driver management module 12 is provided.

[0066] In an optional embodiment, the interface management module 13 includes: a resource management unit 131 for providing management plane functions of RDMA communication and a data transceiver unit 132 for providing data plane functions of RDMA communication; accordingly, the driver management module 12 includes: a transmission service unit 122 for providing communication services based on local shared memory copy for intra-host RDMA communication and an intra-host identification unit 12L for identifying whether it is intra-host RDMA communication.

[0067] The resource management unit 131 is configured to respond to a completion queue (CQ) creation request, a queue pair (QP) creation request, and a memory registration request initiated by the first application, and initiate calls to the transmission service unit 122, respectively. Accordingly, the transmission service unit 122 is configured to respond to the calls of the resource management unit 131, create a local CQ queue and a remote CQ queue, a local QP queue and a remote QP queue for the first application, register memory for the first application, and save registered memory index information. Local work queues include local CQ queues and local QP queues, remote work queues include remote CQ queues and remote QP queues, local QP queues include local send queues (SQ) and local receive queues (RQ), and remote QP queues include remote SQ queues and remote RQ queues.

[0068] In the disclosed embodiment, the interface management module 13 operates in user mode, while the driver management module 12 operates in kernel mode. When a user-mode application needs to perform privileged operations (such as interacting directly with hardware), it must "fall into" kernel mode through system calls or other means. For example, a call from the resource management unit 131 to the transmission service unit 122 represents interaction between user mode and kernel mode. The resource management unit 131 can access the kernel-mode transmission service unit 122 through a system call to prepare for data transmission.

[0069] Further, as shown in Figure 3a, the resource management unit 131 also includes a qp management submodule, an mr management submodule, and a cq management submodule; the data transceiver unit 132 includes a cq query (poll_cq) submodule, a receive (post_recv) submodule, and a send (post_send) submodule. Accordingly, as shown in Figure 4, the transmission service unit 122 includes a resource mapping submodule and a memory copy submodule. The resource mapping submodule corresponds to each submodule in the resource management unit 131 and is used to cooperate with each submodule in the resource management unit 131 to complete the RDMA connection establishment process. During the RDMA connection establishment process, the same-host identification unit 121 in the driver management module 12 is used to identify whether the communication between the first and second applications is same-host RDMA. Specifically, the same-host identification unit 121 determines the IP address of the second application based on the GID in the description information of the second RDMA work queue. In a single-tenant scenario, whether the communication between the first and second applications is intra-host RDMA communication can be determined based on the routing table of the physical machine and the IP address of the second application. Specifically, if the routing table contains the IP address of the second application, it is determined that the communication between the first and second applications is intra-host RDMA communication. Conversely, if the routing table does not contain the IP address of the second application, it is determined that the communication between the first and second applications is inter-host RDMA communication. In a multi-tenant scenario, a physical machine includes a virtual switch (vSwitch). The virtual switch is used to virtualize the functions of the physical switch on the physical machine, dividing a physical switch into multiple logical switches to meet the needs of multi-tenancy. The virtual switch maintains a forwarding table, also known as a flow table. The flow table contains multiple entries, each of which includes at least a header field and an action. The header field stores information such as the switch ingress port, source MAC address, destination MAC address, source IP address, destination IP address, source port number, and destination port number. Based on the flow table of the virtual switch on the physical machine and the IP address of the second application, it is determined whether the communication between the first and second applications is intra-host RDMA. Specifically, the IP address of the second application is matched against the flow table to obtain information such as the source MAC address, destination MAC address, source IP address, destination IP address, source port number, and destination port number of the second application. Based on this information, it is determined whether the communication between the first and second applications is intra-host RDMA.The memory copy submodule corresponds to each submodule in the data transceiver unit 132 and is used to cooperate with each submodule in the data transceiver unit 132 to complete the data transmission operation.

[0070] The resource mapping submodule corresponds to each submodule in the resource management unit 131 and is used to cooperate with each submodule in the resource management unit 131 to complete the RDMA connection establishment process.

[0071] During the RDMA connection process, as shown in Figure 3b, when the first application calls the create_cq programming interface, the cq management submodule of the resource management unit 131 will access the resource mapping submodule in the transmission service unit 122 through a system call, and perform the following steps through the resource mapping submodule: creating a local CQ queue for storing CQEs generated by intra-host RDMA communication; creating a remote CQ queue for storing CQEs generated by cross-host RDMA communication; and establishing a mapping between the local CQ queue and the remote CQ queue to support cross-host RDMA communication. The order of creating the local CQ queue and the remote CQ queue is not important. Make limitations.

[0072] Furthermore, as shown in Figure 3c, when the first application calls the create_qp programming interface, the QP management submodule of the resource management unit 131 accesses the resource mapping submodule of the transport service unit 122 through a system call to perform the following steps: creating local QP queues (including local SQ queues and local RQ queues) for storing send and receive task elements in intra-host RDMA communication; creating remote QP queues for storing send and receive task elements in inter-host RDMA communication; and establishing a mapping between local QP queues and remote QP queues to support inter-host RDMA communication. Optionally, a separate command queue may also be created. The order of creating local QP queues and remote QP queues is not limited.

[0073] The remote work queue has a queue pair number (QPN) and a global identifier (GID). The description of the RDMA work queue to which the remote work queue belongs includes the remote work queue's QPN and GID. A GID is a network layer address that uniquely identifies each node (e.g., a physical machine) in a network. Its function is similar to the network layer address in the Transmission Control Protocol / Internet Protocol (TCP / IP), or the Internet Protocol Address (IP Address). A QPN is a unique number for a node's remote QP queue. By combining the GID and QPN, a QP queue can be uniquely identified within the network.

[0074] To achieve unawareness of RDMA applications, the local work queue can be either not assigned a QPN or a GID, or it can share the QPN and GID of the remote work queue within its RDMA work queue. From the application's perspective, the local and remote work queues are abstracted as the same RDMA work queue. This RDMA work queue has a unified QPN and GID for applications, ensuring compatibility with traditional RDMA programming interfaces and unawareness of the application.

[0075] When the peer QP has already been created, as shown in Figure 3d , when the first application calls the modify_qp programming interface, the QP management submodule of the resource management unit 131 accesses the same-host identification unit 121 of the transport service unit 122 through a system call to perform the following steps: It determines whether the communication is intra-host RDMA communication based on the GID in the description information of the second application's QP queue; if intra-host RDMA communication is identified, the identification result is saved in the resource management unit 131; optionally, if inter-host RDMA communication is identified, the identification result is saved in the resource management unit 131, so that the interface management module 13 can perform traditional RDMA transfer operations using the remote work queue if the communication is inter-host. Of course, it is also possible to save only the identification result for intra-host RDMA communication and not the identification result for inter-host RDMA communication. In this way, if no identification result for intra-host RDMA communication is found, the default is inter-host RDMA communication.

[0076] As shown in Figure 3e, when a first application calls the reg_mr programming interface to register memory, the mr management submodule of the resource management unit 131 accesses the resource mapping submodule in the transmission service unit 122 through a system call. The resource mapping submodule then performs the following steps: 1. Memory registration operations corresponding to intra-host RDMA communication are performed, mapping the registered memory area to its own kernel address space and saving the memory index information (LKey and RKey); 2. Registration of the same memory area for inter-host RDMA communication is performed, allowing the local QP queue and the remote QP queue to use the same memory area for data transmission and reception. Optionally, in the case of intra-host RDMA communication, a second application also calls reg_mr to register memory. The registration process is similar to the above steps, except that the mr management submodule of the resource management unit 131 of the second application accesses the resource mapping submodule in the transmission service unit 122 through a system call, and the resource mapping submodule completes the memory registration operation for the second application.

[0077] It should be noted that during the RDMA connection establishment process, create_cq is executed before create_qp, and there is no restriction on the order between reg_mr and create_cq and create_qp.

[0078] The process of the transmission service unit 122 and the data transceiver unit 132 cooperating with each other to complete the same-host RDMA data transmission operation is as follows.

[0079] When the data transceiver unit 132 identifies the communication as intra-host RDMA communication, it responds to the data transmission task initiated by the first application, generates a Send Queue Entry (SQE), which is used to describe information about the transmission task, writes the SQE into the local SQ queue in the local work queue of the first application, and triggers the transmission service unit 122. Optionally, the data transceiver unit 132 triggers the transmission service unit 122 by: after writing the SQE into the local SQ queue in the local work queue of the first application, issuing a system call to the transmission service unit 122; or, after writing the SQE into the local work queue of the first application, writing a local memory copy command into the command queue, so that the transmission service unit 122 polls the command queue containing the local memory copy command; wherein the command queue is shared by the data transceiver unit 132 and the transmission service unit 122. Accordingly, the transmission service unit 122, triggered by the data transceiver unit 132, uses the local memory copy command to send the local memory copy command to the transmission service unit 122 according to the operation type of the data transmission task. The local work queues of the first application or the local work queues at both the sending and receiving ends (the first application and the second application) perform data copy operations in the memories registered by the two applications, replacing the RDMA data transmission operations between the two applications. In this embodiment, the first application is the sending end and the second application is the receiving end.

[0080] In an optional embodiment, the transmission service unit 122 is specifically configured to: obtain an SQE from the local SQ queue of the first application, and obtain the operation type of the data sending task from the SQE; if the data sending task is a bilateral operation, perform a data copy operation in the memories registered by the two applications using the local work queues at both ends of the sending and receiving ends, thereby replacing the RDMA data transmission operation between the two applications; if the data sending task is a unilateral operation, perform a data copy operation in the memories registered by the two applications using the local work queue of the first application, thereby replacing the RDMA data transmission operation between the two applications.

[0081] Further, optionally, when the data transmission task is a bilateral operation, the data transceiver unit 132 of the second application is further configured to: generate a receive queue element (RQE) in response to the data reception task initiated by the second application, the RQE describing the reception task, and write the RQE into the local RQ queue in the local work queue of the second application. Accordingly, the transport service unit 122 is specifically configured to: determine the description information of the RDMA work queue of the second application based on the context information of the RDMA work queue of the first application; read the SQE from the local SQ queue of the first application, and read the RQE from the local RQ of the second application based on the description information of the RDMA work queue of the second application; copy the data in the memory pointed to by the local memory index information (the LKey of the first application) contained in the SQE to the memory pointed to by the local memory index information (the LKey of the second application) contained in the RQE; and after the data copy is completed, generate a CQE and write the CQE into the local CQ queues of the local work queues of the first application and the second application, respectively. In bilateral operations, both the sending and receiving ends generate CQEs and write them into their respective local CQ queues.

[0082] Optionally, when the type of the data sending task is a unilateral operation, the transmission service unit 122 is specifically used to: if the operation type of the data sending task is a unilateral write operation, read the SQE from the local SQ queue of the first application, and copy the data in the memory pointed to by the local memory index information (LKey of the first application) contained in the SQE to the memory pointed to by the remote memory index information (RKey of the second application) contained in the SQE; if the operation type of the data sending task is a unilateral read operation, read the SQE from the local SQ queue of the first application, and copy the data in the memory pointed to by the remote memory index information (RKey of the second application) contained in the SQE to the memory pointed to by the local memory index information (LKey of the first application) contained in the SQE; after completing the data copy, generate a CQE, and write the CQEs into the local CQ queues in the local work queue of the first application respectively. In unilateral operations, the sending end can generate CQE on demand, while the receiving end does not. In a special case, if the operation type of the data sending task is a write with immediate value (Write with Immediate), the receiving end will generate CQE.

[0083] In the disclosed embodiment, an RDMA application can query the local and remote CQ queues for CQEs by calling the data transceiver unit 132, and return the query result to the RDMA application. The local CQ queue is used for intra-host RDMA communication; CQEs generated after copying are placed in the local CQ queue. The remote CQ queue is used for inter-host RDMA communication; CQEs generated after copying are placed in the remote CQ queue.

[0084] 4 and the memory copy submodule in the transmission service unit 122 corresponds to each submodule in the data transceiver unit 132 shown in FIG. 3a, the process for cooperating with each submodule in the data transceiver unit 132 to complete the data transmission operation is as follows.

[0085] As shown in FIG5 , when the first application calls the post_send programming interface, the post_send submodule in the data transceiver unit 132 determines that intra-host RDMA communication is performed based on the intra-host identification result. The post_send submodule responds to the data sending task initiated by the first application, generates an SQE, writes the SQE to the local SQ queue in the local work queue of the first application, and triggers the memory copy submodule of the transmission service unit 122. Accordingly, the memory copy submodule of the transmission service unit 122, under the triggering of the post_send submodule in the data transceiver unit 132, performs the following steps: obtains the SQE from the local SQ queue of the first application, and obtains the operation type of the data sending task from the SQE; if the type of the data sending task is a bilateral operation, determines the description information of the RDMA work queue of the second application based on the context information of the RDMA work queue of the first application; reads the SQE from the local SQ queue of the first application, and reads the RQE from the local RQ of the second application based on the description information of the RDMA work queue of the second application; copies the data in the memory pointed to by the local memory index information (LKey of the first application) contained in the SQE to the memory pointed to by the local memory index information (LKey of the second application) contained in the RQE; after completing the data copy, generates a CQE, and writes the CQE into the local CQ queues in the local work queues of the first application and the second application, respectively. In bilateral operations, both the sending and receiving ends generate CQEs and write them into their respective local CQ queues.

[0086] In the case where the data transmission task type is a unilateral operation, if the operation type of the data transmission task is a unilateral write operation, the SQE is read from the local SQ queue of the first application, and the local memory index information (the first application's The first application's local work queue (LKey) is copied to the memory pointed to by the remote memory index information (the second application's RKey) contained in the SQE. If the data sending task's operation type is a unilateral read operation, the first application's local SQE is read from the SQE, and the data in the memory pointed to by the remote memory index information (the second application's RKey) contained in the SQE is copied to the memory pointed to by the local memory index information (the first application's LKey) contained in the SQE. After the data copy is completed, a CQE is generated and written to the local CQs of the first application's local work queue. In a unilateral operation, the sending end can generate a CQE on demand, while the receiving end does not. In a special case, if the data sending task's operation type is a write with immediate value (Write with Immediate), the receiving end will generate a CQE.

[0087] As shown in FIG6 , when a first application calls the post_send programming interface to communicate with a second application, and the second application calls the post_recv programming interface, the post_recv submodule in the data transceiver unit 132 of the second application is further configured to: respond to the data reception task initiated by the second application, generate an RQE, write the RQE into a local RQ queue in the local work queue of the second application, and prepare to receive data sent by the first application; upon receiving the data sent by the first application, copy the data into the memory pointed to by the local memory index information (the LKey of the second application) contained in the RQE; and after completing the data copy, generate a CQE, and write the CQEs into the local CQ queues in the local work queue of the second application.

[0088] In the disclosed embodiment, whether it is the first application or the second application, the poll_cq submodule in its data transceiver unit 132 queries whether there are CQEs in the local and remote CQ queues and returns the query results to the corresponding application. For example, as shown in Figure 7, when either application calls the poll_cq interface, the poll_cq submodule in the data transceiver unit 132 performs the following steps: checks whether there are CQEs in the local CQ queue, then checks whether there are CQEs in the corresponding remote CQ queue, and returns the results to the application. The local CQ queue is used for intra-host RDMA communication; CQEs generated after copying are placed in the local CQ queue. The remote CQ queue is used for inter-host RDMA communication; CQEs generated after copying are placed in the remote CQ queue. It should be noted that the poll_cq submodule checks the local and remote CQ queues without going through the kernel. It can periodically poll the local and remote CQ queues or respond to calls to the poll_cq interface to check the local and remote CQ queues. In addition, there is no restriction on the order of checking the local CQ queue and the remote CQ queue.

[0089] Optionally, when performing a data copy operation in memories registered by two applications, the transmission service unit 122 (specifically, the memory copy submodule in the transmission service unit 122) is specifically configured to: execute a memory copy command of the processor CPU to copy data from the memory registered by one application to the memory registered by another application. In this method, CPU instructions are used to perform the memory copy, which generally involves load and store instructions, and thus occupies CPU cores.

[0090] Alternatively, if the CPU has a Direct Memory Access (DMA) acceleration engine, the CPU's DMA acceleration engine can be used to directly copy data from a memory registered by one application to a memory registered by another application without occupying a CPU core.

[0091] FIG8 is a flow chart illustrating an RDMA-based data transmission method according to an exemplary embodiment of the present disclosure. The method is applied to a physical machine on which a first application is deployed. For details regarding the physical machine, please refer to the aforementioned system embodiments and will not be repeated here. As shown in FIG8 , the method includes steps S801 to S804.

[0092] S801: When a first application initiates an RDMA communication request to a second application, a first RDMA work queue is created for the first application and registered in memory. The first RDMA work queue includes a local work queue and a remote work queue.

[0093] S802: Based on the control plane connection between the first application and the second application, obtain description information of a second RDMA work queue corresponding to the second application as context information of the first RDMA work queue.

[0094] S803: Identify whether the first application and the second application perform same-host RDMA communication based on the description information of the second RDMA work queue. The second RDMA work queue also includes a local work queue and a remote work queue.

[0095] S804: When it is identified as intra-host RDMA communication, a data copy operation is performed in the memory registered by the two applications based on the local work queue of at least one of the two applications, to replace the RDMA data transmission operation between the two applications.

[0096] In an optional embodiment, when intra-host RDMA communication is identified, the first application and the second application are applications deployed in different virtualized instances on the physical machine; the method further includes: when the first application initiates an RDMA communication request to the second application, creating a second RDMA work queue for the second application and registering it in memory; and, based on the control plane connection, obtaining description information of the first RDMA work queue as context information of the second RDMA work queue.

[0097] In an optional embodiment, the method further includes: in the case of identifying cross-host RDMA communication, based on the remote work queue of at least one application in the two applications, executing between the memories registered by the two applications through the RDMA network card on the physical machine. Perform RDMA data transmission operations.

[0098] In an optional embodiment, when a first application initiates an RDMA communication request to a second application, a first RDMA work queue is created for the first application and memory registration is performed, including: responding to a completion queue (CQ) creation request initiated by the first application to create a local CQ queue and a remote CQ queue for the first application; responding to a queue pair (QP) creation request initiated by the first application to create a local QP queue and a remote QP queue for the first application; responding to a memory registration request initiated by the first application to register memory for the first application and save index information of the registered memory; wherein the local work queue includes a local CQ queue and a local QP queue, and the remote work queue includes a remote CQ queue and a remote QP queue.

[0099] In an optional embodiment, the remote work queue has a queue pair number QPN and a global identity GID, and the description information of the RDMA work queue to which the remote work queue belongs includes the QPN and GID of the remote work queue; the local work queue does not have a QPN and GID, or the local work queue shares the QPN and GID of the remote work queue in the RDMA work queue to which it belongs.

[0100] In an optional embodiment, identifying, based on the description information of the second RDMA work queue, whether the first application and the second application perform same-host RDMA communication includes: determining the IP address of the second application based on the GID in the description information of the second RDMA work queue; in a single-tenant scenario, identifying, based on a routing table of the physical machine and the IP address of the second application, whether the first application and the second application perform same-host RDMA communication; and in a multi-tenant scenario, identifying, based on a flow table of a virtual switch on the physical machine and the IP address of the second application, whether the first application and the second application perform same-host RDMA communication.

[0101] Based on the above embodiment, optionally, when intra-host RDMA communication is identified, a data copy operation is performed in the memory registered by the two applications based on the local work queue of at least one of the two applications, to replace the RDMA data transfer operation between the two applications. The method includes: when intra-host RDMA communication is identified, in user state, in response to a data sending task initiated by the sender, generating an SQE, and writing the SQE into a local SQ queue in the local work queue of the sender; entering kernel state from user state, and in kernel state, performing a data copy operation in the memory registered by the two applications using the local work queue of the sender or both the sending and receiving ends according to the operation type of the data sending task, to replace the RDMA data transfer operation between the two applications; wherein the sender is any one of the two applications, and the other application is the receiving end.

[0102] In an optional embodiment, the method for entering kernel state from user state includes: responding to a system call initiated by the user state after the user state writes the SQE into a local SQ queue in the local work queue of the sending end, entering kernel state from user state; or polling a command queue shared by the user state and the kernel state, and entering kernel state from user state when the command queue contains a local memory copy command; wherein the local memory copy command is written into the command queue after the user state writes the SQE into the local SQ queue in the local work queue of the sending end.

[0103] In an optional embodiment, in kernel state, based on the operation type of the data sending task, a data copy operation is performed in memories registered by two applications using a local work queue on the sending end or on both ends of the sending and receiving end, thereby replacing the RDMA data transfer operation between the two applications. The method includes: in kernel state, obtaining the SQE from the local SQ queue on the sending end, and obtaining the operation type of the data sending task from the SQE; if the data sending task is a bilateral operation, performing the data copy operation in memories registered by the two applications using the local work queues on both ends of the sending and receiving end, thereby replacing the RDMA data transfer operation between the two applications; if the data sending task is a unilateral operation, performing the data copy operation in memories registered by the two applications using the local work queue on the sending end, thereby replacing the RDMA data transfer operation between the two applications.

[0104] In an optional embodiment, when the data sending task is a bilateral operation, the method further includes: in user mode, responding to the data receiving task initiated by the receiving end, generating an RQE, and writing the RQE into a local RQ queue in a local work queue of the receiving end; using the local work queues of the sending and receiving ends to perform a data copy operation in memories registered by two applications, replacing the RDMA data transfer operation between the two applications, including: determining description information of the RDMA work queue of the receiving end based on context information of the RDMA work queue of the sending end; reading an SQE from the local SQ queue of the sending end, and reading an RQE from the local RQ of the receiving end based on the description information of the RDMA work queue of the receiving end; copying data in the memory pointed to by the local memory index information contained in the SQE to the memory pointed to by the local memory index information contained in the RQE; after completing the data copy, generating a CQE, and writing the CQE into the local CQ queues in the local work queues of the sending end and the receiving end, respectively.

[0105] In an optional embodiment, a data copy operation is performed in the memory registered by the two applications using the local work queue of the sending end, instead of the RDMA data transfer operation between the two applications, including: if the operation type of the data sending task is a one-way write operation, reading the SQE from the local SQ queue of the sending end, and copying the data in the memory pointed to by the local memory index information contained in the SQE to the memory pointed to by the remote memory index information contained in the SQE; if the data sending task is a one-way write operation, reading the SQE from the local SQ queue of the sending end, and copying the data in the memory pointed to by the local memory index information contained in the SQE to the memory pointed to by the remote memory index information contained in the SQE; The operation type is a unilateral read operation. An SQE is read from the local SQ queue of the sending end, and the data in the memory pointed to by the remote memory index information contained in the SQE is copied to the memory pointed to by the local memory index information contained in the SQE. After the data copy is completed, a CQE is generated and written to the local CQ queue in the local work queue of the sending end.

[0106] In an optional embodiment, performing a data copy operation in memories registered by two applications includes: executing a memory copy command of a processor CPU to copy data from a memory registered by one application to a memory registered by another application; or utilizing a DMA acceleration engine of the CPU to copy data from a memory registered by one application to a memory registered by another application.

[0107] It should be noted that the execution entity of each step of the method provided in the above embodiment may be the same device, or the method may be executed by different devices. For example, the execution entity of steps 801 to 803 may be device A; for another example, the execution entity of steps 801 and 802 may be device A, and the execution entity of step 803 may be device B; and so on.

[0108] Furthermore, some of the processes described in the above embodiments and accompanying drawings include multiple operations that appear in a specific order. However, it should be understood that these operations may be executed in a different order than the order in which they appear herein or in parallel. Operation sequence numbers, such as 801 and 802, are merely used to distinguish between different operations and do not represent any specific execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that terms such as "first" and "second" herein are used to distinguish between different messages, devices, modules, and the like, and do not represent a sequential order or limit "first" and "second" to different types.

[109] FIG. 9 is a schematic diagram of the structure of an RDMA-based data transmission device provided in an exemplary embodiment of the present disclosure. As shown in FIG. 9 , the device 900 includes a creation module 91, an acquisition module 92, an identification module 93, and a copy module 94.

[0110] The creation module 91 is configured to create a first RDMA work queue for the first application and register it in memory when the first application initiates an RDMA communication request to the second application. The first RDMA work queue includes a local work queue and a remote work queue.

[0111] The acquisition module 92 is configured to acquire, based on the control plane connection between the first application and the second application, description information of a second RDMA work queue corresponding to the second application as context information of the first RDMA work queue.

[0112] The identification module 93 is configured to identify whether the first application and the second application perform same-host RDMA communication based on description information of the second RDMA work queue, where the second RDMA work queue also includes a local work queue and a remote work queue.

[0113] The copy module 94 is configured to, when identifying intra-host RDMA communication, perform a data copy operation in the memory registered by the two applications based on the local work queue of at least one of the two applications, to replace the RDMA data transmission operation between the two applications.

[0114] Further optionally, when same-host RDMA communication is identified, the first application and the second application are applications deployed in different virtualized instances on the physical machine. The creation module 91 is further configured to: when the first application initiates an RDMA communication request to the second application, create a second RDMA work queue for the second application and register it in memory; and the acquisition module 92 is configured to: based on the control plane connection, obtain description information of the first RDMA work queue as context information of the second RDMA work queue.

[0115] In an optional embodiment, the copy module 94 is further configured to: when cross-host RDMA communication is identified, perform an RDMA data transfer operation between the memories registered by the two applications through the RDMA network card on the physical machine based on the remote work queue of at least one of the two applications.

[0116] In an optional embodiment, when a first application initiates an RDMA communication request to a second application, the creation module 91 creates a first RDMA work queue for the first application and registers the first RDMA work queue in memory. The creation module 91 is specifically configured to: respond to a completion queue (CQ) creation request initiated by the first application to create a local CQ queue and a remote CQ queue for the first application; respond to a queue pair (QP) creation request initiated by the first application to create a local QP queue and a remote QP queue for the first application; respond to a memory registration request initiated by the first application to register memory for the first application and save index information of the registered memory. The local work queue includes a local CQ queue and a local QP queue, and the remote work queue includes a remote CQ queue and a remote QP queue.

[0117] Optionally, the remote work queue has a queue pair number QPN and a global identity GID, and the description information of the RDMA work queue to which the remote work queue belongs includes the QPN and GID of the remote work queue; the local work queue does not have a QPN and GID, or the local work queue shares the QPN and GID of the remote work queue in the RDMA work queue to which it belongs.

[0118] In an optional embodiment, when identifying, based on the description information of the second RDMA work queue, whether the first application and the second application perform same-host RDMA communication, the identification module 93 is specifically configured to: determine the IP address of the second application based on the GID in the description information of the second RDMA work queue; in a single-tenant scenario, identify, based on the routing table of the physical machine and the IP address of the second application, whether the first application and the second application perform same-host RDMA communication; and in a multi-tenant scenario, identify, based on the flow table of the virtual switch on the first physical machine and the IP address of the second application, whether the first application and the second application perform same-host RDMA communication.

[0119] In an optional embodiment, the apparatus of this embodiment further includes: a data transceiver module in user state, configured to, in user state, respond to a data transmission task initiated by a sender, generate an SQE, and write the SQE into a local SQ queue in a local work queue of the sender, when identifying intra-host RDMA communication. A copy module 94, when identifying intra-host RDMA communication, specifically is configured to: enter kernel state from user state, and in kernel state, based on the operation type of the data transmission task, perform a data copy operation in the memory registered by the two applications using the local work queue of at least one of the two applications, to replace the RDMA data transmission operation between the two applications, using the local work queue of the sender or both the sender and the receiver, to replace the RDMA data transmission operation between the two applications; wherein the sender is either of the two applications, and the other application is the receiver.

[0120] In an optional embodiment, the copy module 94 enters the kernel state from the user state in a manner specifically configured to: respond to a system call initiated after the user state data transceiver module writes the SQE into the local SQ queue in the local work queue of the sender, and enter the kernel state from the user state; or poll a command queue shared by the user state and the kernel state, and enter the kernel state from the user state when the command queue contains a local memory copy command; wherein the local memory copy command is written into the command queue after the user state data transceiver module writes the SQE into the local SQ queue in the local work queue of the sender.

[0121] In an optional embodiment, the copy module 94, in kernel state, uses the local work queues of the sending end or both the sending and receiving ends to perform a data copy operation in the memories registered by the two applications, instead of the RDMA data transmission operation between the two applications, based on the operation type of the data transmission task. Specifically, the copy module 94 is configured to: in kernel state, obtain the SQE from the local SQ queue of the sending end, and obtain the operation type of the data transmission task from the SQE; if the data transmission task type is a bilateral operation, use the local work queues of both the sending and receiving ends to perform the data copy operation in the memories registered by the two applications, instead of the RDMA data transmission operation between the two applications; if the data transmission task type is a unilateral operation, use the local work queue of the sending end to perform the data copy operation in the memories registered by the two applications, instead of the RDMA data transmission operation between the two applications.

[0122] In an optional embodiment, when the data sending task is a bilateral operation, the copy module 94 is further configured to: in user mode, respond to the data receiving task initiated by the receiving end, generate an RQE, and write the RQE into a local RQ queue in a local work queue of the receiving end; when using the local work queues of the sending and receiving ends to perform a data copy operation in memories registered by two applications to replace the RDMA data transmission operation between the two applications, specifically to: determine description information of the RDMA work queue of the receiving end based on context information of the RDMA work queue of the sending end; read an SQE from the local SQ queue of the sending end, and read an RQE from the local RQ of the receiving end based on the description information of the RDMA work queue of the receiving end; copy data in the memory pointed to by the local memory index information contained in the SQE to the memory pointed to by the local memory index information contained in the RQE; and after completing the data copy, generate a CQE and write the CQE into the local CQ queues in the local work queues of the sending end and the receiving end, respectively.

[0123] In an optional embodiment, when the copy module 94 uses the local work queue of the sending end to perform a data copy operation in the memory registered by two applications to replace the RDMA data transmission operation between the two applications, it is specifically configured to: if the operation type of the data sending task is a unilateral write operation, read the SQE from the local SQ queue of the sending end, and copy the data in the memory pointed to by the local memory index information contained in the SQE to the memory pointed to by the remote memory index information contained in the SQE; if the operation type of the data sending task is a unilateral read operation, read the SQE from the local SQ queue of the sending end, and copy the data in the memory pointed to by the remote memory index information contained in the SQE to the memory pointed to by the local memory index information contained in the SQE; after completing the data copy, generate a CQE, and write the CQEs into the local CQ queues in the local work queue of the sending end.

[0124] In an optional embodiment, when performing a data copy operation in memories registered by two applications, the copy module 94 is specifically configured to: execute a memory copy command of the processor CPU to copy data from the memory registered by one application to the memory registered by another application; or utilize the DMA acceleration engine of the CPU to copy data from the memory registered by one application to the memory registered by another application.

[0125] The detailed implementation and beneficial effects of each step in the method of this embodiment have been described in detail in the aforementioned embodiments and will not be elaborated on here.

[0126] FIG10 is a schematic diagram of the structure of a physical machine provided by an exemplary embodiment of the present disclosure. As shown in FIG5 , the device includes: one or more memories 104 and one or more processors 105.

[0127] Memory 104 is used to store computer programs and can be configured to store various other data to support operations on the physical machine. Examples of such data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, images, videos, etc.

[0128] The memory 104 may be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0129] The processor 105 is coupled to the memory 104 and configured to execute a computer program in the memory 104, configured to: when a first application initiates an RDMA communication request to a second application, create a first RDMA work queue for the first application and register it in memory, where the first RDMA work queue includes a local work queue and a remote work queue; obtain, based on a control plane connection between the first application and the second application, description information of a second RDMA work queue corresponding to the second application as context information of the first RDMA work queue; identify, based on the description information of the second RDMA work queue, whether intra-host RDMA communication between the first application and the second application is intra-host RDMA communication, where the second RDMA work queue also includes a local work queue and a remote work queue; and, if intra-host RDMA communication is identified, perform a data copy operation in the memory registered by the two applications based on the local work queue of at least one of the two applications, to replace the RDMA data transfer operation between the two applications.

[0130] In an optional embodiment, when the processor 105 identifies intra-host RDMA communication and the first application and the second application are applications deployed in different virtualized instances on the physical machine, the processor 105 is further configured to: create a second RDMA work queue for the second application and register it in memory when the first application initiates an RDMA communication request to the second application; and obtain, based on the control plane connection, description information of the first RDMA work queue as context information of the second RDMA work queue.

[0131] In an optional embodiment, the processor 105 is further configured to: when cross-host RDMA communication is identified, perform an RDMA data transmission operation between memories registered by the two applications through an RDMA network card on the physical machine based on a remote work queue of at least one of the two applications.

[0132] In an optional embodiment, when a first application initiates an RDMA communication request to a second application, the processor 105 creates a first RDMA work queue for the first application and registers the first RDMA work queue in memory. The processor 105 is specifically configured to: respond to a completion queue (CQ) creation request initiated by the first application to create a local CQ queue and a remote CQ queue for the first application; respond to a queue pair (QP) creation request initiated by the first application to create a local QP queue and a remote QP queue for the first application; respond to a memory registration request initiated by the first application to register memory for the first application and save index information of the registered memory. The local work queue includes a local CQ queue and a local QP queue, and the remote work queue includes a remote CQ queue and a remote QP queue.

[0133] Optionally, the remote work queue has a queue pair number QPN and a global identity GID, and the description information of the RDMA work queue to which the remote work queue belongs includes the QPN and GID of the remote work queue; the local work queue does not have a QPN and GID, or the local work queue shares the QPN and GID of the remote work queue in the RDMA work queue to which it belongs.

[0134] In an optional embodiment, when the processor 105 identifies whether the first application and the second application perform same-host RDMA communication based on the description information of the second RDMA work queue, the processor 105 is specifically configured to: determine the IP address of the second application based on the GID in the description information of the second RDMA work queue; in a single-tenant scenario, identify whether the first application and the second application perform same-host RDMA communication based on the routing table of the physical machine and the IP address of the second application; and in a multi-tenant scenario, identify whether the first application and the second application perform same-host RDMA communication based on the flow table of the virtual switch on the first physical machine and the IP address of the second application.

[0135] In an optional embodiment, when the processor 105 identifies intra-host RDMA communication, based on the local work queue of at least one of the two applications, performs a data copy operation in the memory registered by the two applications, instead of the RDMA data transfer operation between the two applications. Specifically, when the processor 105 identifies intra-host RDMA communication, in user mode, In response to a data sending task initiated by the sending end, an SQE is generated and written to a local SQ queue in a local work queue of the sending end. Then, the process enters kernel state from user state. In kernel state, based on the operation type of the data sending task, a data copy operation is performed in the memory registered by the two applications using the local work queues of the sending end or the sending and receiving ends, thereby replacing the RDMA data transfer operation between the two applications. The sending end is either of the two applications, and the other application is the receiving end.

[0136] In an optional embodiment, when entering kernel state from user state, the processor 105 is specifically configured to: respond to a system call initiated by the user state after the user state writes the SQE into the local SQ queue in the local work queue of the sender, and enter kernel state from user state; or poll a command queue shared by the user state and kernel state, and when the command queue contains a local memory copy command, enter kernel state from user state; wherein the local memory copy command is written into the command queue after the user state writes the SQE into the local SQ queue in the local work queue of the sender.

[0137] In an optional embodiment, the processor 105, in kernel state, uses the local work queues of the sending end or both the sending and receiving ends to perform a data copy operation in the memories registered by the two applications, instead of the RDMA data transmission operation between the two applications, based on the operation type of the data transmission task. Specifically, the processor 105 is configured to: in kernel state, obtain the SQE from the local SQ queue of the sending end, and obtain the operation type of the data transmission task from the SQE; if the data transmission task type is a bilateral operation, use the local work queues of the sending and receiving ends to perform the data copy operation in the memories registered by the two applications, instead of the RDMA data transmission operation between the two applications; if the data transmission task type is a unilateral operation, use the local work queue of the sending end to perform the data copy operation in the memories registered by the two applications, instead of the RDMA data transmission operation between the two applications.

[0138] In an optional embodiment, when the type of the data sending task is a bilateral operation, the processor 105 is further configured to: in user mode, respond to the data receiving task initiated by the receiving end, generate an RQE, and write the RQE into a local RQ queue in a local work queue of the receiving end; when using the local work queues of the sending and receiving ends to perform a data copy operation in memories registered by two applications to replace the RDMA data transfer operation between the two applications, specifically configured to: determine description information of the RDMA work queue of the receiving end based on context information of the RDMA work queue of the sending end; read an SQE from the local SQ queue of the sending end, and read an RQE from the local RQ of the receiving end based on the description information of the RDMA work queue of the receiving end; copy data in the memory pointed to by the local memory index information contained in the SQE to the memory pointed to by the local memory index information contained in the RQE; and after completing the data copy, generate a CQE, and write the CQE into the local CQ queues in the local work queues of the sending end and the receiving end, respectively.

[0139] In an optional embodiment, when the processor 105 uses the local work queue of the sending end to perform a data copy operation in the memories registered by two applications to replace the RDMA data transmission operation between the two applications, the processor 105 is specifically configured to: if the operation type of the data sending task is a unilateral write operation, read the SQE from the local SQ queue of the sending end, and copy the data in the memory pointed to by the local memory index information contained in the SQE to the memory pointed to by the remote memory index information contained in the SQE; if the operation type of the data sending task is a unilateral read operation, read the SQE from the local SQ queue of the sending end, and copy the data in the memory pointed to by the remote memory index information contained in the SQE to the memory pointed to by the local memory index information contained in the SQE; after completing the data copy, generate a CQE, and write the CQEs into the local CQ queues in the local work queue of the sending end.

[0140] In an optional embodiment, when the processor 105 performs a data copy operation in memories registered by two applications, it is specifically used to: execute a memory copy command of the processor CPU to copy data from the memory registered by one application to the memory registered by another application; or use the DMA acceleration engine of the CPU to copy data from the memory registered by one application to the memory registered by another application.

[0141] Furthermore, as shown in FIG10 , the physical machine also includes other components, such as a communication component 106, a display 107, a power supply component 108, and an audio component 109. FIG10 schematically illustrates only some components and does not mean that the physical machine only includes the components shown in FIG10 . Furthermore, the components within the dashed box in FIG10 are optional, not mandatory, and their specific requirements depend on the product form factor of the working node. The working node of this embodiment can be implemented as a terminal device such as a desktop computer, laptop computer, smartphone, or IoT device, or as a server-side device such as a conventional server, cloud server, or server array. If the working node of this embodiment is implemented as a terminal device such as a desktop computer, laptop computer, or smartphone, it may include the components within the dashed box in FIG10 ; if the working node of this embodiment is implemented as a server-side device such as a conventional server, cloud server, or server array, it may not include the components within the dashed box in FIG10 .

[0142] The detailed implementation and beneficial effects of each step in the method of this embodiment have been described in detail in the aforementioned embodiments and will not be elaborated on here.

[0143] Accordingly, an embodiment of the present disclosure further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is enabled to implement each step in the above method embodiment.

[0144] Accordingly, an embodiment of the present disclosure further provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the processor is enabled to implement the steps in the above method embodiments.

[0145] The described memory can be implemented by any type of volatile or non-volatile memory device or a combination of them, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0146] The communication component is configured to facilitate wired or wireless communication between the device housing the communication component and other devices. The device housing the communication component can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G, or other mobile communication networks, or a combination thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0147] The display includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, it may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensors can not only detect the boundaries of a touch or slide action, but also the duration and pressure associated with the touch or slide action.

[0148] The power supply assembly provides power to various components of the device in which the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply assembly is located.

[0149] The aforementioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC). When the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode, the microphone is configured to receive external audio signals. The received audio signals can be further stored in a memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0150] Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present disclosure may take the form of one or more computer-readable storage media containing computer-usable program code. The form of a computer program product implemented on a disk storage device (including but not limited to a disk storage device, a compact disc read-only memory (CD-ROM), an optical storage device, etc.).

[0151] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, such that execution of the instructions by the processor of the computer or other programmable data processing device produces means for implementing the functions specified in one or more processes in the flowcharts and / or one or more blocks in the block diagrams.

[0152] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in one or more processes in the flowchart and / or one or more blocks in the block diagram.

[0153] These computer program instructions can also be loaded into a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing one or more processes in the flowchart and / or one or more blocks in the block diagram. The steps of the function specified in .

[0154] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interfaces, network interfaces, and memory.

[0155] Memory may include non-persistent storage in a computer-readable medium in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0156] Computer-readable media, including permanent and non-permanent, removable and non-removable media, can be used to store information using any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPM), and other types of computer-readable storage media. Computer-readable media (EEPROM), flash memory or other memory technology, CD-ROM, Digital Versatile Disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0157] It should also be noted that the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, product, or apparatus comprising a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, product, or apparatus. In the absence of further limitations, the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, product, or apparatus comprising the elements.

[0158] The above are merely examples of the present disclosure and are not intended to limit the present disclosure. Persons skilled in the art will readily appreciate that various modifications and variations are possible with the present disclosure. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present disclosure are intended to be encompassed by the claims of the present disclosure.

Claims

Claims 1. A data transmission method based on RDMA, applied to a physical machine on which a first application is deployed, the method comprising: When a first application initiates an RDMA communication request to a second application, a first RDMA work queue is created for the first application and registered in memory, where the first RDMA work queue includes a local work queue and a remote work queue; based on a control plane connection between the first application and the second application, a description of the second RDMA work queue corresponding to the second application is obtained as context information of the first RDMA work queue; Based on the description information of the second RDMA work queue, it is determined whether the first application and the second application perform intra-host RDMA communication. The second RDMA work queue also includes a local work queue and a remote work queue. If intra-host RDMA communication is determined, a data copy operation is performed in memory registered by the two applications based on the local work queue of at least one of the two applications, replacing the RDMA data transfer operation between the two applications.

2. The method according to claim 1, wherein: In a case where intra-host RDMA communication is identified, the first application and the second application are applications deployed in different virtualized instances on the physical machine; the method further includes: when the first application initiates an RDMA communication request to the second application, creating a second RDMA work queue for the second application and registering it in memory; and obtaining, based on the control plane connection, description information of the first RDMA work queue as context information of the second RDMA work queue.

3. The method according to claim 1, further comprising: In the case of cross-host RDMA communication, an RDMA data transmission operation is performed between the memories registered by the two applications through the RDMA network card on the physical machine based on the remote work queue of at least one of the two applications.

4. The method according to claim 1, wherein: When a first application initiates an RDMA communication request to a second application, a first RDMA work queue is created for the first application and memory registration is performed. This includes: responding to a completion queue (CQ) creation request initiated by the first application to create a local CQ queue and a remote CQ queue for the first application; responding to a queue pair (QP) creation request initiated by the first application to create a local QP queue and a remote QP queue for the first application; responding to a memory registration request initiated by the first application to register memory for the first application and save index information of the registered memory. The local work queue includes a local CQ queue and a local QP queue, and the remote work queue includes a remote CQ queue and a remote QP queue.

5. The method according to claim 4, wherein: The remote work queue has a queue pair number QPN and a global identity GID. The description information of the RDMA work queue to which the remote work queue belongs includes the QPN and GID of the remote work queue; the local work queue does not have a QPN and GID, or the local work queue shares the QPN and GID of the remote work queue in the RDMA work queue to which it belongs.

6. The method according to claim 5, wherein: Identifying, based on the description information of the second RDMA work queue, whether the first application and the second application perform same-host RDMA communication, includes: determining, based on the GID in the description information of the second RDMA work queue, the IP address of the second application; in a single-tenant scenario, identifying, based on a routing table of the physical machine and the IP address of the second application, whether the first application and the second application perform same-host RDMA communication; and in a multi-tenant scenario, identifying, based on a flow table of a virtual switch on the physical machine and the IP address of the second application, whether the first application and the second application perform same-host RDMA communication.

7. The method according to any one of claims 1 to 6, wherein: When intra-host RDMA communication is identified, a data copy operation is performed in the memory registered by the two applications based on the local work queue of at least one of the two applications, thereby replacing the RDMA data transfer operation between the two applications. The method includes: when intra-host RDMA communication is identified, in user state, in response to a data sending task initiated by a sending end, generating an SQE, and writing the SQE into a local SQ queue in the local work queue of the sending end; entering kernel state from user state, and in kernel state, performing a data copy operation in the memory registered by the two applications using the local work queue of the sending end or both the sending and receiving ends according to the operation type of the data sending task, thereby replacing the RDMA data transfer operation between the two applications; wherein the sending end is any one of the two applications, and the other application is the receiving end.

8. The method according to claim 7, wherein: The method of entering the kernel state from the user state includes: responding to a system call initiated by the user state after writing the SQE into the local SQ queue in the local work queue of the sending end, and entering the kernel state from the user state; or The command queue shared by the user state and the kernel state is polled, and when the command queue contains a local memory copy command, the kernel state is entered from the user state; wherein the local memory copy command is written into the command queue after the user state writes the SQE into the local SQ queue in the local work queue of the sending end.

9. The method according to claim 7, wherein: In kernel state, based on the operation type of the data sending task, a data copy operation is performed in memories registered by two applications using a local work queue on the sending end or on both ends of the sending and receiving end, thereby replacing the RDMA data transmission operation between the two applications. This includes: in kernel state, obtaining the SQE from the local SQ queue on the sending end, and obtaining the operation type of the data sending task from the SQE; if the data sending task is a bilateral operation, performing the data copy operation in memories registered by two applications using the local work queues on both ends of the sending and receiving end, thereby replacing the RDMA data transmission operation between the two applications; if the data sending task is a unilateral operation, performing the data copy operation in memories registered by two applications using the local work queue on the sending end, thereby replacing the RDMA data transmission operation between the two applications.

10. The method according to claim 9, wherein: When the data sending task is a bilateral operation, the method further includes: in user mode, responding to the data receiving task initiated by the receiving end, generating an RQE, and writing the RQE into a local RQ queue in a local work queue of the receiving end; using the local work queues at both ends of the sending and receiving end to perform a data copy operation in memories registered by two applications to replace the RDMA data transmission operation between the two applications, including: determining description information of the RDMA work queue of the receiving end based on context information of the RDMA work queue of the sending end; reading an SQE from the local SQ queue of the sending end, and reading an RQE from the local RQ of the receiving end based on the description information of the RDMA work queue of the receiving end; copying data in the memory pointed to by the local memory index information contained in the SQE to the memory pointed to by the local memory index information contained in the RQE; after completing the data copy, generating a CQE, and writing the CQE into the local CQ queues in the local work queues of the sending end and the receiving end, respectively.

11. The method according to claim 9, wherein: Using the local work queue of the sending end, a data copy operation is performed in the memories registered by the two applications to replace the RDMA data transmission operation between the two applications, including: if the operation type of the data sending task is a unilateral write operation, reading an SQE from the local SQ queue of the sending end, and copying the data in the memory pointed to by the local memory index information contained in the SQE to the memory pointed to by the remote memory index information contained in the SQE; if the operation type of the data sending task is a unilateral read operation, reading an SQE from the local SQ queue of the sending end, and copying the data in the memory pointed to by the remote memory index information contained in the SQE to the memory pointed to by the local memory index information contained in the SQE; after completing the data copy, generating a CQE, and writing the CQEs into the local CQ queues in the local work queue of the sending end.

12. The method according to claim 9, wherein: Performing a data copy operation in two application-registered memories includes: executing a memory copy command of a processor CPU to copy data from a memory registered by one application to a memory registered by another application; or using a DMA acceleration engine of a CPU to copy data from a memory registered by one application to a memory registered by another application.

13. A physical machine, wherein: The physical machine includes an RDMA network card, and a first application and an RDMA management program are deployed on the physical machine, wherein the RDMA management program includes: a user-state interface management module and a kernel-state driver management module; the driver management module cooperates with the interface management module to: create a first RDMA work queue for the first application and register it in memory when the first application initiates an RDMA communication request to the second application; obtain description information of a second RDMA work queue corresponding to the second application based on a control plane connection between the first application and the second application, as context information of the first RDMA work queue; the first and second RDMA work queues both include a local work queue and a remote work queue; and identify whether the first application and the second application are communicating with the same host through RDMA based on the description information of the second RDMA work queue; and When it is identified as intra-host RDMA communication, a data copy operation is performed in the memory registered by the two applications based on the local work queue of at least one of the two applications, replacing the RDMA data transmission operation between the two applications.

14. The physical machine according to claim 13, wherein: In the case of intra-host RDMA communication, the first application and the second application are applications deployed in different virtualized instances on the physical machine; the driver management module is further configured to: when the first application initiates an RDMA communication request to the second application, create a second RDMA work queue for the second application and register it in memory; and obtain, based on the control plane connection, description information of the first RDMA work queue as context information of the second RDMA work queue.

15. The physical machine according to claim 13, wherein: The driver management module is further configured to provide the interface management module with an identification result of whether the communication is intra-host RDMA communication; the interface management module is further configured to: when the identification result is inter-host RDMA communication, perform an RDMA data transmission operation between the memories registered by the two applications through the RDMA network card based on the remote work queue of at least one of the two applications.

16. The physical machine according to claim 13, wherein: The interface management module includes a resource management unit, and the driver management module includes a transmission service unit; the resource management unit is configured to respond to a completion queue (CQ) creation request, a queue pair (QP) creation request, and a memory registration request initiated by the first application, and initiate calls to the transmission service unit respectively; the transmission service unit is configured to respond to the calls of the resource management unit, create a local CQ queue and a remote CQ queue, a local QP queue, and a remote QP queue for the first application, register memory for the first application, and save index information of the registered memory; wherein the local work queue includes a local CQ queue and a local QP queue, and the remote work queue includes a remote CQ queue and a remote QP queue.

17. The physical machine according to any one of claims 13 to 16, wherein: The interface management module further includes: a data transceiver unit; the driver management module includes: a transmission service unit; the data transceiver unit is configured to, when identifying intra-host RDMA communication, respond to a data transmission task initiated by a sending end, generate an SQE, write the transmission task description information into a local SQ queue in a local work queue of the sending end, and trigger the transmission service unit; the transmission service unit is configured to: under the triggering of the data transceiver unit, utilize the local work queue of the sending end or both the sending and receiving ends to perform a data copy operation in the memory registered by the two applications based on the operation type of the data transmission task, thereby replacing the RDMA data transmission operation between the two applications; wherein the sending end is any one of the two applications, and the other application is the receiving end.

18. A physical machine, comprising: A memory and a processor; wherein the memory is used to store a computer program, and the processor is coupled to the memory and is used to execute the computer program to implement the steps in the method according to any one of claims 1 to 12.

19. A computer-readable storage medium storing a computer program, wherein: When the computer program is executed by a processor, the processor is enabled to implement the steps of the method according to any one of claims 1 to 12.

20. A computer program product comprising a computer program / instructions, wherein: When the computer program / instructions are executed by a processor, the processor is enabled to implement the steps of the method according to any one of claims 1 to 12.

Images