A DPDK-based virtual rdma device and method of using the same

Abstract

The application discloses a virtual RDMA device based on DPDK and a use method thereof. The virtual RDMA device based on DPDK comprises a front-end virtio-rdma device, which is used for receiving an RDMA device operation request initiated by an application program in a virtual machine, encapsulating the request as an instruction conforming to a virtio protocol, and sending the instruction to a back-end vhost-user- rdma device through a vring; and the back-end vhost-user- rdma device is used for simulating the function of a real RDMA network card, the device can acquire and analyze the instruction conforming to the virtio protocol from a circular queue vring, execute corresponding RDMA device operations, and return the operation results to the front-end virtio-rdma device through the vring. The application can get rid of the dependence on the real RDMA device, and can improve the resource utilization, system flexibility and safety.

Description

Technical Field

[0001] This invention relates to the field of virtualization system technology, and more specifically to a virtual RDMA device based on DPDK and its usage method. Background Technology

[0002] RDMA (Remote Direct Memory Access) is a network communication technology that allows computers to transfer data directly between host memory without going through the operating system and CPU. It features low latency and high throughput, and is widely used in high-performance computing, cloud computing, and big data processing. RDMA is typically implemented through dedicated network interface cards, such as InfiniBand or Ethernet devices that support RDMA over Converged Ethernet (RoCE).

[0003] Virtualization technology creates multiple virtual machines (VMs) on top of physical hardware, enabling multiple operating systems to share the same physical resources. With the widespread adoption of virtualization, fully leveraging the advantages of Redirect DMA (Remote Data Memory) in virtual environments has become increasingly important. In traditional virtualization environments, the performance of RDMA is often not fully realized due to the overhead introduced by the hypervisor. Limited direct communication between VMs and RDMA devices leads to increased transmission latency. Therefore, developing a solution that effectively integrates RDMA and virtualization technologies is a current research hotspot.

[0004] Currently, in virtualization environments, virtual machines use RDMA devices primarily in the following ways: (1) Passthrough method: The physical RDMA network card is directly assigned to the virtual machine. Although the performance of this method is close to that of bare metal, each RDMA device can only be assigned to one virtual machine, resulting in low resource utilization and difficulty in migrating virtual machines.

[0005] (2) SRIOV (Single Root I / O Virtualization): This method uses hardware virtualization technology to virtualize a physical RDMA network card into multiple virtual functions (VFs), which are then allocated to different virtual machines. This method improves resource utilization, but it relies on hardware support, has complex configuration, and a limited number of virtual functions.

[0006] (3) Shared memory method: RDMA communication between the virtual machine and the host machine is achieved through a shared memory mechanism, such as the virtualized RDMA method, system, storage medium and electronic device technical solution disclosed in Chinese patent document CN113867993A. This method requires frequent context switching between the virtual machine and the hypervisor, resulting in high CPU overhead, low efficiency and performance, and insufficient security of the shared memory mechanism. Summary of the Invention

[0007] The technical problem to be solved by this invention is to provide a virtual RDMA device based on DPDK and its usage method, which addresses the above-mentioned problems of existing virtual machines using RDMA devices, such as low efficiency, many dependencies, and low system security and flexibility. This invention can get rid of the dependence on real RDMA devices and improve resource utilization, system flexibility and security.

[0008] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A virtual RDMA device based on DPDK, comprising: The front-end virtio-rdma device is set up in the virtual machine and presented as a standard RDMA device. It is used to receive RDMA device operation requests initiated by applications in the virtual machine, encapsulate the requests into instructions that conform to the virtio protocol, and send them to the back-end vhost-user-rdma device through a circular queue vring. The backend vhost-user-rdma device is set up in the user space of the host machine and implemented based on the DPDK framework. It is used to simulate the function of a real RDMA network card. This device can obtain and parse instructions conforming to the virtio protocol from the circular queue vring, execute the corresponding RDMA device operation, and return the operation result to the frontend virtio-rdma device through vring.

[0009] Optionally, the front-end virtio-rdma device is provided by a hypervisor, simulating a PCI-type RDMA device. The configuration space of the virtio-rdma device contains parameters for identifying the capabilities of the RDMA device, including: the number of physical ports, a globally unique identifier for the system image, vendor ID, part number, hardware version number, maximum size supported by a single memory region, maximum number of queue pairs, maximum number of work requests supported by each queue pair, maximum number of scatter aggregate elements supported by sending / receiving a single work request, maximum number of completion queues, maximum number of memory registered regions, maximum number of protection domains, maximum number of memory windows, and multicast-related parameters.

[0010] Optionally, the DPDK-based virtual RDMA device further includes a virtio-rdma driver running in the virtual machine kernel. The virtio-rdma driver and the front-end virtio-rdma device transmit data through the virtio protocol specification and a circular queue (vring). The virtio-rdma driver is used to register the virtual RDMA device to provide a standard programming interface to the upper-layer application, enabling the application to initiate operation requests to the front-end virtio-rdma device through this interface. The driver encapsulates the RDMA operation requests initiated by the application within the virtual machine into corresponding instructions conforming to the virtio protocol and transmits them to the front-end virtio-rdma device.

[0011] Optionally, the front-end virtio-rdma device and the back-end vhost-user-rdma device negotiate virtio features and initialize RDMA device parameters through a socket control channel.

[0012] Optionally, the backend vhost-user-rdma device uses DPDK's multi-threading and CPU affinity to poll the status of the circular queue vring. When there is new data in the circular queue vring, it attempts to retrieve the new data from the circular queue vring and parse it. If the new data is an instruction that conforms to the virtio protocol, it executes the corresponding RDMA device operation.

[0013] Optionally, when the backend vhost-user-rdma device starts up, it allocates mempool memory pool resources on the host machine to complete RDMA resource initialization; when processing RDMA resource operation requests from the frontend virtio-rdma device, the backend vhost-user-rdma device allocates the corresponding RDMA resource structure from the mempool memory pool and establishes and maintains a one-to-one mapping relationship between the virtual RDMA resource identifier in the virtual machine and the real resource structure allocated on the host machine side.

[0014] Optionally, allocating mempool memory pool resources on the host machine to complete RDMA resource initialization includes allocating mempool memory pool resources for RDMA queue pairs, completion queues, protection domains, and memory regions.

[0015] This invention further discloses a method for using a virtual RDMA device based on DPDK, comprising the following steps: S01, Start the backend vhost-user-rdma device in the host user space using DPDK to complete RDMA resource initialization; S02, the Hypervisor generates a front-end virtio-rdma device and adds it to the virtual machine, establishes a control channel between the front-end virtio-rdma device and the back-end vhost-user-rdma device, and completes virtio feature negotiation and RDMA device parameter initialization; S03, the virtual machine loads the virtio-rdma driver, which completes the binding and initialization with the front-end virtio-rdma device, completes the registration of the virtual RDMA device, registers the virtual RDMA device as a standard InfiniBand device, generates the corresponding standard device node for the application to use, completes the initialization and configuration of the ring queue vring, and establishes a data channel with the back-end vhost-user-rdma device; S04. After the application initiates an RDMA device operation request using the RDMA device, the virtio-rdma driver will encapsulate the RDMA device operation request into a corresponding instruction conforming to the virtio protocol, and transmit it to the backend vhost-user-rdma device through the circular queue vring via the frontend virtio-rdma device. S05, when the backend vhost-user-rdma device detects new data in the circular queue vring, it attempts to retrieve the new data from the circular queue vring and parse it. If the new data conforms to the virtio protocol, it retrieves the corresponding instructions and executes the corresponding RDMA device operation. After the backend vhost-user-rdma device completes the corresponding RDMA device operation, it fills the processing result into the circular queue vring and notifies the frontend virtio-rdma driver to feed back the processing result to the application.

[0016] Optionally, the virtio-rdma driver encapsulates the RDMA device operation request into a corresponding instruction conforming to the virtio protocol, encapsulates the instruction in a scatterlist, and then transmits the scatterlist to the backend vhost-user-rdma device through the circular queue vring via the frontend virtio-rdma device.

[0017] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the method of using the DPDK-based virtual RDMA device by a processor.

[0018] Compared with existing technologies, the DPDK-based virtual RDMA device of the present invention has the following advantages: (1) This invention solves the problems of strong hardware dependence, complex configuration, and poor flexibility in pass-through and SR-IOV methods. The pass-through method exclusively uses physical devices, while the SR-IOV method depends on hardware and has a limited number of virtual functions. Both result in low resource utilization and complex configuration and management. This invention simulates the function of a real RDMA network card by using a backend vhost-user-rdma device implemented entirely based on DPDK software. This allows the system to break free from dependence on specific physical hardware and can flexibly create and configure multiple virtual RDMA devices based on DPDK as needed, thereby improving resource utilization and system scalability.

[0019] (2) This invention solves the problems of low performance and insufficient security of shared memory. Shared memory requires frequent context switching, resulting in high CPU overhead, low efficiency, and security risks. This invention, through a front-end virtio-rdma device and a back-end vhost-user-rdma device, enables efficient (using the DPDK vhost-user mechanism) zero-copy data communication using the virtio protocol and the vring circular queue. This constructs a direct path from the virtual machine application to the host machine's user-space processing engine, reducing context switching between the hypervisor and the virtual machine, and lowering CPU overhead. Furthermore, when multiple DPDK-based virtual RDMA devices of this invention are deployed and assigned to different virtual machines, the DPDK vhost-user mechanism provides effective isolation between the virtual machines and the host machine, reducing interference and attack surfaces between different virtual machines on the same host, thus improving security. Moreover, vhost supports dynamic addition or removal of virtual devices at runtime, greatly improving flexibility.

[0020] The method for using the DPDK-based virtual RDMA device of the present invention has the following advantages: (1) A standardized and flexibly deployable method for using virtual RDMA devices is provided, improving manageability. This method clarifies the complete software-defined process from backend initialization, frontend device generation, driver loading negotiation to request processing, forming a standardized method for enabling and managing virtual machines. This avoids the complex configuration and binding of physical hardware in passthrough or SR-IOV methods, making the enabling of virtual RDMA functions and the dynamic allocation of resources simple and unified, and greatly improving the manageability of virtual machines.

[0021] (2) A highly efficient and reliable virtual RDMA operation processing closed loop was implemented, ensuring performance and consistency. This method ensures that RDMA operations can be completed efficiently and controllably in the virtualized environment by encapsulating application requests, transmitting them via vring, parsing and executing them on the backend, and returning the results via vring. This processing closed loop not only provides low-latency processing capabilities close to those of physical devices, but also ensures that every RDMA operation initiated by the application within the virtual machine is accurately executed and receives status feedback through a clear result return and notification mechanism, thus guaranteeing the reliability of the operation. Attached Figure Description

[0022] Figure 1 This is a virtualization architecture diagram of the virtual RDMA device based on DPDK according to the present invention.

[0023] Figure 2 This is a flowchart illustrating the usage method of the virtual RDMA device based on DPDK according to the present invention. Detailed Implementation

[0024] DPDK: Short for Data Plane Development Kit, it's a software development kit for fast packet processing, widely used in high-performance network application development. It provides a set of libraries and drivers that allow developers to bypass the Linux kernel network stack and directly manipulate the network interface card (NIC) hardware, thereby achieving high-performance packet processing.

[0025] HYPERVISOR: A virtual machine monitor is a key component in virtualization technology. It is responsible for providing an abstraction layer between physical hardware (Host) and virtual machines (Guest), allowing multiple virtual machines to run in parallel on the same physical server.

[0026] GUEST: Guest refers to a virtual machine. It is managed by a virtual machine monitor (Hypervisor) and can run various operating systems.

[0027] HOST: Host machine. In a virtualization environment, Host refers to the physical machine or operating system that runs the hypervisor and hosts virtual machines.

[0028] RDMA (Remote Direct Memory Access) is a network communication technology that allows one computer's memory to directly read and write to another computer's memory without intervention from the other computer's CPU and with minimal consumption of the other computer's system resources. This technology significantly reduces communication latency and improves data transmission efficiency, and is widely used in high-performance computing (HPC), distributed storage, cloud computing, and AI training. VIRTIO: Virtio is a virtualization standard used to provide efficient I / O between virtual machines and hosts. It defines a set of common devices and interfaces, facilitating the implementation of high-performance virtualization devices on different virtualization platforms.

[0029] VRING: Vring is a data structure in Virtio used to manage shared buffers between Virtio devices and virtual machines. It consists of a pair of descriptor rings and available / used rings.

[0030] RDMA APP: refers to an application that runs within a virtual machine and requires RDMA devices for data transmission and reception.

[0031] The virtio-net driver is a regular virtio-net network device driver that runs within a virtual machine.

[0032] virtio-net device: A virtio-net device created based on Hypervisor, which is a standard device that supports the virtio protocol.

[0033] The technical solution of the present invention will now be described in further detail with reference to the accompanying drawings.

[0034] Example 1 like Figure 1 As shown, the DPDK-based virtual RDMA device in this embodiment includes: The front-end virtio-rdma device is set up in the virtual machine and presented as a standard RDMA device. It is used to receive RDMA device operation requests initiated by applications in the virtual machine, encapsulate the requests into instructions that conform to the virtio protocol, and send them to the back-end vhost-user-rdma device through the circular queue vring. The backend vhost-user-rdma device is set up in the user space of the host machine and implemented based on the DPDK framework. It is used to simulate the function of a real RDMA network card. This device can obtain and parse instructions conforming to the virtio protocol from the circular queue vring, execute the corresponding RDMA device operations, and return the operation results to the frontend virtio-rdma device through vring.

[0035] This embodiment of the DPDK-based virtual RDMA device first addresses the problems of strong hardware dependence, complex configuration, and poor flexibility associated with pass-through and SR-IOV methods. It simulates the function of a real RDMA network card through a backend vhost-user-rdma device implemented entirely in DPDK software, enabling the system to break free from dependence on specific physical hardware. Multiple DPDK-based virtual RDMA devices can be flexibly created and configured as needed, significantly improving resource utilization and system scalability. Secondly, it solves the problems of low performance and insufficient security in shared memory methods. Shared memory requires frequent context switching, resulting in high CPU overhead, low efficiency, and security risks. This invention, through a frontend virtio-rdma device and a backend vhost-user-rdma device, utilizes the virtio protocol and a circular queue (vring) for efficient (using the DPDK vhost-user mechanism) zero-copy data communication, constructing a direct path from the virtual machine application to the host machine's user-space processing engine, significantly reducing latency and CPU overhead. Meanwhile, thanks to the DPDK vhost-user mechanism, each virtual RDMA device backend runs as an independent user-space process, providing effective process-level isolation for different virtual machines. This reduces interference and attack surfaces caused by shared resources between different virtual machines on the same host, thus improving security. Furthermore, vhost supports dynamically adding or removing virtual devices at runtime, greatly enhancing flexibility.

[0036] Furthermore, in this embodiment, the front-end virtio-rdma device is provided by a hypervisor, simulating a PCI-type RDMA device. The configuration space of the virtio-rdma device contains parameters for identifying the capabilities of the RDMA device, including: The number of physical ports is used to identify how many physical ports the RDMA device uses; A globally unique identifier for the system image, used to identify the entire RDMA subsystem instance; Manufacturer ID, part number, and hardware version number are used to identify the device manufacturer and model. The maximum size (max_mr_size) that a single memory region can support; Maximum number of queue pairs (max_qp); The maximum number of work requests supported per memory region (QP); The maximum number of scatter-gather elements supported by a single WR for sending / receiving; Maximum number of completion queues; Maximum number of memory-registered regions (max_mr); Maximum number of protection domains (max_pd); Maximum number of memory windows (max_mw); Multicast-related parameter groups include the number of QPs that each group can connect to and the total number of connections.

[0037] Furthermore, in this embodiment, the DPDK-based virtual RDMA device also includes a virtio-rdma driver running on the virtual machine kernel. The virtio-rdma driver and the front-end virtio-rdma device transmit data via the virtio protocol specification and a circular queue (vring). The virtio-rdma driver is used to perform the following: The registration of a virtual RDMA device provides a standard programming interface to upper-layer applications, enabling them to initiate operation requests to the front-end virtio-rdma device. During driver loading, its specific operation functions are "associated" with the operating system kernel's standard RDMA interface functions and bound to the front-end virtio-rdma device model. Subsequently, the kernel's standard RDMA library is invoked to register the virtual RDMA device as a standard InfiniBand (IB) device, and a corresponding control node (e.g., / dev / infiniband / uverbsX) is created in the file system. This node provides the standard user-space RDMA library (rdma-core) and applications (APPs) with a single standard entry point for discovering, identifying, and operating the virtual RDMA device. Through this registration process, a standard RDMA Verbs programming interface is provided to upper-layer applications. This allows applications to initiate operation requests to the front-end virtio-rdma device through this interface, just as they would with a physical RDMA network card. The RDMA operation request initiated by the application within the virtual machine is encapsulated into a corresponding instruction conforming to the virtio protocol and transmitted to the front-end virtio-rdma device, which then forwards it to the back-end vhost-user-rdma device for processing. Specifically, the workflow is as follows: When an application initiates an RDMA resource operation request (such as calling ibv_create_qp to create a queue pair), the request is processed by the standard RDMA interface function associated in the driver. Inside this function, the driver converts (or translates) the semantics (operation type) and parameters of this call into proprietary VIRTIO_CMD_XX command words (e.g., VIRTIO_CMD_CREATE_QP) conforming to the virtio protocol, defined internally by this invention, and their corresponding message payload. This step completes the conversion from the standard API to the private protocol. Subsequently, the virtio-rdma device encapsulates the VIRTIO_CMD_XX command words and corresponding message content obtained in the previous step into a scatterlist. This scatterlist, which encapsulates internal instructions, is passed through the virtqueue and vring channels, and the front-end device notifies (kicks) the back-end DPDKvhost-user-rdma device, thereby initiating the back-end to actually process the request.

[0038] Furthermore, in this embodiment, the front-end virtio-rdma device and the back-end vhost-user-rdma device negotiate virtio features and initialize RDMA device parameters through a socket control channel.

[0039] To achieve rapid response to commands from the front end, in this embodiment, the backend vhost-user-rdma device utilizes DPDK's multi-threading and CPU affinity to poll the status of the circular queue (vring). When new data is available in the vring, it attempts to retrieve and parse the new data. If the new data is a command conforming to the virtio protocol, the corresponding RDMA device operation is executed. This better utilizes system resources and significantly improves the manageability of the virtual machine.

[0040] Furthermore, in this embodiment, when the backend vhost-user-rdma device starts up, it allocates mempool memory pool resources on the host machine to initialize RDMA resources. When processing RDMA resource operation requests from the frontend virtio-rdma device, the backend vhost-user-rdma device allocates the corresponding RDMA resource structure from the mempool memory pool and establishes and maintains a one-to-one mapping relationship between the virtual RDMA resource identifiers within the virtual machine and the actual resource structures allocated on the host machine. This mapping ensures that resource requests in the virtual machine environment are accurately reflected in the host machine environment, ensuring the consistency and continuity of resource allocation.

[0041] Furthermore, in this embodiment, allocating mempool resources on the host machine to complete RDMA resource initialization includes allocating mempool resources for RDMA queue pairs (QP), completion queues (CQ), protection domains (PD), and memory regions (MR). This operation lays the resource foundation for the high-performance and deterministic operation of the entire virtual RDMA device. By pre-allocating dedicated and efficient memory pools for core data structures such as QP, CQ, PD, and MR through the DPDK's mempool mechanism, the backend vhost-user-rdma device can complete memory allocation from the pool at near-zero latency when processing virtual machine resource requests, completely avoiding the overhead, latency uncertainty, and memory fragmentation problems caused by the operating system's dynamic memory management (such as malloc). This improves the efficiency of resource creation, reduces I / O latency, and ensures that the backend vhost-user-rdma device can operate stably and continuously at near-physical hardware efficiency.

[0042] The virtualization architecture diagram of the DPDK-based virtual RDMA device in this embodiment is as follows: Figure 1 As shown: Inside the virtual machine (Guest OS), applications that require RDMA services (RDMA APPs) run. These applications initiate requests through the kernel's standard RDMA library libibverbs (part of the rdma-core software stack) and interact with the kernel via the standard device node / dev / infiniband / uverbsX in the file system. The virtual machine kernel runs the virtio-rdma driver and the standard virtio-net driver for handling regular network I / O (such as TCP / IP protocol stack traffic) within the virtual machine. In the middle virtualization management layer (Hypervisor), the virtual machine monitor provides the corresponding virtual hardware abstraction, namely the front-end virtio-rdma device and the virtio-net device. The front-end virtio-rdma device is the standard RDMA hardware interface presented to the virtual machine. The virtio-net device, in conjunction with the virtio-net driver, provides the virtual machine with traditional network connectivity capabilities. Efficient communication between the front-end and back-end relies on the circular queue vring; The host userspace runs the backend vhost-user-rdma device. It efficiently manages the vring through the vhost-user library and is an independent process that actually simulates the functions of a real RDMA network card and handles all RDMA requests. The tap0 virtual network interface interfaces with the virtio-net path, handling regular network packets from the virtual machine, demonstrating the separation and coexistence of the RDMA path and the standard network path in this embodiment. The host kernel space (KVM module) provides basic CPU and memory virtualization support.

[0043] The crucial "kick" and "call" arrows in the diagram precisely describe the cross-layer notification mechanism: Kick (trigger): When the front-end driver in the virtual machine places the request data into the vring, it will send a kick notification. This signal passes through the Hypervisor layer and reaches the back-end vhost-user-rdma device in the host user space, informing it that there is a new request to be processed.

[0044] call: After the backend vhost-user-rdma device finishes processing the request and puts the result into the vring, it will notify the virtio-rdma driver that the result is ready through the call mechanism via the KVM module of the host kernel.

[0045] Example 2 like Figure 2 As shown, the method for using a DPDK-based virtual RDMA device in this embodiment includes the following steps: S01, Start the backend vhost-user-rdma device in the host user space using DPDK to complete RDMA resource initialization; S02, the Hypervisor generates a front-end virtio-rdma device and adds it to the virtual machine, establishes a control channel between the front-end virtio-rdma device and the back-end vhost-user-rdma device, and completes virtio feature negotiation and RDMA device parameter initialization; S03, the virtual machine loads the virtio-rdma driver, which completes the binding and initialization with the front-end virtio-rdma device, completes the registration of the virtual RDMA device, registers the virtual RDMA device as a standard InfiniBand device, generates the corresponding standard device node for the application to use, completes the initialization and configuration of the ring queue vring, and establishes a data channel with the back-end vhost-user-rdma device; S04. After the application initiates an RDMA device operation request using the RDMA device, the virtio-rdma driver will encapsulate the RDMA device operation request into a corresponding instruction conforming to the virtio protocol, and transmit it to the backend vhost-user-rdma device through the circular queue vring via the frontend virtio-rdma device. S05, when the backend vhost-user-rdma device detects new data in the circular queue vring, it attempts to retrieve the new data from the circular queue vring and parse it. If the new data conforms to the virtio protocol, it retrieves the corresponding instructions and executes the corresponding RDMA device operation. After the backend vhost-user-rdma device completes the corresponding RDMA device operation, it fills the processing result into the circular queue vring and notifies the frontend virtio-rdma driver to feed back the processing result to the application.

[0046] This embodiment provides a standardized and flexibly deployable method for using virtual RDMA devices based on DPDK, improving manageability. The method clarifies the complete software-defined process from backend initialization, frontend device generation, driver loading negotiation to request processing, forming a standardized activation and management method. This avoids the complex configuration and binding of physical hardware in passthrough or SR-IOV methods, making the activation of virtual RDMA functions and dynamic resource allocation simple and unified, significantly improving the manageability of virtual machines; it also achieves an efficient and reliable virtual RDMA operation processing closed loop, ensuring performance and consistency. By encapsulating application requests, transmitting them via Vring, parsing and executing them on the backend, and returning the results via Vring, this method ensures that RDMA operations can be completed efficiently and controllably in the virtualization environment. This processing closed loop not only provides near-physical device low-latency processing capabilities but also ensures that every RDMA operation initiated by an application within the virtual machine is accurately executed and receives status feedback through a clear result return and notification mechanism, guaranteeing operational reliability.

[0047] Furthermore, in this embodiment, the virtio-rdma driver encapsulates the RDMA device operation request into a corresponding instruction conforming to the virtio protocol, then encapsulates the instruction in a scatterlist. This scatterlist is then transmitted to the backend vhost-user-rdma device via a circular queue (vring) through the frontend virtio-rdma device. The virtio-rdma driver only fills the scatterlist with memory address descriptors containing instructions and parameters, rather than the data itself. This allows the backend device to directly access the original data through the descriptors, reducing the overhead of copying memory data between the frontend and backend. This, in turn, reduces I / O latency for applications within the virtual machine and significantly improves CPU utilization efficiency.

[0048] Furthermore, this embodiment also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the above-described method of using a DPDK-based virtual RDMA device via a processor.

[0049] Those skilled in the art will understand that the technical solutions provided by the embodiments of this application may be in the form of a method, system, or computer program product. Therefore, this application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application may take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create an implementation for the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0050] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A virtual RDMA device based on DPDK, characterized in that, include: The front-end virtio-rdma device is set up in the virtual machine and presented as a standard RDMA device. It is used to receive RDMA device operation requests initiated by applications in the virtual machine, encapsulate the requests into instructions that conform to the virtio protocol, and send them to the back-end vhost-user-rdma device through a circular queue vring. The backend vhost-user-rdma device is set up in the user space of the host machine and implemented based on the DPDK framework. It is used to simulate the function of a real RDMA network card. This device can obtain and parse instructions conforming to the virtio protocol from the circular queue vring, execute the corresponding RDMA device operation, and return the operation result to the frontend virtio-rdma device through vring.

2. The virtual RDMA device based on DPDK according to claim 1, characterized in that, The front-end virtio-rdma device is provided by a hypervisor and simulates a PCI-type RDMA device. The configuration space of the virtio-rdma device contains parameters for identifying the capabilities of the RDMA device. These parameters include: number of physical ports, globally unique identifier for the system image, vendor ID, part number, hardware version number, maximum size supported by a single memory region, maximum number of queue pairs, maximum number of work requests supported by each queue pair, maximum number of scatter aggregate elements supported by sending / receiving a single work request, maximum number of completion queues, maximum number of memory registered regions, maximum number of protection domains, maximum number of memory windows, and multicast-related parameters.

3. The virtual RDMA device based on DPDK according to claim 1, characterized in that, The DPDK-based virtual RDMA device also includes a virtio-rdma driver running in the virtual machine kernel. The virtio-rdma driver and the front-end virtio-rdma device transmit data through the virtio protocol specification and the vring ring. The virtio-rdma driver is used to register the virtual RDMA device to provide a standard programming interface to the upper-layer application so that the application can initiate operation requests to the front-end virtio-rdma device through the interface, and encapsulate the RDMA operation requests initiated by the application in the virtual machine into corresponding instructions conforming to the virtio protocol before transmitting them to the front-end virtio-rdma device.

4. The virtual RDMA device based on DPDK according to claim 1, characterized in that, The front-end virtio-rdma device and the back-end vhost-user-rdma device negotiate virtio features and initialize RDMA device parameters through a socket control channel.

5. The virtual RDMA device based on DPDK according to claim 4, characterized in that, The backend vhost-user-rdma device uses DPDK's multi-threading and CPU affinity to poll the status of the circular queue vring. When there is new data in the circular queue vring, it attempts to retrieve the new data from the circular queue vring and parse it. If the new data is an instruction that conforms to the virtio protocol, it executes the corresponding RDMA device operation.

6. The virtual RDMA device based on DPDK according to claim 1, characterized in that, When the backend vhost-user-rdma device starts up, it allocates mempool memory pool resources on the host machine to complete RDMA resource initialization. When processing RDMA resource operation requests from the frontend virtio-rdma device, the backend vhost-user-rdma device allocates the corresponding RDMA resource structure from the mempool memory pool and establishes and maintains a one-to-one mapping relationship between the virtual RDMA resource identifier in the virtual machine and the real resource structure allocated on the host machine.

7. The virtual RDMA device based on DPDK according to claim 6, characterized in that, The host machine allocates mempool memory pool resources for RDMA resources to complete RDMA resource initialization, including allocating mempool memory pool resources for RDMA queue pairs, completion queues, protection domains, and memory regions.

8. A method of using a DPDK-based virtual RDMA device according to any one of claims 1 to 7, characterized in that, Includes the following steps: S01, Start the backend vhost-user-rdma device in the host user space using DPDK to complete RDMA resource initialization; S02, the Hypervisor generates a front-end virtio-rdma device and adds it to the virtual machine, establishes a control channel between the front-end virtio-rdma device and the back-end vhost-user-rdma device, and completes virtio feature negotiation and RDMA device parameter initialization; S03, the virtual machine loads the virtio-rdma driver, which completes the binding and initialization with the front-end virtio-rdma device, completes the registration of the virtual RDMA device, registers the virtual RDMA device as a standard InfiniBand device, generates the corresponding standard device node for the application to use, completes the initialization and configuration of the ring queue vring, and establishes a data channel with the back-end vhost-user-rdma device; S04. After the application initiates an RDMA device operation request using the RDMA device, the virtio-rdma driver will encapsulate the RDMA device operation request into a corresponding instruction conforming to the virtio protocol, and transmit it to the backend vhost-user-rdma device through the circular queue vring via the frontend virtio-rdma device. S05, when the backend vhost-user-rdma device detects new data in the circular queue vring, it attempts to retrieve the new data from the circular queue vring and parse it. If the new data conforms to the virtio protocol, it retrieves the corresponding instructions and executes the corresponding RDMA device operation. After the backend vhost-user-rdma device completes the corresponding RDMA device operation, it fills the processing result into the circular queue vring and notifies the frontend virtio-rdma driver to feed back the processing result to the application.

9. The method of using a virtual RDMA device based on DPDK according to claim 8, characterized in that, The virtio-rdma driver encapsulates the RDMA device operation request into a corresponding instruction conforming to the virtio protocol, then encapsulates the instruction in a scatterlist, and then transmits the scatterlist to the backend vhost-user-rdma device through the circular queue vring via the frontend virtio-rdma device.

10. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the method of using the DPDK-based virtual RDMA device as described in claim 8 or 9.

Images