A method and system for implementing lightweight virtualization based on RDMA technology

Abstract

The present invention provides a method and system for realizing lightweight virtualization based on RDMA technology, the method comprising: establishing a connection between an RDMA device based on RDMA technology and a host side; allocating an RDMA device interface to each container or virtual machine created on the host side; the RDMA device interface receiving a virtualization configuration driven from the host side; wherein the virtualization configuration includes a queue priority configuration; each RDMA device interface manages the queue in the RDMA device based on the received virtualization configuration; the container or virtual machine uses the MMIO register to map to the RDMA device, so that the container or virtual machine can access the RDMA device based on the RDMA device interface, thereby realizing virtualization based on RDMA technology. The present invention can simplify the complexity of the system, save the occupation of memory space and configuration space, and reduce costs.

Description

Technical Field

[0001] The present invention relates to the field of RDMA device virtualization I / O technology, and in particular to a method and system for realizing lightweight virtualization based on RDMA technology. Background Art

[0002] In the process of data packet processing, traditional TCP / IP technology needs to go through the operating system and other software layers, which requires a large amount of server resources and memory bus bandwidth. Data is copied and moved back and forth between system memory, processor cache and network controller cache, which puts a heavy burden on the server's CPU and memory. In particular, the serious "mismatch" between network bandwidth, processor speed and memory bandwidth exacerbates the network delay effect.

[0003] The principle of RDMA technology is to use the low latency characteristics provided by stack bypass and zero copy technology to reduce CPU usage, reduce memory bandwidth bottlenecks, and provide higher bandwidth utilization. RDMA technology has the advantages of very low latency, high throughput, high efficiency, and low CPU resource usage. RDMA technology provides an I / O-based channel that allows an application to directly read and write remote virtual memory through RDMA devices.

[0004] The hardware-based RDMA virtualized I / O technology currently mainly adopts the SR-IOV standard of PCIe. The SR-IOV standard allows a physical function (PF) to create multiple virtual functions (VF), which can be transparently transmitted to a virtual machine (VM), so that multiple virtual machines can share the same physical network card. According to the content of the SR-IOV standard, the new device created allows the virtual machine to be directly connected to the I / O device, bypassing the hypervisor and virtual switch layer, which can bring low latency and speed close to the cable. A single I / O resource can be shared by multiple virtual machines. The shared device will provide dedicated resources and also use shared common resources. In this way, each virtual machine can access unique resources. Therefore, PCIe devices with the SR-IOV standard enabled and with appropriate hardware and operating system support can be displayed as multiple separate physical devices, each with its own PCIe configuration space. Compared with IO simulation virtualization, virtualization based on the SR-IOV standard reduces I / O latency and CPU utilization, and data does not need to be forwarded through the host machine. Therefore, SR-IOV improves I / O performance and data security.

[0005] However, the RDMA virtualized I / O technology based on the SR-IOV standard has some disadvantages: the SR-IOV standard virtualizes VF devices (Visual Function) through hardware, which occupies independent configuration space and memory-mapped I / O space (MMIO). Each virtual device has an independent interrupt vector configuration, which makes the system occupy more memory space and configuration space resources, increases system complexity, and has high implementation costs. Summary of the invention

[0006] In view of this, an embodiment of the present invention provides a method and system for implementing lightweight virtualization based on RDMA technology, so as to eliminate or improve one or more defects existing in the prior art.

[0007] One aspect of the present invention provides a method for implementing lightweight virtualization based on RDMA technology, the method comprising the following steps:

[0008] The RDMA device based on RDMA technology establishes a connection with the host side; / / PCIe, server

[0009] For each container or virtual machine created on the host side, an RDMA device interface is allocated; wherein the host side operating system creates multiple containers or virtual machines based on the virtualization function, and the RDMA device interface includes MMIO registers in units of physical pages;

[0010] The RDMA device interface receives a virtualization configuration from a host-side driver; wherein the virtualization configuration includes a queue priority configuration;

[0011] Each RDMA device interface manages the queues in the RDMA device based on the received virtualization configuration;

[0012] The container or virtual machine uses the MMIO register to map to the RDMA device, so that the container or virtual machine can access the RDMA device based on the RDMA device interface, thereby realizing virtualization based on the RDMA technology.

[0013] In some embodiments of the present invention, the types of the RDMA devices include RDMA network cards and DPU acceleration cards supporting RDMA functions.

[0014] In some embodiments of the present invention, the method further includes: in each container or virtual machine created on the host side, using virtualization device operating software to manage the RDMA device interface and the host-side driver to virtualize the RDMA device into a virtual RDMA device that provides services to each container or virtual machine respectively.

[0015] In some embodiments of the present invention, the step of using virtualization device operating software to manage RDMA device interfaces and host-side drivers includes: on the host side, the virtualization device operating software sends configuration instructions to the host-side driver, so that the host-side driver generates a virtualization configuration for each RDMA device interface and sends it to the RDMA device.

[0016] In some embodiments of the present invention, the step of using virtualization device operating software to manage the RDMA device interface and the host-side driver also includes: the virtualization device operating software manages the resources on the RDMA side by mapping the physical page where the RDMA device interface is located, thereby abstracting an RDMA device for each container or virtual machine.

[0017] In some embodiments of the present invention, the virtualization configuration also includes multiple ones of scheduling configuration, virtual device configuration, software interface configuration, traffic control configuration, MAC address configuration and routing configuration.

[0018] In some embodiments of the present invention, the container and the virtual machine include a virtual memory area for storing and receiving data and a queue pair for sending and receiving data, each RDMA device interface in the RDMA device manages a completion queue for sending and receiving data, and direct access to remote virtual memory is achieved based on the virtual memory area, queue pair and completion queue.

[0019] In some embodiments of the present invention, in the step of each RDMA device interface managing the queue in the RDMA device based on the received virtualization configuration, the method further includes: each container or virtual machine configures the queue as exclusive or shared through an independently configured RDMA device interface.

[0020] Another aspect of the present invention provides a system for implementing lightweight virtualization based on RDMA technology, including a processor and a memory, wherein computer instructions are stored in the memory, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method described in any one of the above embodiments.

[0021] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the above embodiments.

[0022] The method and system for realizing lightweight virtualization based on RDMA technology proposed in the present invention can use RDMA devices based on RDMA technology to assist in building a virtualized I / O processing environment and reduce the pressure on the host side. On the one hand, the RDMA devices are uniformly configured through the host side operating system driver, and on the other hand, independent access is realized through the RDMA device interface separately allocated to each container or virtual machine, thereby streamlining the complexity of the system, saving memory space and configuration space occupation, reducing costs, and reducing the load pressure on the CPU on the host side.

[0023] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0024] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0026] Figure 1 The present invention is a flowchart of a method for implementing lightweight virtualization based on RDMA technology in one embodiment of the present invention.

[0027] Figure 2 The figure is a schematic diagram of the system structure for implementing lightweight virtualization based on RDMA technology in one embodiment of the present invention.

[0028] Figure 3 The figure is a schematic diagram of the system structure for implementing virtualized I / O based on RDMA devices in one embodiment of the present invention.

[0029] Figure 4 The figure is a diagram of the RDMA network card virtualization configuration relationship in one embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0031] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0032] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0033] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0034] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0035] In order to provide a new virtualization technology solution based on the existing technology, so as to achieve the minimum possible occupation of host-side CPU resources and memory resources, and to flexibly perform IO virtualization. The present invention proposes a method for realizing lightweight virtualization based on RDMA technology. The present invention aims to provide a new solution, which occupies less CPU resources and memory resources, and provides a set of flexible IO virtualization solutions. The present invention can reduce hardware complexity, reduce development and use costs, have good scalability and flexible configuration characteristics, and can meet the flexible and changeable needs of customers. Compared with the SR-IOV technical specifications, the lightweight virtualization IO implementation based on RDMA occupies less memory resources, simplifies the complexity of hardware implementation, and eliminates the impact of SR-IOV management configuration VF on the simultaneous use of the same RDMA device. The present invention occupies fewer resources, simplifies the design complexity, and is flexibly configured, which is suitable for flexible and changeable user needs.

[0036] Among them, Remote Direct Memory Access (RDMA) technology is created to solve the delay of server-side data processing in network transmission. RDMA technology directly transfers data to the computer's storage area through the network, quickly moves data from one system to the remote system memory without any impact on the operating system, and saves computer CPU computing resources and reduces power consumption. RDMA technology eliminates the CPU overhead of external memory copying and context switching, and can significantly improve the performance of application systems. RDMA technology is a technology that bypasses the kernel of the remote host operating system to access data in its memory. RDMA technology can directly transfer data from the memory of one computer to another computer without the intervention of the operating systems of both parties, so low-latency network communication can be achieved. Virtual Machine (VMs) is a virtual environment designed to simulate a complete physical computer system in order to run different operating systems and applications in it. Containers technology is a technology that packages applications and their dependencies together and runs them in an isolated environment. Container technology is similar to virtual machine technology, but it uses the virtualization function of the operating system to create containers instead of creating complete virtual machines. This makes containers more lightweight, more efficient, and easier to deploy and manage. The concept of containers originated from UNIX chroot in 1979, which is a command line tool for system calls on the UNIX operating system. It is used to change the root directory of a process and its child processes to a new location in the file system so that these processes can only access this directory. The idea of ​​this function is to provide independent disk space for each process. Later in 1982, it was added to the BSD system (a branch of the UNIX operating system).

[0037] The Single Root I / O Virtualization (SR-IOV) standard used in the prior art is an extension standard of the PCIe specification launched by the PCI-SIG organization. The purpose is to provide independent memory space, interrupts and DMA data streams for virtual machines by providing a standard specification. The SR-IOV standard defines a standard mechanism for PCIe device virtualization technology, which is a technical implementation of I / O virtualization. It is used to split a physical PCIe device into multiple virtual PCIe devices. Each virtual PCIe device has a PCIe configuration space and provides services to virtual machines like a physical PCIe device. The multiple virtual PCIe devices split can be transparently transmitted to one or more virtual machines to achieve the purpose of splitting physical PCIe devices and sharing PCIe devices in virtual machines. Device segmentation is implemented by hardware, with independent memory space, interrupts, and DMA data streams, so virtual machines can obtain I / O performance equivalent to that of local machines. The SR-IOV standard allows a single PCIe device to be virtualized into multiple PCIe devices. Each virtual PCIe device has its own PCIe configuration space and provides services to upper-layer software just like a physical PCIe device. Among them, PF (Physical Function) is the complete function of the SR-IOV device, with an independent PCI device ID, address space and interrupt. PF is responsible for managing the creation and deletion of VF (Visual Function) and the communication between VFs. VF is a part of PF and has the same functions as PF, but with limited resources. VF can be allocated to virtual machines or containers for use. The SR-IOV standard also has the following problems: (1) Based on the SR-IOV standard, VF devices are virtualized through hardware, and have independent configuration space and PCIe BAR space, which will occupy more memory space resources. With the linear growth of hardware resource requirements, the expansion of virtual VF devices is limited. (2) Based on the SR-IOV standard, the creation and deletion of hardware virtual VF devices will affect other VF devices on the same physical card. The configuration strategy needs to be determined during the system deployment phase, and the adjustment flexibility is insufficient. In actual use, user needs change at any time, and the flexible and changeable application scenarios of users cannot be met.

[0038] Figure 1 This is a flow chart of a method for implementing lightweight virtualization based on RDMA technology in an embodiment of the present invention. The method comprises the following steps:

[0039] Step S110: The RDMA device based on the RDMA technology establishes a connection with the host side.

[0040] In the specific implementation process, the RDMA device can use a DPU (Data Processing Unit) device that supports RDMA technology, and the host side can be a server side. The RDMA device is used to share the CPU processing pressure of the server side through virtualization technology. Usually, the DPU device that supports RDMA technology can be connected to the host side through a PCIe interface.

[0041] Step S120: Allocate an RDMA device interface to each container or virtual machine created on the host side; wherein the host side operating system creates multiple containers or virtual machines based on the virtualization function, and the RDMA device interface includes MMIO registers (Memory Mapped IO) in units of physical pages.

[0042] The RDMA device interface (dev interface) is located in the BAR space (Base Address Register) of the RDMA device. The base address register exists in the configuration space and is used to determine the size of the memory space required by different containers or virtual machines, and is mapped to the function memory space to provide a base address. The base address register can be mapped to the storage space or the IO space.

[0043] Step S130: The RDMA device interface receives a virtualization configuration from a host-side driver; wherein the virtualization configuration includes a queue priority configuration.

[0044] Step S140: Each RDMA device interface manages the queue in the RDMA device based on the received virtualization configuration.

[0045] In the specific implementation process, the queues managed by the RDMA device interface in the RDMA device include completion queues, and queue pairs are included in the virtual machine or container. The queue pair (Queue Pair, QP) is composed of a send queue (SQ) and a receive queue (RQ). Based on the queue pair and the completion queue, virtualized I / O based on RDMA technology can be realized, thereby reducing the CPU load pressure on the host side.

[0046] Step S150: The container or virtual machine uses the MMIO register to map to the RDMA device, so that the container or virtual machine can access the RDMA device based on the RDMA device interface, thereby realizing virtualization based on the RDMA technology.

[0047] By utilizing the method and system for realizing lightweight virtualization based on RDMA technology proposed in the present invention, RDMA devices based on RDMA technology can be used to assist in building a virtualized I / O processing environment and reduce the pressure on the host side. On the one hand, RDMA devices are uniformly configured through the host side operating system driver, and on the other hand, independent access is achieved through the RDMA device interface separately allocated to each container or virtual machine, thereby streamlining the complexity of the system, saving memory space and configuration space, reducing costs, and reducing the load pressure on the CPU on the host side.

[0048] In some embodiments of the present invention, the types of RDMA devices include RDMA network cards and DPU accelerator cards that support RDMA functions. RDMA devices refer to devices that support the RDMA protocol. The RDMA protocol is a high-performance network protocol that allows two systems to directly access each other's memory without the involvement of the CPU. RDMA devices include the following categories: (1) RDMA network cards, network interface cards that support the RDMA protocol, have independent DMA controllers, and can directly access memory. (2) RDMA storage devices, which have high-performance network interfaces and can communicate directly with RDMA network cards. (3) RDMA switches, which can connect RDMA devices to form an RDMA network. (4) DPU accelerator cards that support RDMA functions. The above types of RDMA devices are only examples. In fact, as long as the host-side RDMA device that supports the RDMA protocol can be loaded, this solution can be applied through certain pre-settings.

[0049] By adopting the embodiment of the invention, the application scope of the present solution is expanded. Compared with the existing SR-IOV standard, the pressure on the host-side CPU can be reduced through the RDMA device, and the computational complexity of the solution is also significantly lower than the virtualization solution based on the SR-IOV standard.

[0050] In some embodiments of the present invention, the method further includes: in each container or virtual machine created on the host side, using virtualization device operating software to manage the RDMA device interface and the host-side driver, so as to virtualize the RDMA device into a virtual RDMA device that provides services to each container or virtual machine respectively. In the specific implementation process, the virtualization device operating software can be referred to as VDEV, which is a software program installed in a virtual machine environment. The host-side driver can be configured based on VDEV, so that the host-side driver configures the RDMA device. At the same time, the RDMA device interface corresponding to each container or virtual machine can also be accessed based on VDEV, thereby accessing the RDMA device.

[0051] By adopting the embodiment of the invention, the RDMA device can be intuitively set up and the RDMA device interface can be accessed in a container or virtual machine environment, and then the I / O tasks in the container or virtual machine environment can be implemented through the RDMA device, such as forwarding data packets when the host side is a server.

[0052] Furthermore, in some embodiments of the present invention, the step of using virtualized device operating software to manage RDMA device interfaces and host-side drivers includes: on the host side, the virtualized device operating software sends configuration instructions to the host-side driver, so that the host-side driver generates virtualized configurations for each RDMA device interface and sends them to the RDMA device. Specifically, multiple virtualized device operating software VDEVs can communicate with each other to access and share the resources of the RDMA device, and the sharing method can be achieved by setting the queue to be shared or exclusive.

[0053] In some embodiments of the present invention, the step of using virtualization device operating software to manage the RDMA device interface and the host-side driver also includes: the virtualization device operating software manages the resources on the RDMA side by mapping the physical page where the RDMA device interface is located, thereby abstracting an RDMA device for each container or virtual machine.

[0054] In some embodiments of the present invention, the virtualization configuration also includes multiple ones of scheduling configuration, virtual device configuration, software interface configuration, flow control configuration, MAC address configuration and routing configuration. Among them, the virtual device configuration is set based on the RDMA standard, the flow control configuration, MAC address configuration and routing configuration are configured based on the network transmission situation on the host side (or server side), and the software interface configuration refers to the setting of the RDMA device interface. By configuring the RDMA device, the host-side driver can configure the RDMA device interface, flow management and queue priority in the BAR space, and the queue includes the completion queue (CQ) in the RDMA device.

[0055] By adopting the embodiment of the invention, the RDMA device can be fully configured, thereby realizing virtualized I / O based on the RDMA device, alleviating the pressure on the server side, and reducing the system complexity compared with the existing SR-IOV standard.

[0056] In some embodiments of the present invention, the container and the virtual machine include a virtual memory area for storing and receiving data and a queue pair for sending and receiving data, each RDMA device interface in the RDMA device manages a completion queue for sending and receiving data, and direct access to remote virtual memory is achieved based on the virtual memory area, queue pair and completion queue.

[0057] The RDMA standard proposes a queue-based method for implementing RDMA transmission, which will not be described in detail in this solution.

[0058] In some embodiments of the present invention, in the step of each RDMA device interface managing the queue in the RDMA device based on the received virtualization configuration, the method further includes: each container or virtual machine configures the queue as exclusive or shared through an independently configured RDMA device interface.

[0059] Compared with the existing SR-IOV standard, the use efficiency and flexibility of queue-based I / O processing can be improved by adopting the embodiment of the invention.

[0060] Figure 2 The schematic diagram of the system structure for realizing lightweight virtualization based on RDMA technology in one embodiment of the present invention is composed of a container (or virtual machine), a host and an RDMA device. The host-side driver realizes the configuration of the RDMA device, the management configuration of the RDMA device interface (RDMA dev interface), the flow configuration and the priority configuration and other operations. The container (or virtual machine) can directly access the RDMA device through the RDMA device interface to achieve the same I / O performance as the virtualization solution based on the SR-IOV standard. The RDMA device interface is similar to the virtual function (Visual Function, VF) in the SR-IOV standard. The RDMA device interface contains MMIO registers in physical pages. When the container (or virtual machine) is mapped, it is mapped in the form of physical pages so that memory isolation can be achieved and data security can be ensured. Based on the host-side driver (Host driver), RDMA device interface configuration, flow management configuration and queue priority management can be realized. The RDMA device interface is a section of physical memory, in which a large number of function registers are also defined, which will not be discussed in detail in this solution.

[0061] Figure 2 The virtual monitor shown in is a software program that can virtualize a physical machine into multiple virtual machines, each of which runs its own operating system and application programs. The functions of VMM include creating virtual machines, running virtual machines, and managing virtual machines. Among them, creating virtual machines includes allocating resources to virtual machines, such as CPU, memory, storage, and network, running virtual machines includes providing a running environment for virtual machines, and managing virtual machines includes starting, stopping, restarting, and migrating virtual machines.

[0062] Figure 3 FIG. 1 is a schematic diagram of a system structure for implementing virtualized I / O based on RDMA devices in an embodiment of the present invention. Figure 3The VDEV in the file is the operating software for managing resources in the container (or virtual machine). It is generated by mapping the physical page where the RDMA device interface is located. VDEV can be equivalent to an RDMA device and can be equivalent to an RDMA device by creating queue pairs / context / completion queues. The RDMA device interface is the manager that maintains the queuepairs / context / completion queue resources between the hardware and software. The RDMA device interface can achieve shared configuration by managing the queues in the RDMA device. Among them, VDEV (Visual DEVICE) is the abbreviation of virtual device. It is a logical concept in virtualization software and is used to represent the storage device of a virtual machine. VDEV can be composed of multiple physical storage devices, or a physical storage device can contain multiple VDEVs. Each VDEV is a logical storage device used for the storage of virtual machines.

[0063] also, Figure 3 The management domain PD (Physical Disk, also known as physical disk) is shown in the figure. The PD number is physically unique and can be repeated logically. PD is the basis of virtual machine storage. The operating system and data of the virtual machine are stored on the PD. Queue pair (QP) is a resource for I / O communication. Different numbers of queue pairs can be configured in each domain. The RDMA device interface can flexibly configure queue pair resources according to user needs. MR (Memory Region), translated into Chinese as RDMA memory area, refers to an area planned in the memory by the RDMA software layer for storing and receiving data. In the IB (InfiniBand) protocol, after applying for the memory area for storing data, the user needs to register the MR by calling the API provided by the IB framework to allow the RDMA network card to access this memory area. The host driver is opposite to the virtual machine driver, which can realize the management and configuration of the dev interface, traffic, and queue priority. At the same time, the virtual machine can access the host driver through VDEV and configure the RDMA device through the host driver to achieve system flexibility.

[0064] By introducing VDEV in a container or virtual machine, users can directly map device partition resources, independently operate and control device partition resources, and effectively improve IO performance and data security. VDEV is a virtual device in the ZFS file system, which consists of a group of physical disks. VDEV can be used to create RAID arrays, mirrors, or simple logical volumes.

[0065] Figure 4 is a diagram showing a configuration relationship of an RDMA network card virtualization in an embodiment of the present invention. Figure 4The RDMA device can be Figure 3 The RDMA device can provide a set of hardware resources for high-performance network communication between computers. This hardware resource can be divided through configuration and provided to multiple containers (or virtual machines). The network card hardware design needs to provide resource configuration, MAC address configuration, routing configuration, and the network card virtualizes multiple different network card resources. Each network card resource is configured with a user-private MAC address and routing configuration. Traffic configuration, queue priority configuration, configure traffic and priority queues for different users. Virtual device configuration, software interface configuration, configure interfaces and resources for virtual machines and containers, and virtual machines have exclusive access to resources. The scheduling configuration can be used to set the task scheduling process in the I / O process to relieve the pressure of I / O on the host side.

[0066] The method and system for realizing lightweight virtualization based on RDMA technology proposed in the present invention can use RDMA devices based on RDMA technology to assist in building a virtualized I / O processing environment and reduce the pressure on the host side. On the one hand, the RDMA devices are uniformly configured through the host side operating system driver, and on the other hand, independent access is realized through the RDMA device interface separately allocated to each container or virtual machine, thereby streamlining the complexity of the system, saving memory space and configuration space occupation, reducing costs, and reducing the load pressure on the CPU on the host side.

[0067] Compared with the SR-IOV standard, which requires independent configuration space and memory-mapped I / O space (MMIO), and each virtual device has an independent interrupt vector configuration problem, the present invention can reduce the requirements for memory space and configuration space, and uniformly configure RDMA devices through the host-side operating system driver, which can reduce the complexity of the system, save memory space and configuration space, reduce costs, and reduce the load pressure on the host-side CPU.

[0068] Specifically, a DPU device that supports RDMA can be connected to the host side (server) through a PCIe interface. The combination of DPU and RDMA can play a synergistic effect and improve the overall performance of the data center. For example, DPU can be used to accelerate RDMA transmission and reduce the load of the host side CPU, and RDMA can reduce the load of DPU.

[0069] The virtualization implementation method proposed in the present invention can flexibly configure the host-side driver through VDEV, thereby configuring the entire RDMA device through the host-side driver. In the process of configuring the RDMA device, the RDMA device interfaces for different containers or virtual machines can be adjusted respectively, and users using the same RDMA device do not affect each other. Therefore, in actual use, the configuration of the RDMA device interface can be flexibly adjusted as user needs change to meet the user's changing application scenarios (memory isolation can be achieved by mapping in the form of physical pages).

[0070] Corresponding to the above method, the present invention also provides a system for implementing lightweight virtualization based on RDMA technology, the system comprising a computer device, the computer device comprising a processor and a memory, the memory storing computer instructions, the processor being used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0071] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described above are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0072] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0073] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0074] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0075] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for implementing lightweight virtualization based on RDMA technology, characterized in that: The method comprises the following steps: An RDMA device based on RDMA technology establishes a connection with a host side; wherein the RDMA device includes a DPU device supporting RDMA technology, and the host side includes a server side; For each container or virtual machine created on the host side, an RDMA device interface is allocated; wherein the host side operating system creates multiple containers or virtual machines based on the virtualization function, and the RDMA device interface includes MMIO registers in units of physical pages; The RDMA device interface receives a virtualization configuration from a host-side driver; wherein the virtualization configuration includes a queue priority configuration; Each RDMA device interface manages the queues in the RDMA device based on the received virtualization configuration; wherein the queues managed by the RDMA device interface in the RDMA device include a completion queue, a queue pair in the virtual machine or container, the queue pair consists of a sending queue and a receiving queue, and virtualized I / O based on the RDMA technology is performed based on the queue pair and the completion queue; The container or virtual machine uses the MMIO register to map to the RDMA device, so that the container or virtual machine can access the RDMA device based on the RDMA device interface, realize virtualization based on the RDMA technology, and enable the RDMA device to share the CPU processing pressure on the server side through the virtualization technology.

2. The method according to claim 1, characterized in that The types of the RDMA devices include RDMA network cards and DPU acceleration cards that support RDMA functions.

3. The method according to claim 1, characterized in that: The method further includes: In each container or virtual machine created on the host side, the virtualized device operating software is used to manage the RDMA device interface and the host side driver, so as to virtualize the RDMA device into a virtual RDMA device that provides services to each container or virtual machine respectively.

4. The method according to claim 3, characterized in that The steps of using virtualized device operating software to manage the RDMA device interface and the host-side driver include: On the host side, the virtualized device operating software sends a configuration instruction to the host side driver, so that the host side driver generates a virtualized configuration for each RDMA device interface and sends it to the RDMA device.

5. The method according to claim 3, characterized in that: The step of using the virtualized device operating software to manage the RDMA device interface and the host-side driver also includes: The virtualized device operating software manages the resources on the RDMA side by mapping the physical page where the RDMA device interface is located, thereby abstracting an RDMA device for each container or virtual machine.

6. The method according to any one of claims 1 to 5, characterized in that The virtualization configuration also includes multiple ones of scheduling configuration, virtual device configuration, software interface configuration, flow control configuration, MAC address configuration and routing configuration.

7. The method according to claim 1, characterized in that The container and the virtual machine include a virtual memory area for storing and receiving data and a queue pair for sending and receiving data. Each RDMA device interface in the RDMA device manages a completion queue for sending and receiving data. Direct access to remote virtual memory is achieved based on the virtual memory area, the queue pair and the completion queue.

8. The method according to claim 1, characterized in that In the step where each RDMA device interface manages the queue in the RDMA device based on the received virtualization configuration, the method further includes: Each container or virtual machine configures the queue as exclusive or shared through an independently configured RDMA device interface.

9. A system for implementing lightweight virtualization based on RDMA technology, comprising a processor and a memory, characterized in that: The memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method as claimed in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images