Data processing method and apparatus, electronic device, and computer program product

Abstract

A data processing method and apparatus, an electronic device, and a computer program product, relating to the technical field of networks. The method comprises: in a virtual environment of a target object, receiving, by means of a first API, a data processing request initiated by the target object; forwarding the data processing request to a second API by means of the first API; and accessing a virtual RDMA device by means of the second API, so as to execute the data processing request by means of the virtual RDMA device to obtain a data processing result. The present application solves the technical problem in the related art that the costs of using a virtual RDMA function by users is relatively high due to the usage mode of the RDMA function in a virtual machine being inconsistent with that of a physical host and needing to be adapted and debugged by the users.

Description

Data processing methods and apparatus, electronic equipment and computer program products

[0001] This application claims priority to Chinese Patent Application No. 202410950184.1, filed on July 15, 2024, entitled “Data Processing Method and Apparatus, Electronic Device and Computer Program Product”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of data processing technology, and more specifically, to a data processing method and apparatus, electronic equipment, and computer program product. Background Technology

[0003] With the increasing breadth and depth of artificial intelligence (AI) technology applications, cloud service platforms have become a crucial channel for users across various fields to access and utilize AI capabilities. On one hand, Remote Direct Memory Access (RDMA) technology, with its advantages of high throughput, low latency, and low CPU overhead, has become a fundamental technology for cloud platforms to provide AI services. On the other hand, virtualization, with its characteristics of resource isolation, lightweight design, consistency, rapid deployment, and scalability, has also become one of the main ways platforms provide services. Therefore, how to provide RDMA capabilities to tenants in a virtualized environment is a critical technical issue that urgently needs to be addressed.

[0004] Current virtualization of RDMA network devices is primarily based on hardware virtualization, which generates virtual instances of the physical network interface card (NIC) on the host and assigns them to virtual machines or containers to provide RDMA functionality to the virtual environment. However, for RDMA network adapters that lack this capability, virtualization still faces many fundamental technical challenges. One critical issue that urgently needs to be addressed is how to provide users with RDMA services in the virtual space in the same way as in the non-virtualized environment. If the usage of RDMA functionality within a virtual machine or container differs from that on the host, users will need to re-adapt and debug, significantly increasing the cost of use.

[0005] There is currently no effective solution to the problem that the use of virtual RDMA functionality in the aforementioned remote direct memory access virtualization technologies is costly because the usage of RDMA functionality in virtual machines is inconsistent with that of physical hosts, requiring users to adapt and debug. Summary of the Invention

[0006] This application provides a data processing method and apparatus, electronic device and computer program product to at least solve the technical problem in the related art that the use of RDMA function in virtual machines is inconsistent with that of physical hosts, requiring users to adapt and debug, resulting in high costs for users to use virtual RDMA function.

[0007] According to one aspect of the embodiments of this application, a data processing method is provided, comprising: receiving a data processing request initiated by the target object in a virtual environment of the target object through a first API interface, wherein the first API interface is a user-facing interface; forwarding the data processing request to a second API interface through the first API interface, wherein the second API interface is an interface for a virtual RDMA device, the virtual RDMA device having the same device configuration as the RDMA device of a physical host; and accessing the virtual RDMA device through the second API interface to execute the data processing request through the virtual RDMA device and obtain a data processing result.

[0008] Furthermore, before receiving the data processing request initiated by the target object through the first API interface, the method further includes: configuring the first API interface in the virtual environment, wherein the first API interface is the same as the first API interface in the physical host; configuring the second API interface in the virtual environment, wherein the second API interface is the same as the second API interface in the physical host; and deploying the virtual RDMA device in the virtual environment.

[0009] Further, deploying the virtual RDMA device in the virtual environment includes: registering a target bus based on a Linux kernel interface in the virtual environment, wherein the virtual RDMA device consists of at least a Linux device and an InfiniBand device, and the target bus is used to connect at least the Linux device in the virtual RDMA device; registering the Linux device based on a Linux device registration interface and the target bus in the virtual environment; and registering the InfiniBand device based on the InfiniBand kernel interface in the virtual environment to achieve the deployment of the virtual RDMA device.

[0010] Furthermore, in the virtual environment, registering the Linux device based on the Linux device registration interface and the target bus includes: obtaining device parameter information of the RDMA device of the physical host; constructing a first device driver for the Linux device based on the Linux kernel interface; generating the Linux device based on the Linux device registration interface, the device parameter information, and the first device driver, and connecting the Linux device to the target bus; generating the directory, file information, and configuration information of the Linux device in the virtual file system of the virtual environment to complete the registration of the Linux device.

[0011] Further, generating the directory, file information, and configuration information of the Linux device in the virtual file system of the virtual environment to register the Linux device includes: obtaining the directory, file information, and configuration information of the Linux device corresponding to the RDMA device of the physical host; and generating the directory, file information, and configuration information of the Linux device in the virtual file system of the virtual environment based on the directory, file information, and configuration information of the Linux device corresponding to the RDMA device of the physical host to complete the registration of the Linux device.

[0012] Furthermore, in the virtual environment, registering the InfiniBand device based on the InfiniBand kernel interface includes: registering the direct memory access device according to the parameters of the direct memory access device corresponding to the RDMA device of the physical host; configuring the directory, files, and configuration information of the InfiniBand device corresponding to the RDMA device of the physical host in the virtual environment, and associating the operation functions of the InfiniBand device with the virtual environment; and performing the registration process for the InfiniBand device based on the InfiniBand kernel interface, the directory, files, and configuration information of the InfiniBand device in the virtual environment, the operation functions of the InfiniBand device, and the direct memory access device.

[0013] Furthermore, before registering the InfiniBand device, the method further includes: determining whether the target device is associated with the RDMA device of the physical host; if the target device is associated with the RDMA device of the physical host, then in the virtual environment, the target device is associated with the virtual RDMA device.

[0014] Further, registering the direct memory access device based on the parameters of the direct memory access device corresponding to the RDMA device of the physical host includes: constructing a second device driver for registering the InfiniBand device based on the InfiniBand kernel interface; and registering the direct memory access device based on the second device driver and the parameters of the direct memory access device corresponding to the RDMA device of the physical host.

[0015] Furthermore, after deploying the virtual RDMA device in the virtual environment, the method further includes: establishing an association between the virtual RDMA device and the RDMA device of the physical host; and performing remote direct memory access of the virtual RDMA device based on the association.

[0016] According to another aspect of the embodiments of this application, a data processing apparatus is also provided, comprising: a receiving unit configured to receive a data processing request initiated by the target object through a first API interface in a virtual environment of the target object, wherein the first API interface is a user-facing interface; a forwarding unit configured to forward the data processing request to a second API interface through the first API interface, wherein the second API interface is an interface for a virtual RDMA device, the virtual RDMA device having the same device configuration as the RDMA device of a physical host; and an execution unit configured to access the virtual RDMA device through the second API interface to execute the data processing request through the virtual RDMA device and obtain a data processing result.

[0017] Furthermore, the apparatus further includes: a first configuration unit configured to configure the first API interface in the virtual environment before receiving a data processing request initiated by the target object through the first API interface, wherein the first API interface is the same as the first API interface in the physical host; a second configuration unit configured to configure the second API interface in the virtual environment, wherein the second API interface is the same as the second API interface in the physical host; and a deployment unit configured to deploy the virtual RDMA device in the virtual environment.

[0018] Further, the deployment unit includes: a first registration subunit, configured to register a target bus in the virtual environment based on a Linux kernel interface, wherein the virtual RDMA device consists of at least a Linux device and an InfiniBand device, and the target bus is used to connect at least the Linux device in the virtual RDMA device; a second registration subunit, configured to register the Linux device in the virtual environment based on a Linux device registration interface and the target bus; and a third registration subunit, configured to register the InfiniBand device in the virtual environment based on an InfiniBand kernel interface, thereby realizing the deployment of the virtual RDMA device.

[0019] Further, the second registration subunit includes: an acquisition module, configured to acquire device parameter information of the RDMA device of the physical host; a construction module, configured to construct a first device driver for the Linux device based on the Linux kernel interface; a first generation module, configured to generate the Linux device based on the Linux device registration interface, the device parameter information, and the first device driver, and connect the Linux device to the target bus; and a second generation module, configured to generate the directory, file information, and configuration information of the Linux device in the virtual file system of the virtual environment to complete the registration of the Linux device.

[0020] Furthermore, the second generation module includes: an acquisition submodule, configured to acquire the directory, file information, and configuration information of the Linux device corresponding to the RDMA device of the physical host; and a generation submodule, configured to generate the directory, file information, and configuration information of the Linux device in the virtual file system of the virtual environment based on the directory, file information, and configuration information of the Linux device corresponding to the RDMA device of the physical host, so as to complete the registration of the Linux device.

[0021] Further, the third registration subunit includes: a first registration module, configured to register the direct memory access device based on the parameters of the direct memory access device corresponding to the RDMA device of the physical host; a configuration module, configured to configure the directory, files, and configuration information of the InfiniBand device corresponding to the RDMA device of the physical host in the virtual environment, and associate the operation functions of the InfiniBand device with the virtual environment; and a second registration module, configured to perform registration processing on the InfiniBand device based on the InfiniBand kernel interface, the directory, files, and configuration information of the InfiniBand device in the virtual environment, the operation functions of the InfiniBand device, and the direct memory access device.

[0022] Furthermore, the apparatus further includes: a judging unit, configured to judge whether the target device is associated in the RDMA device of the physical host before performing registration processing on the InfiniBand device; and an associating unit, configured to associate the target device with the virtual RDMA device in the virtual environment if the target device is associated in the RDMA device of the physical host.

[0023] Furthermore, the first registration module includes: a construction submodule, configured to construct a second device driver for registering the InfiniBand device based on the InfiniBand kernel interface; and a registration submodule, configured to register the direct memory access device according to the parameters of the second device driver and the direct memory access device corresponding to the RDMA device of the physical host.

[0024] Furthermore, the apparatus further includes: an establishment unit configured to establish an association between the virtual RDMA device and the RDMA device of the physical host after the virtual RDMA device is deployed in the virtual environment; and a processing unit configured to perform remote direct memory access of the virtual RDMA device based on the association.

[0025] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory storing an executable program; and a processor for running the program, wherein the program executes the data processing method described in any one of the above embodiments when it runs.

[0026] According to another aspect of the embodiments of this application, a computer program product is also provided, the computer program product including a stored computer program that, when executed by a processor, implements the data processing method described in any one of the preceding claims.

[0027] In this embodiment, the following steps are adopted: In the virtual environment of the target object, a data processing request initiated by the target object is received through a first API interface, wherein the first API interface is a user-facing interface; the data processing request is forwarded to a second API interface through the first API interface, wherein the second API interface is an interface for a virtual RDMA device, and the virtual RDMA device has the same device configuration as the RDMA device of the physical host; the virtual RDMA device is accessed through the second API interface to execute the data processing request through the virtual RDMA device and obtain the data processing result. This solves the technical problem in related technologies where the usage mode of RDMA function in the virtual machine is inconsistent with that of the physical host, requiring users to adapt and debug, resulting in high costs for users to use virtual RDMA function.

[0028] In this solution, a virtual RDMA device with the same usage environment as the physical host's RDMA device is deployed in the target object's virtual environment. When the target object needs direct remote memory access, the data processing request is forwarded to the second API interface through the first API interface, and then the virtual RDMA device is accessed directly through the second API interface to realize the RDMA function. The target object can use the provided RDMA service in the same way as on the host in the virtual machine or container without additional adaptation work, realizing the consistency of RDMA devices inside and outside the virtual machine, thereby reducing the threshold for using RDMA in the virtualization environment and improving the efficiency of users using RDMA functions. Attached Figure Description

[0029] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0030] Figure 1 is a hardware structure block diagram of a computer terminal provided according to Embodiment 1 of this application;

[0031] Figure 2 is a flowchart of the data processing method provided according to Embodiment 1 of this application;

[0032] Figure 3 is a schematic diagram of the implementation of RDMA function according to the prior art provided in Embodiment 1 of this application;

[0033] Figure 4 is a schematic diagram of the data processing method provided according to Embodiment 1 of this application;

[0034] Figure 5 is a flowchart of generating a virtual RDMA device according to Embodiment 1 of this application;

[0035] Figure 6 is a schematic diagram of a data processing apparatus provided according to Embodiment 2 of this application;

[0036] Figure 7 is a structural block diagram of an electronic device provided according to Embodiment 3 of this application. Detailed Implementation

[0037] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0038] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0039] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0040] RDMA: Remote Direct Memory Access is a technology that provides high-performance network data access by implementing zero-copy and kernel bypass technology in hardware.

[0041] Virtualization: Computer virtualization technology is a resource management technology that refers to abstracting various physical resources of a computer (CPU, GPU, memory, network card, etc.) into divisible and recombinable resource instances, thereby breaking down physical structure barriers and enabling users to use computer resources in a better way than the original configuration.

[0042] Modalias information: A special string in the Linux operating system used to describe and identify a device. When the system detects a new hardware device, the Device Manager can use this string to find and load the appropriate driver module.

[0043] KVM: Kernel-based Virtual Machine is an open-source virtualization technology built into Linux. It turns Linux into a virtual machine discoverer, enabling a host computer to run multiple isolated virtual environments, i.e., virtual guests or virtual machines.

[0044] Linux device: Linux struct device refers to the device model defined by the kernel device in the Linux operating system, that is, the hardware or software interface that can perform input and output operations.

[0045] Net device: Net struct device refers to the network device model defined by the kernel net_device in the Linux operating system, which can include all types of network devices such as Ethernet cards, virtual network devices, etc.

[0046] PCI device: PCI struct device refers to the device model on the PCIe or PCI bus defined by the kernel pci_dev in the Linux operating system.

[0047] IB device: IB struct device refers to the InfiniBand device model defined by the kernel ib_device in the Linux operating system.

[0048] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0049] Example 1

[0050] According to an embodiment of this application, a data processing method is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0051] The method embodiment provided in Embodiment 1 of this application can be executed in a mobile terminal, computer terminal, or similar computing device. Figure 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a data processing method. As shown in Figure 1, the computer terminal (or mobile device) 10 may include a processor set 102 (the processor set 102 may include, but is not limited to, a processing device such as a microprocessor (MCU) or a programmable logic device (FPGA), and the processor set 102 may include a processor set, shown in Figure 1 as 102a, 102b, ..., 102n), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that the structure shown in Figure 1 is only illustrative and does not limit the structure of the above-described electronic device. For example, computer terminal 10 may also include more or fewer components than shown in FIG1, or have a different configuration than shown in FIG1.

[0052] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0053] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the data processing method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the aforementioned data processing method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0054] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0055] The display may be a touchscreen LCD display that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0056] Under the aforementioned operating environment, this application provides a data processing method as shown in Figure 2. Figure 2 is a flowchart of the data processing method according to Embodiment 1 of this application. The data processing method includes:

[0057] Step S201: In the virtual environment of the target object, a data processing request initiated by the target object is received through the first API interface, wherein the first API interface is a user-facing interface.

[0058] Optionally, in the user's (i.e., the target object mentioned above) virtual environment, the user can initiate a request for direct access to remote memory through the user-end application, and obtain the user's request for direct access to remote memory through the first API interface, i.e., the aforementioned receiving of data processing requests initiated by the target object.

[0059] It should be noted that the virtual environment mentioned above can be KVM, an open-source virtualization technology built into Linux, which can turn Linux into a virtual machine detection program, enabling the host computer to run multiple isolated virtual environments, namely virtual guests or virtual machines.

[0060] It should be noted that the first API interface is the API interface in libibverbs. libibverbs is a general-purpose user-space library that provides applications with API interfaces to perform RDMA device-related operations.

[0061] Step S202: Forward the data processing request to the second API interface through the first API interface. The second API interface is an interface for the virtual RDMA device. The virtual RDMA device has the same device structure as the RDMA device of the physical host.

[0062] Optionally, after receiving the response data processing request, the first API interface will forward the data processing request to the second API interface. It should be noted that the second API interface is the API interface for verbs_provider to implement the various APIs in libibverbs based on a specific RDMA device; that is, the aforementioned second API interface is an interface for virtual RDMA devices.

[0063] It should be noted that the virtual RDMA device has the same device structure as the physical host's RDMA device. The virtual RDMA device provides users with the same RDMA usage environment as on the physical host. Users can directly use the same methods as on the physical host (i.e., accessing the virtual RDMA device through the first and second API interfaces as described above) in the virtual environment to use the generated virtual RDMA device, thereby enabling the invocation of the corresponding RDMA device on the host without any additional adaptation work.

[0064] Step S203: Access the virtual RDMA device through the second API interface to execute a data processing request through the virtual RDMA device and obtain the data processing result.

[0065] Optionally, the second API interface accesses the virtual RDMA device, and through the virtual RDMA device, calls are made to the corresponding RDMA device on the host, thereby achieving direct remote memory access and obtaining the above data processing results.

[0066] To ensure that users can use the generated virtual RDMA device in the virtual environment using the same methods as in the physical host (i.e., accessing the virtual RDMA device through the first API interface and the second API interface as described above), the data processing method provided in Embodiment 1 of this application further includes, before receiving the data processing request initiated by the target object through the first API interface: configuring the first API interface in the virtual environment, wherein the first API interface is the same as the first API interface in the physical host; configuring the second API interface in the virtual environment, wherein the second API interface is the same as the second API interface in the physical host; and deploying the virtual RDMA device in the virtual environment.

[0067] Optionally, a virtual RDMA device is deployed in the user's virtual environment. Then, the libibverbs library and verbs_provider library used in the physical host are specified, and the libibverbs library and verbs_provider library are configured in the virtual environment, that is, the first API interface and the second API interface are configured in the virtual environment as described above.

[0068] By configuring the libibverbs library, verbs_provider library, and virtual RDMA device in a virtual environment, a virtual RDMA device can be generated within a virtual machine or container, providing users with the same RDMA usage environment as on the host, thereby enabling the invocation of the corresponding RDMA device on the host without additional adaptation work.

[0069] How to deploy a virtual RDMA device is crucial. In the data processing method provided in Embodiment 1 of this application, deploying a virtual RDMA device in a virtual environment includes: registering a target bus based on a Linux kernel interface in the virtual environment, wherein the virtual RDMA device consists of at least a Linux device and an InfiniBand device, and the target bus is used to connect at least the Linux device in the virtual RDMA device; registering the Linux device based on a Linux device registration interface and the target bus in the virtual environment; and registering the InfiniBand device based on the InfiniBand kernel interface in the virtual environment to realize the deployment of the virtual RDMA device.

[0070] Optionally, in the Linux kernel, registering a `struct bus_type` instance using the `bus_register` function creates and initializes a new bus type. This registers the target bus within the user's virtual environment through the Linux kernel interface. This process is a crucial step in creating the device driver model, enabling the kernel to manage devices and drivers connected to this new bus type. After registering a new bus type, the kernel can manage device discovery, registration, deregistration, and driver matching through that bus type.

[0071] Then, in the virtual environment, the Linux device is registered based on the Linux device registration interface and the target bus. Specifically, in the Linux kernel, an instance of type `struct device` (the Linux device mentioned above) is added to the kernel device model using the `device_register` function. Finally, in the virtual environment, the IB device (the InfiniBand device mentioned above) is registered based on the InfiniBand kernel interface. RDMA functionality is then implemented through the IB device and the Linux device, thus completing the deployment of the virtual RDMA device.

[0072] Adding an instance of type struct device to the kernel device model via the device_register function and registering IB devices based on the InfiniBand kernel interface can accurately deploy virtual RDMA devices in virtual machines that are identical to those on physical hosts, improving the user experience of using RDMA.

[0073] To quickly register Linux devices, the data processing method provided in Embodiment 1 of this application, in a virtual environment, registering a Linux device based on a Linux device registration interface and a target bus includes: obtaining device parameter information of the RDMA device of the physical host; constructing a first device driver for the Linux device based on the Linux kernel interface; generating a Linux device based on the Linux device registration interface, device parameter information, and the first device driver, and connecting the Linux device to the target bus; and generating the directory, file information, and configuration information of the Linux device in a virtual file system in the virtual environment to complete the registration of the Linux device.

[0074] Optionally, depending on the requirements of the virtual device to be created, relevant parameters of the RDMA device are obtained from the physical host, such as vendor information, device number, NUMA information, modalias string, and associated other devices. Then, a Linux device driver (i.e., the first device driver mentioned above) is developed through the kernel interface provided by Linux. The device driver is responsible for managing and controlling a specific type of device. When a device driver registers using the `driver_register` function, it adds itself to the kernel's driver model, which allows the driver to be detected by the bus code and bound to compatible devices. Device drivers are typically represented in the form of `struct device_driver`, which contains information about the driver, such as its name, bus type, and the interface for interacting with the device.

[0075] Then, using the Linux device registration interface, device parameter information, and the first device driver, a Linux device is generated. First, `device_register` registers the first device driver to the virtual environment using the Linux device registration interface. Then, the first device driver uses the `device_create` function to create a device instance (the aforementioned Linux device) and connect it to the target bus. Finally, the directory, file information, and configuration information of the Linux device, such as vendor information, device number, NUMA information, and modalias string, are generated in the sysfs file system (the aforementioned virtual file system) within the virtual environment, completing the registration process of the Linux device within the virtual machine.

[0076] Registering drivers and creating device instances via the Linux device registration interface allows for the rapid registration of Linux devices within a virtual machine.

[0077] To ensure that the virtual RDMA device within the virtual machine is identical to that within the physical host, the data processing method provided in Embodiment 1 of this application generates the directory, file information, and configuration information of the Linux device in the virtual file system of the virtual environment to register the Linux device. This includes: obtaining the directory, file information, and configuration information of the Linux device corresponding to the RDMA device on the physical host; and generating the directory, file information, and configuration information of the Linux device in the virtual file system of the virtual environment based on the directory, file information, and configuration information of the Linux device corresponding to the RDMA device on the physical host, thereby completing the registration of the Linux device.

[0078] Optionally, obtain the directory and file information and configuration information of the Linux device of the RDMA device on the physical host in the sysfs file system, and then refer to the directory and file information and configuration information of the Linux device corresponding to the RDMA device on the host in the virtual file system. Use sysfs-related functions to generate the corresponding directory and file information and configuration information, such as vendor information, device number, NUMA information, modalias string, etc., in the sysfs file system of the registered virtual machine Linux device to complete the registration process of the Linux device in the virtual machine.

[0079] The above steps ensure that the information in the virtual RDMA device within the virtual machine is completely consistent with the information in the physical host of the Linux device.

[0080] How to register the IB device structure is crucial. In the data processing method provided in Embodiment 1 of this application, registering the InfiniBand device based on the InfiniBand kernel interface in a virtual environment includes: registering the direct memory access device according to the parameters of the direct memory access device corresponding to the RDMA device of the physical host; configuring the directory, files, and configuration information of the InfiniBand device corresponding to the RDMA device of the physical host in the virtual environment, and associating the operation functions of the InfiniBand device with the virtual environment; and performing the registration process for the InfiniBand device based on the InfiniBand kernel interface, the directory, files, and configuration information of the InfiniBand device in the virtual environment, the operation functions of the InfiniBand device, and the direct memory access device.

[0081] Optionally, the DMA device (i.e., the direct memory access device mentioned above) is registered in the virtual environment through the kernel interface provided by ib_core (i.e., the InfiniBand kernel interface mentioned above) and the DMA (i.e., the direct memory access device mentioned above) parameters of the host RDMA device (i.e., the RDMA device of the physical host mentioned above). The DMA device is used for RDMA data transfer. After registration is complete, the directory, files, and configuration information of the physical host's RDMA device are obtained, such as the device name, bitmask settings, and the userspace verbs commands, IB node type, node descriptor, number of completion vectors, and unique identifiers supported by this device.

[0082] Then, the virtual environment is configured using the relevant parameters or custom parameters of the host RDMA device, and the operation functions of the InfiniBand device are associated with the virtual environment. It should be noted that configuring in the virtual environment refers to writing the directory, files, and configuration information of the physical host's RDMA device—such as the device name, bitmask settings, userspace verbs supported by the device, IB node type, node descriptor, number of completion vectors, unique identifiers, etc.—into an object, and associating the operation functions of the InfiniBand device with this object. This object is not visible in the virtual machine.

[0083] Finally, the InfiniBand kernel interface processes the directory, files, configuration information, and operation functions of the InfiniBand device in the virtual environment, and associates the direct memory access device with the InfiniBand device in the virtual machine to complete the registration of the InfiniBand device. It should be noted that the InfiniBand device registration process includes associating the direct memory access device with the aforementioned objects, and then completing the registration of the InfiniBand device through the InfiniBand kernel interface.

[0084] By using the kernel interface provided by ib_core, the directory, files, configuration information, and operation functions of InfiniBand devices in the virtual environment can be processed, and direct memory access devices can be associated with InfiniBand devices in the virtual machine. This allows the registration of the same IB device as the host in the virtual environment.

[0085] In order to determine the consistency between the IB device in the virtual environment and the physical host, the data processing method provided in Embodiment 1 of this application further includes, before registering the InfiniBand device, determining whether the target device is associated with the RDMA device of the physical host; if the target device is associated with the RDMA device of the physical host, then in the virtual environment, the target device is associated with the virtual RDMA device.

[0086] Optionally, before registering the InfiniBand device, it is necessary to further determine whether the target device is associated with the RDMA device on the physical host. The target device can be a PCI device, a net device, other custom devices, or the parent device of an IB device. If the target device exists, it is associated with the virtual RDMA device in the virtual environment. After the association is complete, the IB device registration can be completed through the kernel interface provided by the ib_core module. During the process, ib_core can automatically obtain some basic information about the device through its implemented IB device operation functions, including the device's immutable information, port information, globally unique identifier, security partitioning mechanism, etc., to complete the registration process of the IB device within the virtual machine.

[0087] The above-described association process for the target devices ensures consistency between the IB devices in the virtual environment and those in the physical host.

[0088] Furthermore, in the data processing method provided in Embodiment 1 of this application, the direct memory access device is registered according to the parameters of the direct memory access device corresponding to the RDMA device of the physical host: a second device driver is constructed based on the InfiniBand kernel interface for registering the InfiniBand device; the direct memory access device is registered according to the second device driver and the parameters of the direct memory access device corresponding to the RDMA device of the physical host.

[0089] Optionally, a custom IB device driver (i.e., the second device driver mentioned above) can be developed through the kernel interface provided by ib_core. Then, the IB device driver is associated with the target bus to register the direct memory access device using the parameters of the IB device driver and the direct memory access device corresponding to the RDMA device on the physical host.

[0090] To ensure that the virtual RDMA device can be used to access the host RDMA device, in the data processing method provided in Embodiment 1 of this application, after deploying the virtual RDMA device in the virtual environment, the method further includes: establishing an association between the virtual RDMA device and the RDMA device of the physical host; and performing remote direct memory access of the virtual RDMA device based on the association.

[0091] Optionally, after deploying the virtual RDMA device in the virtual environment, an association is established between the virtual RDMA device and the RDMA device on the physical host, and finally, remote direct memory access of the virtual RDMA device is performed based on the association.

[0092] Optionally, in a non-virtualized scenario, the way users use RDMA devices is shown in Figure 3. libibverbs is a user-space general-purpose library that provides applications with API interfaces for performing RDMA device-related operations. verbs_provider implements various APIs in libibverbs based on specific RDMA devices to control and use the corresponding RDMA devices. In the data processing method provided in Embodiment 1 of this application, a virtual RDMA device is generated within a virtual machine or container, providing users with the same RDMA usage environment as on the host. The expected effect is shown in Figure 4. After installing the same libibverbs and verbs_provider libraries as in a non-virtualized environment within a virtual machine or container, users can use the generated virtual RDMA device in the virtual environment in the same way, thereby enabling calls to the corresponding RDMA device on the host without additional adaptation work.

[0093] In an optional embodiment, a virtual RDMA device can be generated using the flowchart shown in Figure 5. The virtual RDMA device generation method mainly includes three steps: registering the bus, registering the Linux device, and registering the IB device. As shown in Figure 5, the specific implementation of each step is as follows:

[0094] (1) Register bus

[0095] The Linux kernel interface is used to register a bus for connecting and managing virtual RDMA devices and their drivers.

[0096] (2) Registering Linux devices

[0097] This process involves developing a custom Linux device driver using the Linux kernel interface to register the Linux device within a virtual RDMA device. First, based on the requirements for creating the virtual device, the relevant parameters of the RDMA physical network adapter (i.e., the device parameter information of the RDMA device on the physical host) are obtained from the host machine, such as vendor information, device ID, NUMA information, modalias string, and associated devices. Then, the corresponding Linux device is generated through the device registration interface provided by the Linux kernel and connected to the bus created in the first step. Next, referring to the directories and files represented in the sysfs filesystem of the host RDMA device, corresponding directories and files are generated in the sysfs filesystem of the registered virtual machine Linux device, including vendor information, device ID, NUMA information, and modalias string, completing the registration process for the Linux device within the virtual machine.

[0098] (3) Register IB devices

[0099] Develop a custom IB device driver using the kernel interface provided by ib_core to register IB devices within a virtual RDMA device. First, register the DMA device for RDMA data transfer, using relevant parameters or custom parameters from the host RDMA physical network adapter to generate corresponding directories and files in the sysfs filesystem of the registered DMA device, such as vendor information and device number. Then, configure the IB device to be registered using relevant parameters or custom parameters from the host RDMA device, including adding the device name, setting a bitmask to determine the supported userspace verbs commands, IB node type, node descriptor, number of completion vectors, unique identifier, etc., and associating it with the implemented IB device operation functions. As needed, associate the registered DMA device with the corresponding PCI devices, net devices, and other custom devices using methods provided by ib_core, set the parent device of the IB device, and complete IP address configuration. Based on this, IB device registration is completed through the kernel interface provided by the ib_core module. During the process, ib_core can automatically obtain some basic information about the device through the implemented IB device operation functions, including the device's immutable information, port information, globally unique identifier, security partitioning mechanism, etc., to complete the registration process of IB devices within the virtual machine.

[0100] Using the above method, a virtual RDMA device with the same device structure and sysfs file as the host RDMA device can be generated within a virtual machine or container. Then, by combining relevant technologies, operations on the virtual RDMA device can be mapped to the host RDMA device. Users can then identify and use the virtual RDMA device in the same way as on the host machine, without additional adaptation and debugging work. By achieving consistency between RDMA devices inside and outside the virtual machine, the barrier to using RDMA services in a virtualized environment can be lowered, and debugging efficiency can be improved.

[0101] In the data processing method provided in Embodiment 1 of this application, a data processing request initiated by the target object is received through a first API interface in the virtual environment of the target object. The first API interface is a user-facing interface. The data processing request is forwarded to a second API interface through the first API interface. The second API interface is an interface for a virtual RDMA device, and the virtual RDMA device has the same device configuration as the RDMA device of the physical host. The virtual RDMA device is accessed through the second API interface to execute the data processing request and obtain the data processing result. This solves the technical problem in related technologies where the usage of RDMA function in a virtual machine is inconsistent with that of the physical host, requiring users to adapt and debug, resulting in high costs for users to use virtual RDMA function.

[0102] In this solution, a virtual RDMA device with the same usage environment as the physical host's RDMA device is deployed in the target object's virtual environment. When the target object needs direct remote memory access, the data processing request is forwarded to the second API interface through the first API interface, and then the virtual RDMA device is accessed directly through the second API interface to realize the RDMA function. The target object can use the provided RDMA service in the same way as on the host in the virtual machine or container without additional adaptation work, realizing the consistency of RDMA devices inside and outside the virtual machine, thereby reducing the threshold for using RDMA in the virtualization environment and improving the efficiency of users using RDMA functions.

[0103] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0104] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0105] Example 2

[0106] According to an embodiment of this application, a data processing apparatus for implementing the above-described data processing method is also provided, as shown in FIG6. The apparatus includes a receiving unit 601, a forwarding unit 602, and an execution unit 603.

[0107] The receiving unit 601 is configured to receive a data processing request initiated by the target object through a first API interface in the virtual environment of the target object, wherein the first API interface is a user-facing interface.

[0108] Forwarding unit 602 is configured to forward data processing requests to a second API interface through a first API interface, wherein the second API interface is an interface for a virtual RDMA device, and the virtual RDMA device has the same device configuration as the RDMA device of the physical host.

[0109] The execution unit 603 is configured to access the virtual RDMA device through the second API interface to execute data processing requests through the virtual RDMA device and obtain data processing results.

[0110] In the data processing apparatus provided in Embodiment 2 of this application, the receiving unit 601 receives a data processing request initiated by the target object in the virtual environment of the target object through a first API interface, wherein the first API interface is a user-facing interface; the forwarding unit 602 forwards the data processing request to a second API interface through the first API interface, wherein the second API interface is an interface for a virtual RDMA device, and the virtual RDMA device has the same device configuration as the RDMA device of the physical host; the execution unit 603 accesses the virtual RDMA device through the second API interface to execute the data processing request through the virtual RDMA device and obtain the data processing result. This solves the technical problem in the related art that the usage mode of the RDMA function in the virtual machine is inconsistent with that of the physical host, requiring users to adapt and debug, resulting in high costs for users to use the virtual RDMA function.

[0111] In this solution, a virtual RDMA device with the same usage environment as the physical host's RDMA device is deployed in the target object's virtual environment. When the target object needs direct remote memory access, the data processing request is forwarded to the second API interface through the first API interface, and then the virtual RDMA device is accessed directly through the second API interface to realize the RDMA function. The target object can use the provided RDMA service in the same way as on the host in the virtual machine or container without additional adaptation work, realizing the consistency of RDMA devices inside and outside the virtual machine, thereby reducing the threshold for using RDMA in the virtualization environment and improving the efficiency of users using RDMA functions.

[0112] Optionally, in the data processing apparatus provided in Embodiment 2 of this application, the apparatus further includes: a first configuration unit, configured to configure the first API interface in a virtual environment before receiving a data processing request initiated by a target object through the first API interface, wherein the first API interface is the same as the first API interface in the physical host; a second configuration unit, configured to configure a second API interface in the virtual environment, wherein the second API interface is the same as the second API interface in the physical host; and a deployment unit, configured to deploy a virtual RDMA device in the virtual environment.

[0113] Optionally, in the data processing apparatus provided in Embodiment 2 of this application, the deployment unit includes: a first registration subunit, configured to register a target bus in a virtual environment based on a Linux device registration interface, wherein the virtual RDMA device is composed of at least a Linux device and an InfiniBand device, and the target bus is used to connect at least the Linux device in the virtual RDMA device; a second registration subunit, configured to register a Linux device in a virtual environment based on a Linux device registration interface and a target bus; and a third registration subunit, configured to register the InfiniBand device in a virtual environment based on an InfiniBand kernel interface, so as to realize the deployment of the virtual RDMA device.

[0114] Optionally, in the data processing apparatus provided in Embodiment 2 of this application, the second registration subunit includes: an acquisition module configured to acquire device parameter information of the RDMA device of the physical host; a construction module configured to construct a first device driver for the Linux device based on the Linux kernel interface; a first generation module configured to generate the Linux device based on the Linux device registration interface, device parameter information and the first device driver, and connect the Linux device to the target bus; and a second generation module configured to generate the directory, file information and configuration information of the Linux device in a virtual file system in a virtual environment to complete the registration of the Linux device.

[0115] Optionally, in the data processing apparatus provided in Embodiment 2 of this application, the second generation module includes: an acquisition submodule, configured to acquire the directory, file information, and configuration information of the Linux device corresponding to the RDMA device of the physical host; and a generation submodule, configured to generate the directory, file information, and configuration information of the Linux device in a virtual file system in a virtual environment based on the directory, file information, and configuration information of the Linux device corresponding to the RDMA device of the physical host, so as to complete the registration of the Linux device.

[0116] Optionally, in the data processing apparatus provided in Embodiment 2 of this application, the third registration subunit includes: a first registration module, configured to register a direct memory access device based on the parameters of the direct memory access device corresponding to the RDMA device of the physical host; a configuration module, configured to configure the InfiniBand device in the virtual environment based on the directory, files, and configuration information of the InfiniBand device corresponding to the RDMA device of the physical host, and associate the operation functions of the InfiniBand device with the virtual environment; and a second registration module, configured to perform registration processing on the InfiniBand device based on the InfiniBand kernel interface, the directory, files, and configuration information of the InfiniBand device in the virtual environment, as well as the operation functions of the InfiniBand device and the direct memory access device.

[0117] Optionally, in the data processing apparatus provided in Embodiment 2 of this application, the apparatus further includes: a judgment unit, configured to determine whether the target device is associated in the RDMA device of the physical host before performing registration processing on the InfiniBand device; and an association unit, configured to associate the target device with the virtual RDMA device in the virtual environment if the target device is associated in the RDMA device of the physical host.

[0118] Optionally, in the data processing apparatus provided in Embodiment 2 of this application, the first registration module includes: a construction submodule, configured to construct a second device driver for registering an InfiniBand device based on the InfiniBand kernel interface; and a registration submodule, configured to register a direct memory access device based on the parameters of the second device driver and the direct memory access device corresponding to the RDMA device of the physical host.

[0119] Optionally, in the data processing apparatus provided in Embodiment 2 of this application, the apparatus further includes: an establishment unit, configured to establish an association between the virtual RDMA device and the RDMA device of the physical host after deploying the virtual RDMA device in the virtual environment; and a processing unit, configured to perform remote direct memory access of the virtual RDMA device based on the association.

[0120] It should be noted that the receiving unit 601, forwarding unit 602, and execution unit 603 mentioned above correspond to steps S201 to S203 in the previous embodiment. The three units and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the previous embodiment. It should also be noted that the above units, as part of the device, can run in the computer terminal 10 provided in the previous embodiment.

[0121] It should be noted that the preferred implementation schemes involved in the above embodiments of this application are the same as the schemes, application scenarios and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0122] Example 3

[0123] Embodiments of this application may provide an electronic device, which may be any one of a group of electronic device terminals. Optionally, in this embodiment, the aforementioned electronic device may also be replaced by a terminal device such as a mobile terminal.

[0124] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0125] In this embodiment, the above-mentioned electronic device can execute the program code of the following steps in the data processing method: in the virtual environment of the target object, receiving a data processing request initiated by the target object through a first API interface, wherein the first API interface is a user-facing interface; forwarding the data processing request to a second API interface through the first API interface, wherein the second API interface is an interface for a virtual RDMA device, the virtual RDMA device having the same device configuration as the RDMA device of the physical host; accessing the virtual RDMA device through the second API interface to execute the data processing request through the virtual RDMA device and obtain the data processing result.

[0126] The aforementioned electronic device can execute program code for the following steps in the data processing method: before receiving a data processing request initiated by a target object through a first API interface, the method further includes: configuring a first API interface in a virtual environment, wherein the first API interface is the same as the first API interface in the physical host; configuring a second API interface in the virtual environment, wherein the second API interface is the same as the second API interface in the physical host; and deploying a virtual RDMA device in the virtual environment.

[0127] The aforementioned electronic device can execute program code for the following steps in the data processing method: Deploying a virtual RDMA device in a virtual environment includes: registering a target bus based on a Linux kernel interface in the virtual environment, wherein the virtual RDMA device consists of at least a Linux device and an InfiniBand device, and the target bus is used to connect at least the Linux device in the virtual RDMA device; registering the Linux device based on a Linux device registration interface and the target bus in the virtual environment; and registering the InfiniBand device based on the InfiniBand kernel interface in the virtual environment to achieve the deployment of the virtual RDMA device.

[0128] The aforementioned electronic device can execute the program code for the following steps in the data processing method: In a virtual environment, registering a Linux device based on the Linux device registration interface and the target bus includes: obtaining device parameter information of the RDMA device of the physical host; building a first device driver for the Linux device based on the Linux kernel interface; generating the Linux device based on the Linux device registration interface, device parameter information, and the first device driver, and connecting the Linux device to the target bus; generating the directory, file information, and configuration information of the Linux device in the virtual file system of the virtual environment to complete the registration of the Linux device.

[0129] The aforementioned electronic device can execute the program code for the following steps in the data processing method: generating the directory, file information, and configuration information of the Linux device in a virtual file system in a virtual environment to register the Linux device, including: obtaining the directory, file information, and configuration information of the Linux device corresponding to the RDMA device of the physical host; and generating the directory, file information, and configuration information of the Linux device in a virtual file system in a virtual environment based on the directory, file information, and configuration information of the Linux device corresponding to the RDMA device of the physical host to complete the registration of the Linux device.

[0130] The aforementioned electronic device can execute the program code for the following steps in the data processing method: Registering an InfiniBand device in a virtual environment based on the InfiniBand kernel interface includes: registering the direct memory access device (DMI) according to the parameters of the DMI corresponding to the RDMA device on the physical host; configuring the directory, files, and configuration information of the InfiniBand device corresponding to the RDMA device on the physical host in the virtual environment, and associating the operation functions of the InfiniBand device with the virtual environment; and performing registration processing on the InfiniBand device based on the InfiniBand kernel interface, the directory, files, and configuration information of the InfiniBand device in the virtual environment, as well as the operation functions of the InfiniBand device and the DMI.

[0131] The aforementioned electronic device can execute the program code for the following steps in the data processing method: before registering the InfiniBand device, the method further includes: determining whether the target device is associated with the RDMA device of the physical host; if the target device is associated with the RDMA device of the physical host, then in the virtual environment, associating the target device with the virtual RDMA device.

[0132] The aforementioned electronic device can execute the program code for the following steps in the data processing method: registering the direct memory access device based on the parameters of the direct memory access device corresponding to the RDMA device of the physical host, including: building a second device driver for registering the InfiniBand device based on the InfiniBand kernel interface; registering the direct memory access device based on the second device driver and the parameters of the direct memory access device corresponding to the RDMA device of the physical host.

[0133] The aforementioned electronic device can execute program code for the following steps in the data processing method: After deploying a virtual RDMA device in a virtual environment, the method further includes: establishing an association between the virtual RDMA device and the RDMA device of the physical host; and performing remote direct memory access of the virtual RDMA device based on the association.

[0134] Optionally, FIG7 is a structural block diagram of an electronic device according to an embodiment of the present application. As shown in FIG7, the electronic device 20 may include: one or more (only one is shown in FIG7) processors 202 and memory 204. The electronic device 20 may also include a memory controller for controlling and managing the memory 204; the electronic device 20 may also include a peripheral interface for connecting to a radio frequency module, an audio module, and a display screen, etc.

[0135] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the data processing method and apparatus in this embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the aforementioned data processing method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the electronic device 20 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0136] The processor can invoke information and applications stored in memory via a transmission device to perform the following steps: In the virtual environment of the target object, receive a data processing request initiated by the target object through a first API interface, wherein the first API interface is a user-facing interface; forward the data processing request to a second API interface through the first API interface, wherein the second API interface is an interface for a virtual RDMA device, and the virtual RDMA device has the same device configuration as the RDMA device of the physical host; access the virtual RDMA device through the second API interface to execute the data processing request through the virtual RDMA device and obtain the data processing result.

[0137] Optionally, the processor may also execute program code for the following steps: before receiving a data processing request initiated by the target object through the first API interface, the method further includes: configuring the first API interface in a virtual environment, wherein the first API interface is the same as the first API interface in the physical host; configuring a second API interface in the virtual environment, wherein the second API interface is the same as the second API interface in the physical host; and deploying a virtual RDMA device in the virtual environment.

[0138] Optionally, the processor may also execute program code for the following steps: Deploying a virtual RDMA device in a virtual environment includes: registering a target bus based on a Linux kernel interface in the virtual environment, wherein the virtual RDMA device consists of at least a Linux device and an InfiniBand device, and the target bus is used to connect at least the Linux device in the virtual RDMA device; registering a Linux device based on a Linux device registration interface and a target bus in the virtual environment; and registering the InfiniBand device based on the InfiniBand kernel interface in the virtual environment to achieve the deployment of the virtual RDMA device.

[0139] Optionally, the processor may also execute program code that performs the following steps: In a virtual environment, registering a Linux device based on the Linux device registration interface and the target bus includes: obtaining device parameter information of the RDMA device on the physical host; building a first device driver for the Linux device based on the Linux kernel interface; generating the Linux device based on the Linux device registration interface, device parameter information, and the first device driver, and connecting the Linux device to the target bus; generating the directory, file information, and configuration information of the Linux device in the virtual file system of the virtual environment to complete the registration of the Linux device.

[0140] Optionally, the processor may also execute program code that performs the following steps: generating directory, file information, and configuration information of a Linux device in a virtual file system within a virtual environment to register the Linux device, including: obtaining the directory, file information, and configuration information of the Linux device corresponding to the RDMA device of the physical host; and generating the directory, file information, and configuration information of the Linux device in a virtual file system within a virtual environment based on the directory, file information, and configuration information of the Linux device corresponding to the RDMA device of the physical host to complete the registration of the Linux device.

[0141] Optionally, the processor may also execute program code with the following steps: Registering the InfiniBand device in a virtual environment based on the InfiniBand kernel interface includes: registering the direct memory access device (DMI) according to the parameters of the DMI corresponding to the RDMA device on the physical host; configuring the directory, files, and configuration information of the InfiniBand device corresponding to the RDMA device on the physical host in the virtual environment, and associating the operation functions of the InfiniBand device with the virtual environment; and performing registration processing on the InfiniBand device based on the InfiniBand kernel interface, the directory, files, and configuration information of the InfiniBand device in the virtual environment, and the operation functions and DMI of the InfiniBand device.

[0142] Optionally, the processor may also execute program code that performs the following steps: before registering the InfiniBand device, the method further includes: determining whether the target device is associated with the RDMA device of the physical host; if the target device is associated with the RDMA device of the physical host, then associating the target device with the virtual RDMA device in the virtual environment.

[0143] Optionally, the processor may also execute program code that performs the following steps: registering a direct memory access device based on the parameters of the direct memory access device corresponding to the RDMA device of the physical host, including: building a second device driver for registering the InfiniBand device based on the InfiniBand kernel interface; and registering the direct memory access device based on the second device driver and the parameters of the direct memory access device corresponding to the RDMA device of the physical host.

[0144] Optionally, the processor may also execute program code for the following steps: after deploying a virtual RDMA device in a virtual environment, the method further includes: establishing an association between the virtual RDMA device and the RDMA device of the physical host; and performing remote direct memory access of the virtual RDMA device based on the association.

[0145] This application provides a data processing solution. In the virtual environment of the target object, a data processing request initiated by the target object is received through a first API interface, which is a user-facing interface. The data processing request is forwarded to a second API interface, which is an interface for a virtual RDMA device. The virtual RDMA device has the same device configuration as the RDMA device on the physical host. The virtual RDMA device is accessed through the second API interface to execute the data processing request and obtain the data processing result. This achieves direct access to virtual remote memory, thus solving the technical problem in related technologies where the usage of RDMA functions in virtual machines differs from that of physical hosts, requiring user adaptation and debugging, resulting in high costs for users using virtual RDMA functions.

[0146] It will be understood by those skilled in the art that the structure shown in Figure 7 is merely illustrative, and the electronic device 20 may also be a smartphone (such as an Android phone, iOS phone, etc.), tablet computer, PDA, mobile internet device (MID), PAD, and other terminal devices. Figure 7 does not limit the structure of the aforementioned electronic device. For example, the electronic device 20 may also include more or fewer components (such as network interfaces, display devices, etc.) than shown in Figure 7, or have a different configuration than that shown in Figure 7.

[0147] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0148] Example 4

[0149] Embodiments of this application also provide a computer program product. Optionally, in this embodiment, the computer program product can be used to store the program code executed by the data processing method provided in Embodiment 1.

[0150] Optionally, in this embodiment, the computer program product may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0151] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0152] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0153] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0154] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0155] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0156] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0157] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application. Industrial applicability

[0158] The data processing method provided in this application deploys a virtual RDMA device with the same usage environment as the RDMA device on the physical host in the virtual environment of the target object. When the target object needs direct remote memory access, it forwards the data processing request to the second API interface through the first API interface, and then directly accesses the virtual RDMA device through the second API interface to realize the RDMA function. The target object can use the provided RDMA service in the virtual machine or container in the same way as on the host, without additional adaptation work, to achieve consistency between RDMA devices inside and outside the virtual machine, thereby reducing the threshold for using RDMA in the virtualization environment and improving the efficiency of users using RDMA functions.

Claims

1. A data processing method, comprising: receiving, in a virtual environment of a target object, a data processing request initiated by the target object through a first API interface, wherein the first API interface is a user-side interface; forwarding the data processing request to a second API interface through the first API interface, wherein the second API interface is a virtual RDMA device-side interface, and the virtual RDMA device has the same device structure as an RDMA device of a physical host; accessing the virtual RDMA device through the second API interface to execute the data processing request through the virtual RDMA device to obtain a data processing result.

2. The method of claim 1, wherein, Before receiving the data processing request initiated by the target object through the first API interface, the method further comprises: configuring the first API interface in the virtual environment, wherein the first API interface is the same as a first API interface in the physical host; configuring the second API interface in the virtual environment, wherein the second API interface is the same as a second API interface in the physical host; deploying the virtual RDMA device in the virtual environment.

3. The method of claim 1, wherein, Deploying the virtual RDMA device in the virtual environment comprises: registering a target bus based on a Linux kernel interface in the virtual environment, wherein the virtual RDMA device is composed of at least a Linux device and an InfiniBand device, and the target bus is used to connect at least the Linux device in the virtual RDMA device; registering the Linux device based on a Linux device registration interface and the target bus in the virtual environment; registering the InfiniBand device based on an InfiniBand kernel interface in the virtual environment to implement deployment of the virtual RDMA device.

4. The method of claim 3, wherein, Registering the Linux device based on the Linux device registration interface and the target bus in the virtual environment comprises: obtaining device parameter information of an RDMA device of the physical host; constructing a first device driver of the Linux device based on a Linux kernel interface; generating the Linux device based on the Linux device registration interface, the device parameter information and the first device driver, and connecting the Linux device to the target bus; generating directory, file information and configuration information of the Linux device in a virtual file system in the virtual environment to complete registration of the Linux device.

5. The method of claim 4, wherein, Generating the directory, file information and configuration information of the Linux device in the virtual file system in the virtual environment to register the Linux device comprises: obtaining the directory, file information and configuration information of the Linux device corresponding to the RDMA device of the physical host; According to the directory, file information and configuration information of the Linux device corresponding to the RDMA device of the physical host, the directory, file information and configuration information of the Linux device are generated in a virtual file system in the virtual environment, so as to complete the registration of the Linux device.

6. The method of claim 4, wherein, In the virtual environment, the InfiniBand device is registered based on the InfiniBand kernel interface. According to the parameters of the direct memory access device corresponding to the RDMA device of the physical host, the direct memory access device is registered. According to the directory, file and configuration information of the InfiniBand device corresponding to the RDMA device of the physical host, the InfiniBand device is configured in the virtual environment, and the operation function of the InfiniBand device is associated to the virtual environment. According to the InfiniBand kernel interface, the directory, file, configuration information of the InfiniBand device in the virtual environment, and the operation function of the InfiniBand device and the direct memory access device, the InfiniBand device is registered.

7. The method of claim 6, wherein, Before the registration processing of the InfiniBand device, the method further comprises: determining whether the target device is associated in the RDMA device of the physical host; if the target device is associated in the RDMA device of the physical host, the target device is associated to the virtual RDMA device in the virtual environment.

8. The method of claim 6, wherein, According to the parameters of the direct memory access device corresponding to the RDMA device of the physical host, the direct memory access device is registered. According to the second device driver and the parameters of the direct memory access device corresponding to the RDMA device of the physical host, the direct memory access device is registered. After the virtual RDMA device is deployed in the virtual environment, the method further comprises:

9. The method of any one of claims 2 to 8, wherein, establishing an association relationship between the virtual RDMA device and the RDMA device of the physical host; According to the association relationship, the remote memory direct access of the virtual RDMA device is performed.

10. A data processing apparatus, comprising: a receiving unit configured to receive a data processing request initiated by a target object in a virtual environment of the target object through a first API interface, wherein the first API interface is a user-side interface; a forwarding unit configured to forward the data processing request to a second API interface through the first API interface, wherein the second API interface is a virtual RDMA device-side interface, and the virtual RDMA device has the same device structure as a RDMA device of a physical host; an execution unit configured to access the virtual RDMA device through the second API interface, so as to execute the data processing request through the virtual RDMA device and obtain a data processing result.

11. An electronic device, comprising: a memory storing an executable program; ​ A processor configured to execute the program, wherein the program performs the data processing method of any one of claims 1 to 9 when the program is executed.

12. A computer program product comprising a stored computer program which, when executed by a processor, performs the data processing method of any one of claims 1 to 9.

Images