Data processing method and device and electronic equipment

Abstract

The embodiment of the invention provides a data processing method and device and electronic equipment, the method is applied to the electronic equipment, a host machine and a virtual machine run in the electronic equipment, and the method comprises the steps that an IO request of an application program of the virtual machine is received through a virtual machine driver; converting the format of the IO request into a format which can be identified by an IO queue of a host machine through a conversion module to obtain a new IO request; writing the new IO request into an IO queue according to the information of the IO queue pre-registered in the virtual machine drive through the virtual machine drive; and obtaining a new IO request in the IO queue through an IO processing module in the host machine, and transmitting the new IO request to a disk of the electronic equipment. Therefore, the path of accessing the disk by the virtual machine can be shortened, and the efficiency of accessing the disk by the virtual machine is improved.

Description

Technical Field

[0001] This application relates to the field of cloud computing technology, and in particular, to a data processing method, apparatus, and electronic device. Background Art

[0002] An electronic device may include a host and virtual machines. Application programs may be installed in both the host and virtual machines. Whether it is an application program in the host or an application program in a virtual machine, it can access the disk of the electronic device to read data from the disk or write data to the disk.

[0003] Currently, when an application program in the host accesses the disk, the IO request sent by the application program can access the disk through the interface, disk file system, IO scheduling layer, block device driver, etc. in the host. When an application program in a virtual machine accesses the disk, the IO request sent by the application program needs to first transmit the IO request to the host through the interface, disk file system, IO scheduling layer, block device driver, etc. in the virtual machine, and then access the disk through the interface, disk file system, IO scheduling layer, block device driver, etc. in the host.

[0004] However, in the prior art, the path for a virtual machine to access the disk is relatively long, which may result in low efficiency of the virtual machine accessing the disk. Summary of the Invention

[0005] Embodiments of this application provide a data processing method, apparatus, and electronic device, which can shorten the path for a virtual machine to access the disk and improve the efficiency of virtual disk access.

[0006] In a first aspect, embodiments of this application provide a data processing method, which is applied to an electronic device. A host and virtual machines are running in the electronic device. The data processing method includes:

[0007] Receiving an IO request of an application program of the virtual machine through a virtual machine driver;

[0008] Converting the format of the IO request into a format recognizable by an IO queue of the host through a conversion module to obtain a new IO request;

[0009] Writing the new IO request into the IO queue through the virtual machine driver according to information of the IO queue pre-registered in the virtual machine driver;

[0010] Obtaining the new IO request from the IO queue through an IO processing module in the host and transmitting the new IO request to a disk of the electronic device.

[0011] In a possible implementation, the conversion module includes a memory address conversion module. By means of the conversion module, the format of the IO request is converted into a format allowed by the IO queue of the host, and a new IO request is obtained, including:

[0012] Through the memory address conversion module, the memory address carried in the IO request is converted into a memory address recognizable by the IO queue, and a new IO request is obtained.

[0013] In a possible implementation, the conversion module further includes a disk address conversion module. By means of the conversion module, the format of the IO request is converted into a format allowed by the IO queue of the host, and a new IO request is obtained, further including:

[0014] Through the disk address conversion module, the disk address carried in the IO request is converted into a disk address recognizable by the IO queue, and the new IO request is obtained.

[0015] In a possible implementation, by means of the disk address conversion module, the disk address carried in the IO request is converted into a disk address recognizable by the IO queue, and the new IO request is obtained, including:

[0016] If the virtual machine disk of the electronic device is a file storage in file format, then through the disk address conversion module, the corresponding disk offset in the IO request is converted into a file offset address recognizable by the IO queue, and the new IO request is obtained;

[0017] If the virtual machine disk of the electronic device is a block storage in block data format, then through the disk address conversion module, the corresponding disk offset in the IO request is converted into a block offset address recognizable by the IO queue, and the new IO request is obtained.

[0018] In a possible implementation, the method further includes:

[0019] Through the passthrough module, the information of the IO queue of the host is obtained, and the information of the IO queue is registered in the virtual machine driver, so that the virtual machine driver can directly access the IO queue.

[0020] In a possible implementation, the new IO request includes a new write IO request. By means of the IO processing module in the host, the new IO request is transmitted to the disk of the electronic device, including:

[0021] By means of the IO processing module in the host, the new write IO request is transmitted to the disk of the electronic device, and the data carried in the new write IO request is written into the disk.

[0022] In a possible implementation, the new I / O request includes a new read I / O request. Transmitting the new I / O request to the disk of the electronic device by the I / O processing module in the host includes:

[0023] Transmitting the new read I / O request to the disk of the electronic device by the I / O processing module in the host, and obtaining data corresponding to the new read I / O request in the disk according to the new read I / O request.

[0024] In a second aspect, an embodiment of the present application provides a data processing device, and the data processing device includes:

[0025] A receiving unit, configured to receive an I / O request of an application program of a virtual machine through a virtual machine driver;

[0026] A conversion unit, configured to convert the format of the I / O request into a format recognizable by an I / O queue of the host through a conversion module, to obtain a new I / O request;

[0027] A processing unit, configured to write the new I / O request into the I / O queue according to information of the I / O queue pre-registered in the virtual machine driver through the virtual machine driver; obtain the new I / O request from the I / O queue through the I / O processing module in the host, and transmit the new I / O request to the disk of the electronic device.

[0028] In a possible implementation, the conversion module includes a memory address conversion module, and specifically, the conversion unit is configured to convert the memory address carried in the I / O request into a memory address recognizable by the I / O queue through the memory address conversion module, to obtain a new I / O request.

[0029] In a possible implementation, the conversion module further includes a disk address conversion module, and specifically, the conversion unit is configured to convert the disk address carried in the I / O request into a disk address recognizable by the I / O queue through the disk address conversion module, to obtain a new I / O request.

[0030] In a possible implementation, specifically, when the virtual machine disk of the electronic device is a file storage in a file format, the conversion unit is configured to convert the corresponding disk offset in the I / O request into a file offset address recognizable by the I / O queue through the disk address conversion module, to obtain the new I / O request;

[0031] The conversion unit is specifically configured to, when the virtual machine disk of the electronic device is a block storage in block data format, convert the corresponding disk offset in the IO request into a block offset address recognizable by the IO queue through the disk address conversion module, so as to obtain the new IO request.

[0032] In a possible implementation, the processing unit is further configured to obtain information of the IO queue of the host through the passthrough module, and register the information of the IO queue into the virtual machine driver, so that the virtual machine driver can directly access the IO queue.

[0033] In a possible implementation, the new IO request includes a new write IO request. The processing unit is specifically configured to transmit the new write IO request to the disk of the electronic device through the IO processing module in the host, and write the data carried in the new write IO request into the disk.

[0034] In a possible implementation, the new IO request includes a new read IO request. The processing unit is specifically configured to transmit the new read IO request to the disk of the electronic device through the IO processing module in the host, and obtain the data corresponding to the new read IO request from the disk according to the new read IO request.

[0035] In a third aspect, an embodiment of the present application further provides an electronic device, which includes: a processor, and a memory communicatively connected to the processor;

[0036] The memory stores computer-executable instructions;

[0037] The processor executes the computer-executable instructions stored in the memory to implement the method in any possible implementation manner of the first aspect described above.

[0038] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the processor executes the computer-executable instructions, the method in any possible implementation manner of the first aspect described above is implemented.

[0039] In a fifth aspect, an embodiment of the present application further provides a computer program product, including a computer program. When the computer program is executed by a processor, the method in any possible implementation manner of the first aspect described above is implemented.

[0040] As can be seen, the embodiments of the present application provide a data processing method, apparatus, and electronic device. The method is applied to an electronic device in which a host and a virtual machine are running, and includes: receiving an IO request of an application program of the virtual machine through a virtual machine driver; converting the format of the IO request into a format recognizable by an IO queue of the host through a conversion module to obtain a new IO request; writing the new IO request into the IO queue through the virtual machine driver according to the information of the IO queue pre-registered in the virtual machine driver; and obtaining the new IO request from the IO queue through an IO processing module in the host and transmitting the new IO request to the disk of the electronic device. In this way, the IO queue can directly process the IO requests written to the virtual machine disk, so that the IO requests do not need to pass through other modules of the host and then reach the IO queue, which can shorten the path for the virtual machine to access the disk. Moreover, the format of the IO requests written to the IO queue is a format recognizable by the IO queue, so that the IO queue can recognize the IO requests and process them. Therefore, the data processing method provided by the embodiments of the present application can improve the efficiency of the virtual machine accessing the disk. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 FIG. is a schematic diagram of a path for a virtual machine to access a disk provided by an embodiment of the present application;

[0042] Figure 2 FIG. is a schematic flowchart of a data processing method provided by an embodiment of the present application;

[0043] Figure 3 FIG. is a schematic diagram of data transmission of a data processing method provided by an embodiment of the present application;

[0044] Figure 4 FIG. is a schematic diagram of a transmission path of an IO request provided by an embodiment of the present application;

[0045] Figure 5 [[ID=XX]]FIG. is a schematic structural diagram of a data processing apparatus provided by an embodiment of the present application;

[0046] Figure 6 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

[0047] Through the above-mentioned drawings, specific embodiments of the present disclosure have been shown, and more detailed descriptions will be given later. These drawings and textual descriptions are not intended to limit the scope of the concept of the present disclosure in any way, but to illustrate the concept of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numerals in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0049] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects and indicates that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. In the written description of the present application, the character " / " generally represents an "or" relationship between the associated objects before and after.

[0050] In the field of cloud computing, an electronic device may include a host and a virtual machine (VM for short). Both the host and the virtual machine can access the data stored in the disk of the electronic device. The virtual machine disk corresponding to the virtual machine is not a real disk and is usually a file or a block device.

[0051] Currently, when an application program in the host accesses the disk and reads and writes data in the disk, the path can be: application program → system interface → virtual file system → disk file system → general block layer → IO scheduler layer → block device driver → disk.

[0052] It should be noted that the IO scheduler layer may include multiple IO queues, and the multiple IO queues can be used to store IO requests.

[0053] Since the disk corresponding to the virtual machine of the electronic device is not a real disk but a file or a block device. Therefore, the path for an application program in the virtual machine to access the disk can be referred to Figure 1 as shown. Figure 1 It is a schematic diagram of the path for a virtual machine to access a disk provided by an embodiment of the present application.

[0054] As Figure 1As shown in the figure, when an application in a virtual machine accesses a disk and reads and writes data on the disk, the path can be: Application (virtual machine) → System interface (virtual machine) → Virtual file system (virtual machine) → Disk file system (virtual machine) → General block layer (virtual machine) → IO scheduling layer (virtual machine) → Block device driver (virtual machine) → VM exit → kvm module (host) => QEMU application (host) → System interface (host) → Virtual file system (host) → Disk file system (host) → General block layer (host) → IO scheduling layer (host) → Block device driver (host) → Disk.

[0055] As Figure 1 shown, VM exit can transfer the IO requests of the virtual machine disk driver in the virtual machine to the kvm module. The kvm module can intercept the IO requests and hand them over to the Qemu application. The Qemu application can modify the information of the IO requests and convert the information of the IO requests into information recognizable by the host.

[0056] It should be noted that Figure 1 the block device in

[0057] is the disk driver of the virtual machine and can transfer the IO requests in the virtual machine driver to the kvm module. Figure 1 Of course, VM exit can also transfer the IO requests of the virtual machine block device driver to the kvm module. The embodiments of this application only use

[0058] As Figure 1 shown, the path for the virtual machine to read and write data on the disk is different from the path for the host to read and write data on the disk. When the virtual machine reads and writes data, there is a consumption of context switching during VM exit in the virtual machine, resulting in a certain gap between the read and write performance of the virtual machine and the direct read and write performance of the host. Compared with the path for an application in the host to access the disk, the path for an application in the virtual machine to access the disk is longer, which may lead to a longer access time and a longer data acquisition time. Therefore, the efficiency of the virtual machine accessing the disk is relatively low.

[0059] In order to improve the efficiency of virtual machine access to disks, in some implementations, a request acceleration module can be set between the virtual machine and the backend storage. This request acceleration module connects the virtual machine and the backend storage and processes IO requests by adopting the method of shared memory and request queue polling. Specifically, when the virtual machine generates an IO request, the IO request is written into the IO request queue in the shared memory, and the IO request queue is a circular queue. The electronic device can obtain the IO request from the IO request queue through a polling process, determine the actual storage end address of the corresponding IO data according to the IO request, and access the disk through interfaces, disk file systems, IO scheduling layers, block device drivers, etc. in the host machine.

[0060] However, in the above implementation, although the efficiency of disk access can be improved to a certain extent by the method of shared memory and request queue polling, the path of the IO request is still long, and there is still a problem of low efficiency of virtual machine access to disks.

[0061] Based on this, the embodiments of the present application provide a data processing method. The information of the IO queue is pre-registered in the virtual machine driver. When the virtual machine driver receives an IO request, it can directly write the IO request into the IO queue according to the pre-registered information of the IO queue. In this way, the IO queue can directly process the IO requests written to the virtual machine disk, so that the IO requests do not need to pass through other modules of the host machine and then reach the IO queue, which can shorten the path of virtual machine access to disks. And, the format of the IO requests written into the IO queue is the format that the IO queue can recognize. In this way, the IO queue can recognize the IO requests and process the IO requests. Therefore, the data processing method provided by the embodiments of the present application can improve the efficiency of virtual machine access to disks.

[0062] Next, the data processing method provided by the present application will be described in detail through specific embodiments. It can be understood that these specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0063] Figure 2 It is a schematic flowchart of a data processing method provided by an embodiment of the present application. This data processing method can be executed by software and / or hardware devices. For example, the hardware device can be a data processing device, and the data processing device can be an electronic device or a processing chip in the electronic device.

[0064] It should be noted that a host machine and a virtual machine can run in the electronic device, and the number of virtual machines can be one or more. The embodiments of the present application do not limit this.

[0065] For example, please refer to Figure 2 As shown, this data processing method can include:

[0066] S201. Receive an I / O request of an application program of a virtual machine through a virtual machine driver.

[0067] Exemplarily, the virtual machine driver is a driver in the virtual machine of an electronic device for receiving an I / O request of the I / O scheduling layer of the virtual machine, and can be Figure 1 the block device driver or the virtual machine disk driver in , and the embodiments of the present application do not limit this.

[0068] As Figure 1 shown, the I / O request of the application program can be transmitted to the virtual machine driver through the system interface of the virtual machine, the virtual file system of the virtual machine, the disk file system of the virtual machine, the general block layer of the virtual machine, and the I / O scheduling layer of the virtual machine in sequence.

[0069] S202. Convert the format of the I / O request into a format recognizable by the I / O queue of the host machine through a conversion module, and obtain a new I / O request.

[0070] The I / O queue of the host machine can be the I / O queue in the I / O scheduling layer or the general block layer of the host machine. There can be multiple I / O queues in the I / O scheduling layer of the host machine. The I / O queue in the present application can be the queue for caching the I / O requests of the virtual machine among multiple queues, and the embodiments of the present application do not limit this.

[0071] In the embodiments of the present application, the conversion module can include a memory address conversion module. The electronic device can convert the memory address carried in the I / O request into a memory address recognizable by the I / O queue through the memory address conversion module, and obtain a new I / O request.

[0072] Exemplarily, the memory address conversion module can convert the GVA corresponding to buf in the I / O request into HVA. Other forms of memory address conversion are obtained, and the embodiments of the present application do not limit this.

[0073] In this way, the memory address of the I / O request of the virtual machine is converted into a memory address recognizable by the I / O queue through the memory address conversion module, so that the I / O queue can process the I / O request.

[0074] In the embodiments of the present application, the conversion module can further include a disk address conversion module. The electronic device can convert the disk address carried in the I / O request into a disk address recognizable by the I / O queue through the disk address conversion module, and obtain a new I / O request.

[0075] In this way, the disk address of the I / O request of the virtual machine is converted into a disk address recognizable by the I / O queue through the memory address conversion module, so that the I / O queue can process the I / O request.

[0076] Exemplarily, the disk address conversion module may convert the corresponding disk offset in the IO request into the offset address in the storage format corresponding to the virtual machine disk, that is, convert it into the offset address of the actual host-side storage device.

[0077] Since the virtual machine disk and the disk may be file storage in file format or block storage in block data format, therefore, the disk address conversion module may convert the corresponding disk offset in the IO request into the offset address in the storage format corresponding to the virtual machine disk, which may include the following two possible implementations:

[0078] In one possible implementation, if the virtual machine disk is file storage in file format, the disk address conversion module may convert the corresponding disk offset in the IO request into a file offset address recognizable by the IO queue. For example, if the virtual machine backend is file storage, it is correspondingly converted into a file offset, and the offset address is the file offset address.

[0079] In another possible implementation, if the virtual machine disk is block storage in block data format, the disk address conversion module may convert the corresponding disk offset in the IO request into a block offset address recognizable by the IO queue. For example, if the virtual machine backend is block storage, it is correspondingly converted into a block offset, and the offset address is the block offset address.

[0080] The embodiments of the present application do not make specific limitations on the disk address.

[0081] In this way, when the electronic device performs conversion, it can perform different conversions according to the different storage formats corresponding to the virtual machine disks of the virtual machines.

[0082] S203. Through the virtual machine driver, write the new IO request into the IO queue according to the information of the IO queue pre-registered in the virtual machine driver.

[0083] In the embodiments of the present application, the electronic device may obtain the information of the IO queue of the host machine through the passthrough module and register the information of the IO queue in the virtual machine driver so that the virtual machine driver can directly access the IO queue.

[0084] Exemplarily, the passthrough module may register the information of multiple IO queues in the virtual machine driver, and the multiple IO queues correspond to different services or different application programs, so that the virtual machine writes the IO request into different IO queues according to the different application programs or services to which the IO request belongs.

[0085] It can be understood that the information of the IO queue may include information such as the address information of the IO queue and the identifier of the IO queue, and the embodiments of the present application do not make limitations on this.

[0086] In this way, the IO queue of the host kernel is directly provided to the virtual machine through pass-through, reducing the path of IO requests and context switching, and improving the IO read / write performance of the virtual machine.

[0087] S204. Through the IO processing module in the host, obtain a new IO request from the IO queue, and transmit the new IO request to the disk of the electronic device.

[0088] In a possible implementation, the new IO request can be a new write IO request. The electronic device can transmit the new write IO request to the disk of the electronic device through the IO processing module in the host, and write the data carried in the new write IO request into the disk.

[0089] In another possible implementation, the new IO request can also be a new read IO request. The electronic device can transmit the new read IO request to the disk of the electronic device through the IO processing module in the host, and obtain the data corresponding to the new read IO request from the disk according to the new read IO request.

[0090] As described above Figure 1 The IO processing module in the host can be the block device driver or disk driver in the host, and the embodiments of the present application do not limit this.

[0091] In this way, the IO processing module in the host can directly process the IO requests in the IO queue, shortening the processing time of the IO requests by shortening the transmission path of the IO requests, and improving the processing efficiency of the IO requests.

[0092] It can be seen that in the data processing method provided by the embodiments of the present application, the information of the IO queue is pre-registered in the virtual machine driver. When the virtual machine driver receives an IO request, it can directly write the IO request into the IO queue according to the pre-registered information of the IO queue. In this way, the IO queue can directly process the IO requests written to the virtual machine disk, so that the IO requests do not need to pass through other modules of the host and then reach the IO queue, shortening the path for the virtual machine to access the disk. And, the format of the IO requests written into the IO queue is the format that the IO queue can recognize. In this way, the IO queue can recognize the IO requests and process the IO requests. Therefore, the data processing method provided by the embodiments of the present application can improve the efficiency of the virtual machine accessing the disk.

[0093] In an embodiment of the present application, after the host processes the IO requests of the virtual machine, the processing results can be cached in the IO completion queue. The virtual machine driver can obtain the processing results of the IO requests in the IO completion queue by accessing it regularly. For example, when the IO request is a write IO request, the data write result of the write IO request can be obtained in the IO completion queue. When the IO request is a read IO request, the data read from the disk for the read IO request can be obtained in the IO completion queue.

[0094] To facilitate understanding of the data processing method provided in the embodiments of the present application, hereinafter, through Figure 3 the modules shown below, taking the virtual machine disk driver as an example of the virtual machine driver, the data processing method provided in the embodiments of the present application will be described in detail. Figure 3 It is a schematic diagram of data transmission for a data processing method provided in an embodiment of the present application.

[0095] As Figure 3 shown, an electronic device may include a virtual machine disk driver, a memory address translation module, a disk address translation module, a passthrough module, a kernel IO queue, and a kernel IO processing module. Among them, the virtual machine disk driver is in the virtual machine of the electronic device, and the kernel IO queue and the kernel IO processing module are both in the host of the electronic device. The memory address translation module, the disk address translation module, and the passthrough module can be in the virtual machine, can also be in the host, or can be a process independent of the host and the virtual machine. The embodiments of the present application do not limit this.

[0096] The passthrough module can be used to obtain information about the kernel's IO queue and register the information of the IO queue into the disk driver in the virtual machine operating system, so that the virtual machine disk driver can directly write the IO request into the corresponding kernel IO queue.

[0097] The memory address translation module can be used to convert the GVA corresponding to the buf in the IO request into HVA.

[0098] The disk address translation module can be used to convert the disk offset corresponding to the IO request into the offset address of the actual host-side storage device.

[0099] The virtual machine disk driver can be used to interact with the IO passthrough module during initialization to obtain kernel IO queue information and register it into the corresponding address space of the driver. When reading and writing IO, it calls the memory address translation module and the disk address translation module to convert the buf address and disk offset of the IO request into the address that the host kernel can process and the backend storage offset. And directly write the new converted IO request into the IO queue.

[0100] Exemplarily, taking the kernel IO queue as the kernel io_uring queue and the backend storage as the raw file format, the data processing method provided by the embodiments of this application may include the following steps:

[0101] Step 1: The passthrough module registers the raw file corresponding to the virtual machine with the io_uring of the kernel, obtains the corresponding IO queue, and registers the IO queue with the disk driver of the virtual machine.

[0102] Step 2: The virtual machine disk driver converts the address of the IO request buf into the address of the host-side QEMU process through the memory address conversion module. For example, it converts the GVA of the IO request into HVA.

[0103] Step 3: The virtual machine disk driver converts the IO offset in the IO request into the offset address of the backend raw file through the disk address conversion module.

[0104] Step 4: The disk driver of the virtual machine writes the converted IO request into the IO queue of the kernel.

[0105] Step 5: After detecting the IO request in the IO queue, the io_uring mechanism of the host kernel processes it directly according to the request information, and puts the processed result back into the completion queue of the IO queue.

[0106] Step 6: The virtual machine disk driver obtains the result from the completion queue, and finally completes the IO process.

[0107] Based on the above steps, the transmission path of the IO request is as Figure 4 shown, Figure 4 which is a schematic diagram of the transmission path of an IO request provided by the embodiments of this application.

[0108] As Figure 4 shown, the transmission path of the IO request is: virtual machine application program → kernel IO path of the virtual machine → virtual machine disk driver → kernel IO queue → host kernel IO path → disk.

[0109] As can be Figure 4 seen, the IO request of the virtual machine skips the QEMU device, the QEMU block layer, and the io engine used by QEMU (i.e., the host application part). And it reduces the context switching when the VM exit delivers the IO request to the kvm module, achieving the effect of improving the IO performance of the virtual machine.

[0110] In the embodiments of the present application, the rates in the following various situations are detected. The rate at which the host directly uses io_uring for raw files is 7640; the rate at which the host directly uses libaio for raw files is 5698; the rate at which the virtual machine uses iothread is 5043; the rate at which the virtual machine uses native (aio) is 5329; the rate at which the virtual machine uses the io_uring passthrough method is 7457.

[0111] From the above rates, it can be seen that when using a raw file as the virtual machine disk, using io_uring itself improves the performance by 35% compared to aio, and after io_uring passthrough, the IO read performance reaches 97% of the host. Therefore, the embodiments of the present application can effectively improve the processing efficiency of IO requests.

[0112] It should be noted that the form of the backend storage not only supports raw files, but can also be other formats, such as qcow2 files, block storage, and network distributed storage, etc., as long as the offset address corresponding to the io request can be recognized by the backend. The embodiments of the present application only take raw files as an example for illustration and do not constitute any limitation. Also, the IO queue of the kernel can not only be obtained through io_uring, but other io engines can be used, as long as the corresponding queue is registered to the disk driver of the virtual machine, and the disk driver can directly access the IO queue of the kernel. The embodiments of the present application only take io_uring as an example for illustration and do not constitute any limitation.

[0113] In summary, the data processing method provided by the embodiments of the present application shortens the IO path of the virtual machine and improves the IO performance. Also, it reduces VM exit, reduces the consumption of the virtualization layer, and thus improves the IO performance.

[0114] Figure 5 It is a schematic structural diagram of a data processing device 50 provided by an embodiment of the present application. For example, please refer to Figure 5 As shown, the data processing device 50 may include:

[0115] A receiving unit 501, configured to receive an IO request of an application program of a virtual machine through a virtual machine driver.

[0116] A conversion unit 502, configured to convert the format of the IO request into a format recognizable by the IO queue of the host through a conversion module, and obtain a new IO request.

[0117] A processing unit 503 is configured to write a new I / O request into an I / O queue according to the information of the pre-registered I / O queue in the virtual machine driver through the virtual machine driver; and obtain the new I / O request from the I / O queue through the I / O processing module in the host machine and transmit the new I / O request to the disk of the electronic device.

[0118] In a possible implementation, the conversion unit 502 is specifically configured to convert the memory address carried in the I / O request into a memory address recognizable by the I / O queue through the memory address conversion module included in the conversion module, so as to obtain a new I / O request.

[0119] In a possible implementation, the conversion unit 502 is specifically configured to convert the disk address carried in the I / O request into a disk address recognizable by the I / O queue through the disk address conversion module included in the conversion module, so as to obtain a new I / O request.

[0120] In a possible implementation, when the virtual machine disk of the electronic device is a file storage in a file format, the conversion unit 502 is specifically configured to convert the corresponding disk offset in the I / O request into a file offset address recognizable by the I / O queue through the disk address conversion module, so as to obtain a new I / O request.

[0121] The conversion unit 502 is specifically configured to convert the corresponding disk offset in the I / O request into a block offset address recognizable by the I / O queue through the disk address conversion module when the virtual machine disk of the electronic device is a block storage in a block data format, so as to obtain a new I / O request.

[0122] In a possible implementation, the processing unit 503 is further configured to obtain the information of the I / O queue of the host machine through the passthrough module and register the information of the I / O queue into the virtual machine driver, so that the virtual machine driver can directly access the I / O queue.

[0123] In a possible implementation, the new I / O request includes a new write I / O request. The processing unit 503 is specifically configured to transmit the new write I / O request to the disk of the electronic device through the I / O processing module in the host machine and write the data carried in the new write I / O request into the disk.

[0124] In a possible implementation, the new I / O request includes a new read I / O request. The processing unit 503 is specifically configured to transmit the new read I / O request to the disk of the electronic device through the I / O processing module in the host machine and obtain the data corresponding to the new read I / O request from the disk according to the new read I / O request.

[0125] The data processing device provided by the embodiments of the present application can execute the technical solutions of the data processing methods in any of the foregoing embodiments. Its implementation principle and beneficial effects are similar to those of the data processing methods. For details, refer to the implementation principle and beneficial effects of the data processing methods, which will not be elaborated here.

[0126] Figure 6 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 6 shown, the electronic device 60 may include: at least one processor 601 and a memory 602.

[0127] The memory 602 is used to store programs. Specifically, the program may include program codes, and the program codes include computer operation instructions.

[0128] The memory 602 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0129] The processor 601 is configured to execute the computer execution instructions stored in the memory 602 to implement the data processing method described in the foregoing method embodiments. The processor 601 may be a central processing unit (CPU), or a specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. Specifically, when implementing the data processing method described in the foregoing method embodiments, the electronic device may be an electronic device with processing functions, such as a terminal or a server.

[0130] Optionally, the electronic device 60 may further include a communication interface 603. In a specific implementation, if the communication interface 603, the memory 602, and the processor 601 are implemented independently, the communication interface 603, the memory 602, and the processor 601 may be connected to each other through a bus and communicate with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0131] Optionally, in a specific implementation, if the communication interface 603, the memory 602, and the processor 601 are integrated on a single chip, the communication interface 603, the memory 602, and the processor 601 can communicate through an internal interface.

[0132] The present application also provides a computer-readable storage medium, which may include: various media such as USB flash drives, external hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes. Specifically, the computer-readable storage medium stores program instructions, and the program instructions are used for the methods in the above embodiments.

[0133] The present application also provides a program product, which includes execution instructions stored in a readable storage medium. At least one processor of the electronic device can read the execution instructions from the readable storage medium, and the at least one processor executes the execution instructions to enable the electronic device to implement the data processing methods provided by the above various implementation manners.

[0134] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present application.

Claims

1. A data processing method, characterized in that: Applied to an electronic device, wherein a host machine and a virtual machine are running in the electronic device, the data processing method includes: Receiving, via a virtual machine driver, an IO request from an application of the virtual machine; The conversion module converts the format of the IO request into a format recognizable by the IO queue of the host machine to obtain a new IO request; Writing the new IO request into the IO queue according to the information of the IO queue pre-registered in the virtual machine driver through the virtual machine driver; The new IO request is obtained from the IO queue through the IO processing module in the host machine, and the new IO request is transmitted to the disk of the electronic device.

2. The method according to claim 1, characterized in that The conversion module includes a memory address conversion module, and the conversion module converts the format of the IO request into a format allowed by the IO queue of the host machine to obtain a new IO request, including: The memory address conversion module converts the memory address carried in the IO request into a memory address that can be recognized by the IO queue to obtain the new IO request.

3. The method according to claim 2, characterized in that The conversion module further includes a disk address conversion module, which converts the format of the IO request into a format allowed by the IO queue of the host machine to obtain a new IO request, and further includes: The disk address conversion module converts the disk address carried in the IO request into a disk address that can be recognized by the IO queue to obtain the new IO request.

4. The method according to claim 3, characterized in that The step of converting the disk address carried in the IO request into a disk address recognizable by the IO queue by the disk address conversion module to obtain the new IO request includes: If the virtual machine disk of the electronic device is a file storage in a file format, the disk address conversion module converts the disk offset corresponding to the IO request into a file offset address recognizable by the IO queue to obtain the new IO request; If the virtual machine disk of the electronic device is a block storage in a block data format, the disk address conversion module converts the corresponding disk offset in the IO request into a block offset address recognizable by the IO queue to obtain the new IO request.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: The information of the IO queue of the host machine is obtained through the transparent transmission module, and the information of the IO queue is registered in the virtual machine driver, so that the virtual machine driver can directly access the IO queue.

6. The method according to any one of claims 1 to 4, characterized in that The new IO request includes a new write IO request, and the transmitting, through the IO processing module in the host machine, the new IO request to the disk of the electronic device includes: The new write IO request is transmitted to the disk of the electronic device through the IO processing module in the host machine, and the data carried in the new write IO request is written into the disk.

7. The method according to any one of claims 1 to 4, characterized in that The new IO request includes a new read IO request, and the transmitting, through the IO processing module in the host machine, the new IO request to the disk of the electronic device includes: The new read IO request is transmitted to the disk of the electronic device through the IO processing module in the host machine, and data corresponding to the new read IO request is obtained from the disk according to the new read IO request.

8. A data processing device, characterized in that: include: A receiving unit, configured to receive an IO request from an application of the virtual machine through a virtual machine driver; a processing unit configured to convert, through a conversion module, a format of the IO request into a format recognizable by an IO queue of the host machine to obtain a new IO request; and write, through the virtual machine driver, the new IO request into the IO queue according to information of the IO queue pre-registered in the virtual machine driver; The new IO request is obtained from the IO queue through the IO processing module in the host machine, and the new IO request is transmitted to the disk of the electronic device.

9. An electronic device, characterized in that: comprising a memory and a processor; wherein, The memory is used to store computer programs; The processor is configured to read the computer program stored in the memory and execute the data processing method according to any one of claims 1 to 7 according to the computer program in the memory.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, it implements a data processing method according to any one of claims 1 to 7.

Images