A method for virtual machine disk partitioning and accelerated access based on NVMe disk

Abstract

The application discloses a virtual machine disk partitioning and accelerated access method based on an NVMe disk, which comprises the following steps: step one, partitioning a virtual machine disk, specifying a virtual machine disk partition format and a virtual machine disk metadata format, and specifying the layout of the partition on a GPT partition table; recording the first N-1 partition formats as DVMFS and the last partition format as VMFR in the first and last two GPT records of the GPT partition table of a host computer, wherein the VMFR contains a pointer to the DVMFS; and step two, performing NVMe disk transmission in a Hypervisor. The application proposes a virtual machine data partition format DVMFS and a metadata management partition format VMFR, which effectively solve the problems that a direct access cannot use the same disk for multiple tenants and a file mode management virtual machine disk cannot fully exert the performance of an NVMe disk.

Description

TECHNICAL FIELD

[0001] The application relates to a virtual machine disk partitioning and acceleration method, and belongs to the technical field of NVMe solid state disks. BACKGROUND

[0002] The widespread use of NVMe solid state disks has significantly improved the performance of cloud servers, but the cloud servers are generally used in a virtualized manner, and a virtual machine disk is mapped to a file on a host computer using a virtualization layer. Since file reading and writing need to pass through a system call of the host computer, the performance advantage of the NVMe solid state disk cannot be fully exerted in this way. Meanwhile, due to the large use of NVMe disks in the U.2 form, the capacity of the NVMe disk is getting larger and larger, and the situation that a virtual machine exclusively occupies an entire NVMe disk device is becoming unrealistic.

[0003] To solve the above performance problems, the prior art uses a virtual machine Virtio driver to create a virtual disk device in the virtual machine, and transmits the write instruction of the virtual disk to the NVMe disk of the host computer. However, this method cannot allow multiple virtual machines to share the same NVMe disk. To solve this problem, a file management method is used, for example, ESXi uses the VMDK format to store virtual machine disks, and QEMU uses the QCOW2 format to store virtual machine disks. Certain optimization is made for the actual space occupied by the virtual machine disk, but the performance is still compromised. SUMMARY

[0004] The technical problem to be solved by the application is how to improve the isolation and security of the disk between multiple tenants for the NVMe disk and the virtual disk.

[0005] To solve the above technical problems, the application provides a virtual machine disk partitioning and acceleration method based on an NVMe disk, which comprises the following steps:

[0006] Step 1. Partitioning the virtual machine disk, specifying the virtual machine disk partition format and the virtual machine disk metadata format, and specifying the layout of the partition on the GPT partition table; recording the first N-1 partition formats as DVMFS and the last partition format as VMFR in the first and last two GPT records of the GPT partition table of the host computer, and the VMFR contains a pointer to the DVMFS;

[0007] Step 2: NVMe disk transmission in Hypervisor.

[0008] The foregoing virtual machine disk partitioning and acceleration method based on an NVMe disk, in step 1, the DVMFS is the area for storing the virtual machine virtual disk, which comprises the following fields:

[0009] 1) FS Header: Header for identifying the file system;

[0010] 2) Metadata: Information of the current partition is saved;

[0011] 3) Next File System Ptr and Prev File System Ptr are used to indicate the LBA address of the previous and next continuous space on the current partition for fast addressing.

[0012] The foregoing method for partitioning and accelerating access of a virtual machine disk based on an NVMe disk, in step 1, in the Hypervisor, adjusts the disk capacity of the virtual machine by using Algorithm 1, including operations of expanding and shrinking the disk, including the following steps:

[0013] 1) The ADJUSTVMDISK(vmdisk, dst_size, force = false) function is called, and the input parameters include the virtual disk object vmdisk, the target size dst_size, and a forced parameter force;

[0014] 2) If dst_size is less than or equal to 0, a parameter error exception ParamErrorException is thrown;

[0015] 3) If the target size dst_size is less than the current disk size, and force is true, the SHRINKVMDISK(vmdisk, dst_size) function is called to perform the shrinking operation;

[0016] 4) If force is false, and the target size is less than the current disk size, a data loss exception DataLossException is thrown;

[0017] 5) If the target size dst_size is greater than the current disk size, the EXPANDVMDISK(vmdisk, dst_size) function is called to perform the expanding operation.

[0018] The foregoing method for partitioning and accelerating access of a virtual machine disk based on an NVMe disk, in step 3), the disk shrinking process SHRINKVMDISK(vmdisk, dst_size) includes:

[0019] 31) The size to be shrunk nr_size is calculated, that is, the current size minus the target size;

[0020] 32) When nr_size is greater than 0, the following steps are performed:

[0021] Recursively access the partition pointed by NextPtr of the partition until the last host partition of the virtual disk is found;

[0022] Check if the size of the last host partition of the virtual disk is greater than or equal to nr_size;

[0023] If yes, call the CutDVMFS function to cut, and set nr_size to 0; the CutDVMFS function sets the size of the target partition to

original size - nr_size

[0024] If not, subtract the size of the partition from nr_size, delete the partition, delete the VMFR partition metadata of the partition, set the NextPtr of the previous DVMFS partition to NULL, and finally update the occupied space bit table on the VMFR.

[0025] The foregoing method for partitioning and accelerating access of a virtual machine disk based on an NVMe disk, in step 5), the EXPANDVMDISK(vmdisk, dest_size) expansion disk process includes:

[0026] 51) Calculate the size nr_size that needs to be expanded, i.e., the target size minus the current size;

[0027] 52) Check the available space of the host, and if it is insufficient, throw a NoEnoughSpaceException exception of insufficient space, and the operation ends;

[0028] 53) When nr_size is greater than 0, perform the following steps:

[0029] Find the last host partition of the virtual disk;

[0030] Get the available space outside the last host partition of the virtual disk by finding the unused space from the occupied space bit table of the VMFR;

[0031] If the available space is greater than or equal to nr_size, expand the partition and set nr_size to 0, and update the occupied space bit table in the VMFR;

[0032] If the available space is insufficient, recursively find the free space block and add new partition as needed, update nr_size as nr_size-current free space size, and update the occupied space bit table in VMFR.

[0033] The foregoing method for partitioning and accelerating access of a virtual machine disk based on an NVMe disk, in step one, uses the Consistency bit in the VMFR for each DVMFS to determine whether it is correctly written. If the Consistency bit is set to 1, it indicates that the metadata of the partition is modified and not correctly written. According to the condition of the Consistency bit, the modification of the DVMFS or the modification of the VMFR is selected. The method is as follows:

[0034] Condition 1: Check whether the existing checksum is the same as the checksum recalculated from the current partition FS Header;

[0035] Condition 2: Whether the Consistency bit in the VMFR is 0;

[0036] If conditions 1 and 2 are not met, the metadata of the VMFR is accepted. If condition 1 is met and condition 2 is not met, the metadata of the DVMFS is accepted. If condition 1 is not met and condition 2 is met, the metadata of the VMFR is accepted. If condition 1 is met and condition 2 is met, there is no data inconsistency behavior.

[0037] The foregoing method for partitioning and accelerating access of a virtual machine disk based on an NVMe disk, in step two, includes the following steps:

[0038] 21) An intermediate layer DNVMe Driver is added between the Hypervisor and the VFIO-pci. The intermediate layer uses IOMMU to isolate the device address. According to the physical address LBA of the first DVMFS to which the virtual disk of the virtual machine belongs, the LBA mapping block of each virtual machine disk is obtained, and the LBA mapping block of each virtual machine disk is registered with IOMMU. The start LBA address and the end LBA address of each DVMFS are registered one by one, and the start LBA address and the end LBA address of each DVMFS are mapped into a continuous virtual address space.

[0039] 22) Directly obtain physical address through IOMMU for virtual machine disk access, NVMe disk, each NVMe disk is provided with an Admin QPair and a plurality of I / O QPair for sending and transmitting management instructions, data operation instructions, the NVMe disk contains physical NVMe disk and virtual NVMe disk;In the Hypervisor, the NVMe disk device of each virtual machine uses the way of host software to simulate an NVMe admin qpair, and according to the metadata information of virtual machine disk, it is decided whether the admin command has corresponding permission to execute.

[0040] The foregoing method for partitioning and accelerating access of virtual machine disks based on NVMe disks, in step 22), the virtual NVMe disk has at most the same number of Admin qpair and I / O qpair as the physical NVMe disk, the admin qpair is taken over by DNVMe Driver, and the I / O qpair is transmitted to VFIO-pci driver.

[0041] The I / O qpair processes all normal NVMe read and write instructions, and converts virtual device addresses into actual physical disk addresses;Using IOMMU, first convert the virtual address into the physical address of the physical NVMe disk, and then put the command located in the qpair into the sq queue of the physical NVMe disk;The mapping relationship between virtual address and physical address is submitted to IOMMU according to the first and last addresses of DVMFS data partition at initialization.

[0042] The foregoing method for partitioning and accelerating access of virtual machine disks based on NVMe disks, an LBA address conversion table mechanism is provided, and the LBA address conversion table is saved in memory for quick lookup;If the LBA address mapping information cannot be found in IOMMU, the current mapping information is obtained from the LBA address conversion table, and the LBA address conversion table is constructed using the first and last addresses of each DVMFS data partition at initialization of the host physical NVMe disk.

[0043] The foregoing method for partitioning and accelerating access of virtual machine disks based on NVMe disks, the minimum unit of mapping between virtual qpair and physical qpair is queue, and the queues of virtual NVMe disk are mapped to the queues of physical NVMe disk in one-to-one correspondence, qpair0 of the starting virtual queue is mapped to different qpair of physical NVMe disk according to the initialization order of virtual machine, and then the following queues are mapped in order.

[0044] The foregoing method for partitioning and accelerating access of virtual machine disks based on NVMe disks, the LBA mapping list and the IOMMU mapping are linked, and the steps are:

[0045] When the I / O request reaches the IOMMU, the IOMMU first converts the virtual address of the I / O request into a physical address while checking the source virtual machine of the I / O request and the virtual machine occupying the virtual disk to which the corresponding physical address belongs, using the unique number of the virtual machine for the checking, if the unique number of the source virtual machine of the request is the same as the unique number of the virtual machine occupying the virtual disk to which the corresponding physical address belongs, the I / O request proceeds normally, if the virtual machines are found to be mismatched, an error is returned and the execution of the instruction is terminated.

[0046] The present application achieves the beneficial effects: in order to solve the problem that the VFIO hardware pass-through mode cannot make multiple virtual machines reuse the same NVMe disk, and the way of using files for management still has certain compromise in performance, the present application proposes virtual machine data partition format DVMFS and metadata management partition format VMFR, which effectively solve the problems that pass-through cannot be used by multiple tenants to use the same disk, and the file management way of virtual machine disk cannot fully exert the performance of the NVMe disk.

[0047] Meanwhile, the present application uses IOMMU to prevent virtual machine disk access from data outside the management range, enhances the isolation of data, and uses staggered way to pair the queue pair of the virtual disk and the queue pair of the physical NVMe disk, so that the advantages of NVMe multi-queue can be fully utilized when multiple virtual machines perform I / O. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 is a partition table layout schematic diagram of the virtual machine disk GPT format in the prior art;

[0049] Figure 2 is a layout schematic diagram of DVMFS and VMFR in embodiment 1 of the present application;

[0050] Figure 3 is a principle diagram of Virtio and VFIO in embodiment 1 of the present application;

[0051] Figure 4 is a principle diagram of modifying Hypervisor to add DNVMe Driver in embodiment 1 of the present application;

[0052] Figure 5 is a relationship schematic diagram of virtual disk queue pair and physical disk queue pair mapping in embodiment 1 of the present application. DETAILED DESCRIPTION

[0053] The technical solutions of the present application are further described below in combination with the drawings and specific embodiments. Embodiment 1

[0054] The embodiment provides a virtual machine disk partitioning and accelerated access method based on an NVMe disk, which comprises the following steps:

[0055] Step 1. Partitioning a virtual machine disk;

[0056] As shown in the figure, Figure 1 A cloud service provider uses a virtual machine to separate different users, so that the different users do not interfere with each other, and provides the users with operating system level isolation and customizability. A current server cluster uses a disk in the following manner: a blank disk is formatted into a GPT partition table format, and the GPT partition table format is compatible with an EFI system firmware.

[0057] The GPT design of the prior art can guarantee compatibility and portability. On the basis, the embodiment comprises specifying a virtual machine disk partition format and a virtual machine disk metadata format in a disk partitioning process, and simultaneously specifying a layout of the partition on a GPT partition table.

[0058] The specified overall layout is shown in the figure, Figure 2 The yellow shaded part is the specified layout format, and the part outside the frame is a GPT partition table of a virtual disk in a virtual machine, which guarantees minimum compatibility between a host disk and other disks.

[0059] The first and last two GPT records of the GPT partition table of the host machine record the first N-1 partition formats as DVMFS (Dynamic Virtual Machine File System), and the last partition format as VMFR (Virutal Machine Fallback Record). The DVMFS and the VMFR are both legal partition formats in the GPT partition table. There can be multiple DVMFS and one VMFR in each host disk. Due to the limitation of the GPT partition table, the number of partitions on the disk is at most 128, and therefore the DVMFS is at most 127.

[0060] The DVMFS corresponds to Figure 2 Partition1data sector-N-1data sector in the figure, and each partition Partition is a GPT partition in the DVMFS format.

[0061] The VMFR corresponds to Figure 2The virtual machine - record, VMFR, is connected to the Partition 1 data sector-N-1 data sector through a pointer (LBA on the disk), and the address of the Partition 1 data sector-N-1 data sector is stored in the VMFR. The VMFR contains a pointer to the DVMFS, i.e. an index to find a certain DVMFS is stored in the VMFR.

[0062] The DVMFS is a region for virtual machine virtual disk storage, and includes the following fields:

[0063] 1) FS Header: a header for identifying a file system, containing Magic Number, DVMFS version number, revision number, checksum, etc. information, which is a fixed-length field.

[0064] The checksum is the sum of the hexadecimal values of all fields of the FS Header, and the lower 8 bits are taken as the checksum. When power is restored, the checksum is checked to see whether it is consistent with the recalculated checksum after startup.

[0065] 2) Metadata: saving information of the current partition, including size, virtual machine disk GUID, etc., which is a fixed-length field.

[0066] 3) Next File System Ptr / Prev File System Ptr: used for pointing to the next block of DVMFS space of the same disk used by the virtual machine. The Next File System Ptr and the Prev File System Ptr are used to indicate the LBA addresses of the previous and next continuous spaces on the current partition for fast addressing. The above two fields are fixed-length.

[0067] If the DVMFS is the first physical partition mapped by the virtual machine virtual partition, the serial number of the physical partition is 1, and there is no previous partition at this time, so the Prev File System Ptr is NULL (empty value).

[0068] The header metadata information of the partition is fixed-length, and the data position can be quickly found according to the GPT partition table Entry information.

[0069] Since it is inconvenient to modify the size of disk partition, if the virtual machine needs to expand the disk, the data in the disk partition may need to be migrated to a larger partition on the host side, therefore, the embodiment uses a method that can cross multiple partitions to connect the spaces of multiple DVMFS through LBA pointers, which is an integrated space in the view of the virtual machine, but in fact, it is multiple partitions in different physical locations on the host, and the Hypervisor (virtual machine management program / virtual machine monitor) only needs to establish the mapping of virtual machine LBA to the physical NVMe disk LBA on the host, if there is still free space in the partition after expansion, the size of the current partition can be directly expanded.

[0070] Adjusting the disk capacity of the virtual machine by using algorithm 1, including the operations of expanding and shrinking the disk, both of which are performed in the Hypervisor, and the virtual machine itself cannot perform this operation, including the following steps:

[0071] 1) Call process: call the ADJUSTVMDISK(vmdisk, dst_size, force = false) function for calling operation, the input parameters include the virtual disk object vmdisk, the target size dst_size, and a forced parameter force (the default value is false);

[0072] 2) Parameter check: if the dst_size is less than or equal to 0, a parameter error exception ParamErrorException is thrown;

[0073] 3) Shrinking the disk: if the target size dst_size is less than the current disk size, and force is true, the SHRINKVMDISK(vmdisk, dst_size) function is called for shrinking operation;

[0074] 4) Data loss exception: if force is false, and the target size is less than the current disk size, a data loss exception DataLossException is thrown;

[0075] 5) Expanding the disk: if the target size dst_size is greater than the current disk size, the EXPANDVMDISK(vmdisk, dst_size) function is called for expansion operation.

[0076] In step 3), the disk shrinking process SHRINKVMDISK(vmdisk, dst_size) includes:

[0077] 31) Calculate the size to be reduced: calculate the size to be reduced nr_size, which is the current size minus the target size;

[0078] 32) Loop shrink: when nr_size is greater than 0, perform the following steps:

[0079] Recursively visit the partition pointed by NextPtr of the partition until the last host partition of the virtual disk is found;

[0080] Check if the size of the last host partition of the virtual disk is greater than or equal to nr_size;

[0081] If yes, call the CutDVMFS function to cut, and set nr_size to 0; the CutDVMFS function sets the size of the target partition to

original size - nr_size

[0082] If not, subtract the size of the partition from nr_size, delete the partition, delete the VMFR partition metadata of the partition, and set the NextPtr of the previous DVMFS partition to NULL (empty), and finally update the occupied space bit table on the VMFR, at this time the DVMFS does not exist from the beginning.

[0083] In step 5), the EXPANDVMDISK(vmdisk, dest_size) expansion disk process includes:

[0084] 51) Calculate the size to be expanded: calculate the size to be expanded nr_size, which is the target size minus the current size;

[0085] 52) Check available space: check the available space of the host, if insufficient, throw a NoEnoughSpaceException indicating that the available space of the host is insufficient to expand the partition, and the operation ends;

[0086] 53) Loop expansion: when nr_size is greater than 0, perform the following steps:

[0087] Find the last host partition of the virtual disk;

[0088] Get the available space outside the last host partition of the virtual disk by finding the unused space from the occupied space bit table of the VMFR;

[0089] If the available space is greater than or equal to nr_size, extend the partition and set nr_size to 0, and update the occupied space bit table in the VMFR;

[0090] If the available space is insufficient, recursively find the free space block and add a new partition as needed, update nr_size to nr_size - current free space size, and update the occupied space bit table in the VMFR.

[0091] The VMFR is also used to save the correspondence between the DVMFS and the virtual machine, and is backed up with the metadata information such as the Header in the front of the DVMFS. At the same time, the VMFR partition is located at the end of the LBA position of the disk, close to the Secondary GPT Record. The VMFR uses a fixed-length area to store the metadata of each DVMFS, and sets an upper limit of 127 for the DVMFS accommodated by the VMFR, so the field length of the VMFS is fixed at 127 * the field length of each DVMFS partition metadata. The VMFR includes the header, Metadata, PrevPtr, and NextPtr of each DVMFS partition, and uses the occupied space bit table to save the LBA allocation of the current disk, for quick partition space size adjustment.

[0092] Each modification of the target DVMFS partition triggers a synchronous modification of the VMFR, and because of the 2 backups, a transactional commit is formed. Each modification of the DVMFS partition is considered invalid operation if it is not synchronized to the VMFR, and a repair operation is required.

[0093] The Consistency bit in the VMFR for each DVMFS is used to determine whether it is correctly written. If the Consistency bit is set to 1, it means that the metadata of the partition is modified and not correctly written, so the modification of the DVMFS or the modification of the VMFR needs to be selected according to the situation of the Consistency bit, and the specific method is as follows:

[0094] Condition 1: Check whether the existing checksum is the same as the checksum recalculated from the current partition FS Header;

[0095] Condition 2: Whether the Consistency bit in the VMFR is 0;

[0096] If conditions 1 and 2 are not met, accept the metadata of the VMFR; if condition 1 is met and condition 2 is not met, accept the metadata of the DVMFS; if condition 1 is not met and condition 2 is met, accept the metadata of the VMFR; if conditions 1 and 2 are met, there is no data inconsistency behavior.

[0097] Step two: NVMe disk pass-through in Hypervisor;

[0098] The NVMe disk pass-through access process in the prior art includes Virtio-blk, VFIO, etc., wherein Virtio needs to pass through the file system of the host end, and the performance is poor. Virtio needs to pass through the file system of the host end, so the call stack is longer than that of the pass-through disk, and the time spent on the functions of the call stack for each I / O is more, so the performance is relatively poor compared with the direct access of the host and the pass-through access of the virtual machine. However, the use of files to organize and isolate disk data between different tenants has good flexibility, and VFIO completely passes the entire disk to a virtual machine, which can minimize the performance loss of the virtual machine compared with the host. However, since too many hardware needs to be passed through, the flexibility is limited. The difference between the two is as shown in Figure 3 .

[0099] As shown in Figure 4 , NVMe disk pass-through in Hypervisor, namely DNVMe, includes the following steps:

[0100] 21) An intermediate layer DNVMe Driver is added between the Hypervisor and the VFIO-pci. The intermediate layer uses IOMMU for device address isolation like VFIO. The LBA mapping block of each virtual machine disk is obtained according to the physical address (LBA) of the first DVMFS to which the virtual machine bound virtual disk belongs, and the LBA mapping block of each virtual machine disk is registered by IOMMU. Since the size of the DVMFS partition is variable, and the different DVMFS data areas of the same virtual disk are not continuous, the start LBA address and the end LBA address of each DVMFS need to be registered one by one, and the start LBA address and the end LBA address of each DVMFS are mapped to a continuous virtual address space. From the perspective of the virtual machine, the hard disk space is continuous;

[0101] 22) Then the physical address is obtained directly through the IOMMU for the disk access of the virtual machine, and each NVMe disk (the NVMe disk refers to the virtual concept of the NVMe disk, including the physical NVMe disk and the virtual NVMe disk, that is, the software-implemented NVMe disk) is provided with an Admin QPair and a plurality of I / O QPairs for sending and transmitting management instructions and data operation instructions. Since each NVMe disk device needs an NVMe admin qpair to accept management information, in the Hypervisor, the NVMe disk device of each virtual machine uses the host software to simulate an NVMe admin qpair, and simultaneously determines whether the admin command has a corresponding permission to execute according to the metadata information of the virtual machine disk.

[0102] The embodiment uses one of the commonly used Hypervisors, QEMU, to develop DNVMe, and writes a virtual NVMe disk for the virtual machine. The virtual NVMe device has the same number of Admin qpairs (1) and I / O qpairs as the physical NVMe disk, wherein the admin qpair is taken over by the DNVMe Driver, and the I / O qpairs are transmitted to the VFIO-pci drive.

[0103] The admin qpair in the virtual NVMe disk is used to process the operations of the disk from the virtual machine. Since the physical disk cannot be directly managed by the virtual machine, the operation management instructions of the disk of the virtual machine are mainly mapped to the operations of the metadata / data of the DVMFS and the metadata VMFR partition.

[0104] In addition, since the execution of the admin instruction in the virtual NVMe disk is not disturbed between different virtual machines, only the correct order of the admin instruction in the virtual machine needs to be ensured. In order to ensure the fairness of command processing, a round-robin algorithm is used in the mapping of the multi-admin sq to the physical device sq.

[0105] The I / O qpairs process all normal NVMe read and write instructions, and convert the virtual device address into the actual physical disk address. The IOMMU is used to first convert the virtual address into the physical address of the physical NVMe disk, and then put the command located in the qpair into the sq queue of the physical NVMe disk. The mapping relationship between the virtual address and the physical address has been submitted to the IOMMU according to the first and last addresses of the data partition of the DVMFS in the initialization.

[0106] Due to the finiteness of the IOMMU mapping table, a LBA address conversion table mechanism similar to a page table is set, and the LBA address conversion table is saved in the memory for fast lookup. If the LBA address mapping information cannot be found in the IOMMU, the current mapping information is obtained from the LBA address conversion table, which is constructed using the first and last addresses of the data partition of each DVMFS when the host physical NVMe disk is initialized.

[0107] In order to fully utilize the qpair of the physical NVMe disk, according to the current number of virtual machines and the queue width supported by the physical NVMe disk, the tasks are evenly distributed to the physical NVMe disk qpair.

[0108] Suppose the number of virtual machines is N, and the maximum number of I / O qpairs supported by the physical NVMe disk is M, then the maximum number of queues of the virtual NVMe disk in the virtual machine is M, and n.m represents the mth queue of the nth virtual machine.

[0109] In order to maintain the order of instructions within the same qpair (NVMe does not guarantee the order of instructions between qpairs), the minimum unit of mapping between virtual qpair and physical qpair is queue instead of single command, and the queue of the virtual NVMe disk is mapped to the queue of the physical NVMe disk in a one-to-one manner. In order to avoid the competition of qpair0, staggered mode is used, as shown in the figure, according to the initialization order of the virtual machine, the starting virtual queue qpair0 is mapped to different qpairs on the physical NVMe disk, and then the next queue is mapped in sequence. Figure 5

[0110] In addition, in order to ensure the reusability of the operation, the operations in the same qpair set are preferentially allocated to the operations in the same qpair set in order to increase the data reuse opportunity when the device traverses the qpair to execute the command.

[0111] Since the I / O qpair is shared by all virtual machines, each virtual machine can access the address at will, in order to ensure that each virtual machine disk can only access the DVMFS space belonging to itself, the linkage mechanism of the LBA mapping list maintained by the software in the Hypervisor and the IOMMU mapping is used, and the steps are as follows:

[0112] ​When the I / O request reaches the IOMMU, the IOMMU first converts the virtual address of the I / O request into a physical address while checking the source virtual machine of the I / O request and the virtual machine occupying the virtual disk to which the corresponding physical address belongs, using the unique number of the virtual machine for the checking, if the unique number of the source virtual machine of the request is the same as the unique number of the virtual machine occupying the virtual disk to which the corresponding physical address belongs, the I / O request proceeds normally, if the virtual machines are found to be mismatched, an error is returned and the execution of the instruction is terminated. Since the I / O request is performed by the IOMMU hardware, the impact on performance is almost the same as that of the normal virtual machine using the IOMMU, and the worst performance is 1.13% different from that of the plain case. Embodiment 2

[0113] In step two, in the process of NVMe disk transparent transmission, on the basis of the existing Virtio-blk mechanism, the performance is improved by optimizing the cache strategy and improving the data transmission efficiency, an efficient cache layer is set, frequently accessed data is saved in the memory, and the dependence on the host file system is reduced, so that the read and write speed can be improved while the flexibility is maintained. Embodiment 3

[0114] In step one, the Hypervisor can be VMware, Virtualbox, KVM, etc.

[0115] The embodiments of the application are described in detail above with reference to the accompanying drawings, but the above description is only the preferred embodiments of the application, and it should be pointed out that for ordinary skilled in the art, without departing from the technical principles of the application, a number of improvements and modifications can be made, and these improvements and modifications should be regarded as the protection scope of the application.

Claims

1. A method for virtual machine disk partitioning and accelerated access based on NVMe disks, characterized in that, Comprising the following steps: Step one. Partition the virtual machine disk, specify the virtual machine disk partition format and virtual machine disk metadata format, specify the layout of the partition on the GPT partition table; record the first N-1 partition format as DVMFS and the last partition format as VMFR in the first and last two GPT records of the GPT partition table of the host; the VMFR contains a pointer to the DVMFS; Step two: NVMe disk transparent transmission in Hypervisor, including: 21) An intermediate layer DNVMe Driver is added between Hypervisor and VFIO-pci, the intermediate layer uses IOMMU to isolate device addresses, obtains the LBA mapping block of each virtual machine disk according to the physical address LBA of the first DVMFS to which the virtual machine disk belongs, and registers each virtual machine disk LBA mapping block with IOMMU, registers the start LBA address and end LBA address of each DVMFS one by one, and maps the start LBA address and end LBA address of each DVMFS to a continuous virtual address space; 22) Obtain the physical address directly through IOMMU to access the virtual machine disk, NVMe disk, each NVMe disk is provided with an Admin QPair and multiple I / O QPairs for sending and transmitting management instructions and data operation instructions, the NVMe disk includes a physical NVMe disk and a virtual NVMe disk; each virtual machine NVMe disk device in Hypervisor uses the host software to simulate an NVMe admin qpair, and determines whether the admin command has corresponding permission to execute according to the virtual machine disk metadata information; In step 22), the virtual NVMe disk has at most the same number of Admin qpair and I / O qpairs as the physical NVMe disk, the admin qpair is taken over by DNVMe Driver, and the I / O qpairs are transmitted to VFIO-pci driver; The I / O qpairs process all normal NVMe read and write instructions, convert virtual device addresses into actual physical disk addresses; use IOMMU to first convert virtual addresses into physical addresses of physical NVMe disk, and then put the commands located in qpair into the sq queue of physical NVMe disk; the mapping relationship between virtual address and physical address is submitted to IOMMU according to the data partition start and end address of DVMFS during initialization.

2. The method of claim 1, wherein, In step one, DVMFS is the area for virtual machine virtual disk storage, including the following fields: 1) FS Header: header for identifying file system; 2) Metadata: save the information of the current partition; 3) Next File System Ptr and Prev File System Ptr are used to indicate the LBA address of the next and previous continuous space on the current partition for fast addressing.

3. The method of claim 1, wherein, In step one, in the Hypervisor, the disk capacity of the virtual machine is adjusted by using Algorithm 1, including operations of expanding and shrinking the disk, including the following steps: 1) Call the operation by using the ADJUSTVM DISK (vmdisk, dst_size, force = false) function, input parameters including the virtual disk object vmdisk, the target size dst_size, and a forced parameter force; 2) If dst_size is less than or equal to 0, throw a parameter error exception ParamErrorException; 3) If the target size dst_size is less than the current disk size, and force is true, call the SHRINKVM DISK (vmdisk, dst_size) function to perform the shrinking operation; 4) If force is false, and the target size is less than the current disk size, throw a data loss exception DataLossException; 5) If the target size dst_size is greater than the current disk size, call the EXPANDVM DISK (vmdisk, dst_size) function to perform the expansion operation.

4. The method of claim 3, wherein, In step 3), the disk shrinking process SHRINKVM DISK (vmdisk, dst_size) includes: 31) Calculate the size nr_size that needs to be reduced, which is the current size minus the target size; 32) When nr_size is greater than 0, perform the following steps: Recursively access the partition pointed to by the NextPtr of the partition until the last host partition of the virtual disk is found; Check if the size of the last host partition of the virtual disk is greater than or equal to nr_size; If yes, call the CutDVMFS function to cut, and set nr_size to 0; the CutDVMFS function sets the size of the target partition to 【original size - nr_size】, modifies the Metadata part of DVMFS to set NextPtr to NULL, updates the size field of the VMFR part, and finally updates the occupied space bit table in the VMFR using the actual occupied LBA position of the DVMFS partition; If not, subtract the size of the partition from nr_size, delete the partition, delete the VMFR partition metadata of the partition, set the NextPtr of the previous DVMFS partition to NULL, and finally update the occupied space bit table in the VMFR.

5. The method of claim 3, wherein, In step 5), the disk expansion process EXPANDVM DISK (vmdisk, dest_size) includes: 51) Calculate the size nr_size that needs to be expanded, which is the target size minus the current size; 52) Check the available space of the host, if insufficient, throw NoEnoughSpaceException, and the operation ends; 53) When nr_size is greater than 0, perform the following steps: Find the last host partition of the virtual disk; Get the available space outside the last host partition of the virtual disk, by finding the unused space from the occupied space bit table of the VMFR; If the available space is greater than or equal to nr_size, extend the partition and set nr_size to 0, and update the occupied space bit table in the VMFR; If the available space is insufficient, recursively find the free space block and add a new partition as needed, update nr_size to nr_size-current free space size, and update the occupied space bit table in the VMFR.

6. The method of claim 1, wherein, In step one, use the Consistency bit in the VMFR for each DVMFS to determine whether it is correctly written, if the Consistency bit is set to 1, it means that the metadata of the partition is modified and not correctly written, according to the situation of the Consistency bit, choose to accept the modification of the DVMFS or the modification of the VMFR, the method is: Condition 1: Check if the existing checksum is the same as the recalculated checksum of the current partition FS Header; Condition 2: Whether the Consistency bit in the VMFR is 0; If conditions 1 and 2 are not met, accept the metadata of the VMFR; if condition 1 is met and condition 2 is not met, accept the metadata of the DVMFS; if condition 1 is not met and condition 2 is met, accept the metadata of the VMFR; if condition 1 is met and condition 2 is met, there is no data inconsistency behavior.

7. The method of claim 1, wherein, In step 22, set up the LBA address conversion table mechanism and save the LBA address conversion table in the memory for quick lookup, if the LBA address mapping information cannot be found in the IOMMU, get the current mapping information from the LBA address conversion table, which is constructed using the data partition start and end addresses of each DVMFS when the host physical NVMe disk is initialized.

8. The method of claim 1, wherein, The minimum unit of mapping between virtual qpair and physical qpair is queue, the queue of virtual NVMe disk is mapped to the queue of physical NVMe disk in a one-to-one manner, according to the initialization order of virtual machine, the starting virtual queue qpair0 is mapped to different qpair on the physical NVMe disk, and then the following queues are mapped in order.

9. The method of claim 1, wherein, The LBA mapping list and the IOMMU mapping are linked together, the steps are: When an I / O request reaches the IOMMU, the IOMMU first converts the virtual address of the I / O request into a physical address, and checks the source virtual machine of the I / O request with the virtual machine occupying the virtual disk to which the corresponding physical address belongs, using the unique number of the virtual machine for the check. If the unique number of the source virtual machine of the request is the same as the unique number of the virtual machine occupying the virtual disk to which the corresponding physical address belongs, the I / O request proceeds normally; if the virtual machines are found to be mismatched, an error is returned, and the execution of the instruction is terminated.

Images