A method and system for reducing disk space occupied by a virtualization image file

Abstract

The application discloses a method for reducing the disk space occupied by a virtualization image file, which comprises the following steps: converting a virtual machine disk to be processed into a qcow2 format virtual machine disk; performing a hole elimination on the qcow2 format virtual machine disk until the disk space is fully occupied; judging whether the virtual machine disk file can be written again, if yes, continuing to perform the hole elimination, otherwise, performing a forensic erasure on the virtual machine disk to release the disk space; performing an operation on the virtual machine after the disk space is released, and exporting the virtualization image file; performing a directional conversion compression on the exported virtualization image file to obtain a reduced qcow2 format virtualization image file, thereby saving the disk space. The application can release the occupied disk space, reduce the size of the disk space occupied by the virtual machine in the system, reduce the disk pressure caused by a large number of image files, save the disk space, and reduce the production cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of cloud computer and virtualization technology, and particularly relates to a method and system for reducing the disk space occupied by a virtualization image file. BACKGROUND

[0002] A cloud system of cloud computing is actually a large distributed system, and virtualization is one of the key technologies indispensable for constructing a cloud infrastructure of the cloud system. Virtualization virtually creates multiple virtual platforms on a physical platform, and each virtual platform is regarded as an independent terminal and is added to the distributed system of the cloud. Compared with directly using a physical platform, virtualization has great advantages in efficient use of resources, dynamic deployment and high reliability. By using virtualization technology, an enterprise can construct a brand-new information infrastructure without discarding the existing infrastructure, thereby being able to more fully utilize the original computer resources.

[0003] A virtualization image file is necessary in the deployment, generation and transmission of a virtual machine. Different virtualization image files exist in different formats and types according to different virtualization platforms. For example, the format of an image file under a VMware virtualization platform is vmdk, the format of an image file under a VirtualBox virtualization platform is vdi, and the formats of image files under a KVM virtualization platform are qcow2, raw or img. At present, the common method of generating a virtualization image file using an open-source cloud computing management platform Openstack has the problems of large virtualization image files and large disk space occupied by dirty data existing in the virtualization image files, which causes waste of physical hardware resources. In addition, the transmission and file copying of these large file format virtual machine images are very slow, which affects the efficiency of data transmission. Furthermore, the disk occupancy of a virtual machine image file is extremely high after a period of disk expansion, and the disk space occupied by the virtual machine image file will not be reduced after the contents on the disk are deleted. These occupied spaces cannot be utilized, which causes great waste. SUMMARY

[0004] The present application aims at overcoming the deficiencies in the prior art, and provides a method, device and system for reducing the disk space occupied by a virtualization image file, which solves the technical problem of slow file copying and transmission and resource waste caused by large virtual machine disk occupancy files in the conventional image making process of the virtualization image file in the prior art.

[0005] To solve the above technical problems, the present application is implemented by using the following technical scheme:

[0006] The first aspect, the present application provides a kind of method for reducing virtualization mirror file occupies disk space, the method comprises: the virtual machine disk to be processed is converted into qcow2 format virtual machine disk;

[0007] The qcow2 format virtual machine disk is carried out hole filling, until disk space is occupied full;

[0008] Whether the virtual machine disk file can be written again, if yes, continue to carry out hole filling, otherwise, the virtual machine disk is carried out evidence erasing, and disk space is released;

[0009] The virtual machine after the disk space is released is carried out virtualization mirror file operation, and the virtualization mirror file is exported;The exported virtualization mirror file is carried out directional conversion compression, and the reduced qcow2 format virtualization mirror file is obtained, and disk space is saved.

[0010] In combination with the first aspect, preferably, the qcow2 format virtual machine disk is carried out hole filling, comprising the following steps:

[0011] All spaces of the virtual machine disk are zero filled, and the zero filling is carried out by infinite null character writing in the root directory of virtual machine, to form one or more zero filling files, until the remaining disk space is occupied full.

[0012] In combination with the first aspect, preferably, all spaces of the virtual machine disk include the disk space free of disk itself and the disk space occupied by corresponding virtual machine mirror file after deleting the content on the disk.

[0013] In combination with the first aspect, preferably, the virtual machine disk is carried out evidence erasing, comprising the following steps:

[0014] The zero filling file is recorded with identification bit, and the zero filling file is deleted by comparing the identification bit, to release the occupied disk space.

[0015] In combination with the first aspect, preferably, the directional conversion compression of the exported virtualization mirror file comprises the following steps:

[0016] The exported virtualization mirror file is marked as uninitialization state;

[0017] The virtualization mirror file marked as uninitialization state is divided into multiple different virtual disk blocks according to system standard;The sectors of multiple virtual disk blocks are respectively rewritten, to obtain multiple corresponding non-compressed sectors;

[0018] Multiple non-compressed sectors are respectively generated as multiple incremental files after sparse mapping;

[0019] The plurality of the incremental files are merged to obtain a compressed virtualization image file in qcow2 format.

[0020] In combination with the first aspect, preferably, the virtual machine is kept in a powered-on state when the virtual machine disk is hole filling.

[0021] In combination with the first aspect, preferably, the virtual machine is kept in a powered-off state when the virtual machine after the disk space is released is operated on by the virtualization image file.

[0022] The second aspect of the present application provides a system for reducing the disk space occupied by a virtualization image file, the system comprising:

[0023] A conversion module is configured to convert a virtual machine disk to be processed into a virtual machine disk in qcow2 format.

[0024] A hole filling module is configured to perform hole filling on the virtual machine disk in qcow2 format until the disk space is fully occupied; a forensic erasing module is configured to determine whether the virtual machine disk file can be written again, if yes, continue to perform hole filling, otherwise, perform forensic erasing on the virtual machine disk to release disk space.

[0025] An export module is configured to perform virtualization image file operation on the virtual machine after the disk space is released, and export the virtualization image file.

[0026] A compression module is configured to perform directional conversion compression on the exported virtualization image file to obtain a virtualization image file in qcow2 format with reduced size, thereby saving disk space.

[0027] The third aspect of the present application provides a device for reducing the disk space occupied by a virtualization image file, comprising a processor and a storage medium.

[0028] The storage medium is configured to store instructions.

[0029] The processor is configured to operate according to the instructions to perform the steps of the method for reducing the disk space occupied by a virtualization image file according to any one of the first aspect.

[0030] The fourth aspect of the present application provides a computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of the method for reducing the disk space occupied by a virtualization image file according to any one of the first aspect.

[0031] Compared with the prior art, the present application has the following beneficial effects:

[0032] The application recovers the real available space in the virtual machine by hollowing out and rounding up the evidence of the virtual machine disk, thereby releasing the unused space on the virtualization image file; and the exported virtualization image file is directionally converted and compressed to generate the final converted image file, which greatly reduces the occupancy of the disk space, reduces the disk pressure caused by a large number of image files, and achieves the effects of reducing cost, improving production efficiency, and avoiding resource waste. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 A flowchart of the method for reducing the disk space occupied by the virtualization image file provided by the embodiment of the application is shown.

[0034] Figure 2 A structure principle block diagram of the system for reducing the disk space occupied by the virtualization image file provided by the embodiment of the application is shown. DETAILED DESCRIPTION

[0035] The technical solutions of the application will be described in detail below with reference to the drawings and specific embodiments. It should be understood that the specific features in the embodiments and the embodiments are detailed descriptions of the technical solutions of the application, rather than limitations of the technical solutions of the application. In the case of no conflict, the technical features in the embodiments and the embodiments can be combined with each other.

[0036] In this paper, the term "and / or" is only a description of the association relationship between the associated objects, which means that there can be three kinds of relationships, for example, A and / or B, which can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. In addition, the character " / " in this paper generally represents a "or" relationship between the associated objects before and after it.

[0037] Embodiment one:

[0038] As shown in Figure 1 The embodiment of the application introduces a method for reducing the disk space occupied by the virtualization image file, which is realized based on the open source cloud computing management Openstack platform, and specifically includes the following steps:

[0039] Step 1: converting the virtual machine disk to be processed into a qcow2 format virtual machine disk;

[0040] Step 2: hollowing out the qcow2 format virtual machine disk until the disk space is full;

[0041] It should be noted that in the use of the virtual machine, the files in the virtual machine are subjected to a large number of adding, deleting, modifying and other operations, so that the size of the virtual machine disk becomes larger, even if the files in the virtual machine are deleted, the space allocated to the virtual disk will not be recycled; as an embodiment of the present application, the space not used by the virtual machine can be manually recycled, and the disk space not recycled after the virtual machine disk is used is marked; when recycling, the embodiment of the present application fills the remaining space of the virtual machine disk with zeros in the virtual machine startup state, that is, writes unlimited null characters to the root directory of the virtual machine to form one or more zero padding files until the remaining disk space is full; wherein the remaining space of the virtual machine disk includes the idle disk space of the disk itself and the disk space occupied by the virtual machine image file after deleting the content on the disk. When operating, the dd command (copy a file with a specified size block and perform a specified conversion while copying) is used, the file name is input as the parameter, the default is standard input, that is, the source file is specified here; the output file name is input as the parameter, the default is standard input, that is, the target file is specified here, and the target is filled with zeros until the space is used up; at this time, the virtual machine disk space has been completely occupied, and more operations or increase of disk files are prohibited.

[0042] Step 3: judging whether the virtual machine disk file can be written again, if yes, continue to fill the hole, otherwise, perform forensic wiping on the virtual machine disk to release the disk space;

[0043] As an embodiment of the present application, after judging that the virtual machine disk file cannot be written again, the zero padding file is marked with an identification bit, and the zero padding file formed in step 2 is deleted by continuously comparing the identification bit, so that the occupied disk space is released; when operating, the rm command (a command for deleting a file or directory) can be used to delete the disk identification bit file.

[0044] Step 4: performing virtualization image file operation on the virtual machine after releasing the disk space, and exporting the virtualization image file;

[0045] When performing this step, first, the virtual machine is closed, and the virtual machine is kept in the shutdown state, the legality of the virtual machine to be exported and whether it meets the system resource preset threshold requirement are judged, then the virtual machine disk file path is specified, and the disk file is copied combined with the loaded virtual machine script information XML file, and then the virtual machine disk file is exported to the specified path after creating the domain dumpxml configuration file.

[0046] Step 5: directing the converted and compressed virtualization image file to obtain a virtualization image file in qcow2 format, saving disk space.

[0047] The step 5 provided by the embodiment of the present application comprises the following steps:

[0048] Step 5.1: Marking the exported virtualization image file as an uninitialization state;

[0049] Step 5.2: Dividing the virtualization image file marked as the uninitialization state into a plurality of different virtual disk blocks according to the system standard;

[0050] Step 5.3: Rewriting the sectors of the plurality of virtual disk blocks respectively to obtain a plurality of corresponding non-compressed sectors;

[0051] Step 5.4: Generating a plurality of incremental files by performing sparse mapping on the plurality of non-compressed sectors respectively;

[0052] Step 5.5: Merging the plurality of incremental files to obtain a compressed virtualization image file in the qcow2 format.

[0053] As an embodiment of the present application, in the operation of step 5, first, the qemu-img command is used to perform the directed conversion compression of the image file qcow2 (qemu-img is a disk management tool of QEMU, which is used to create, convert and modify disk image files offline), since the qemu-img command does not copy the 0 data block in the virtual disk, the purpose of compressing the disk is achieved; second, the convert method is used to convert the original format image file into the target format image file according to the options, at this time, the default image format is the raw format (bare disk format, the default virtual disk format), which needs to be converted into the image file in the qcow2 format; third, the output image file is compressed, here only the image in the qcow2 or qcow format supports image compression, and this compression operation type is read-only, if the compressed sector is rewritten, it will be rewritten into uncompressed data.

[0054] The effectiveness and working performance of the method for reducing the disk space occupied by the virtualization image file provided in the embodiment of the present application are analyzed through experiments. Various related basic services are configured on an open-source cloud computing Openstack platform: identity authentication service, image service, network service, computing service and control plane service, two virtual machines with completely consistent operating systems are built based on the above services, the operating system is taken as an example of CentOS7 or windows, here we command to name the two virtual hosts as VMA and VMB, during the running of the virtual hosts, it is ensured that the executed actions and operation contents are the same, so that the disk consumptions caused by them are basically consistent, VMA and VMB are configured with the same instance type resource of the disk. The virtual machine is started by using the method commonly used in the prior art for VMA, after a period of use, the virtual machine VMA is first executed to shut down, and then the image file of the virtual machine is exported, the exported way is that the image file of the virtual machine is exported in the qcow2 format through the export API interface of the encapsulated Openstack image service; during the operation, the exported command glance download can be directly used; after the virtual machine image file is exported, it is found by the operating system native system tool df that the occupied space size of the image file is 2.7G. For VMB, the steps of the method for reducing the disk space occupied by the virtualization image file provided in the embodiment of the present application are performed, finally we see that the size of the virtualization disk image file occupied by VMB is 700MB, compared with the virtualization disk image file of VMA, the capacity of the virtualization image disk file occupied is greatly reduced, and the qcow2 virtualization image file of VMB is the virtualization image file after the virtualization disk space is reduced, the virtualization image file consumes less network bandwidth resources in network transmission and file transmission, and is more convenient and efficient to use.

[0055] It is further illustrated that the embodiment of the present application provides test data of the original disk space size of the virtual machine of a plurality of typical operating system versions under different specifications, the disk space size occupied by the virtualization image file generated by the method commonly used in the prior art, and the disk space size occupied by the virtualization image file generated by the method provided in the present application, and the specific experimental data is shown in the following table:

[0056]

[0057] From the experimental data, it can be seen that the method for reducing the disk space occupied by the virtualization image file provided in the embodiment of the present application can effectively reduce the disk space size occupied by the virtual machine in the system, reduce the disk pressure caused by a large number of image files, and greatly save the disk space.

[0058] In summary, the method provided by the embodiment of the present application can effectively solve the technical problem of slow file copying and transmission and resource waste caused by too large virtual machine disk file in the conventional image making process of the virtualization image file in the prior art; the method provided by the embodiment of the present application recycles the real available space in the virtual machine, releases the unused space on the virtualization image file, greatly reduces the disk space occupancy, achieves the effect of reducing cost, improving production efficiency and avoiding resource waste, and has a wide application prospect.

[0059] Embodiment two:

[0060] As shown in Figure 2 The embodiment of the present application provides a system for reducing the disk space occupied by a virtualization image file, which can be used to implement the method described in embodiment one, and the system comprises:

[0061] A conversion module is configured to convert a to-be-processed virtual machine disk into a virtual machine disk in qcow2 format.

[0062] A hole filling module is configured to fill holes in the virtual machine disk in qcow2 format until the disk space is fully occupied.

[0063] A forensic erasing module is configured to determine whether the virtual machine disk file can be written again, and if so, continue to fill holes, otherwise, perform forensic erasing on the virtual machine disk to release disk space.

[0064] An export module is configured to perform virtualization image file operation on the virtual machine after the disk space is released, and export the virtualization image file.

[0065] A compression module is configured to perform directional conversion compression on the exported virtualization image file to obtain a reduced virtualization image file in qcow2 format, thereby saving disk space.

[0066] The system for reducing the disk space occupied by a virtualization image file provided by the embodiment of the present application is based on the same technical concept as the method for reducing the disk space occupied by a virtualization image file provided by embodiment one, and can produce the beneficial effects described in embodiment one. The content not described in detail in the present embodiment can be referred to embodiment one.

[0067] Embodiment three:

[0068] The embodiment of the present application provides a device for reducing the disk space occupied by a virtualization image file, comprising a processor and a storage medium; the storage medium is configured to store instructions;

[0069] The processor is configured to operate according to the instructions to perform the steps of any method according to embodiment one.

[0070] The embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize steps of any method in the embodiment one.

[0071] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can be in the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware. In addition, the present application can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage and the like) containing computer-usable program codes.

[0072] The present application is described with reference to flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate a device implemented in the flowcharts and / or block diagrams. Figure 1 The function of one flow or multiple flows and / or blocks Figure 1 The function of one block or multiple blocks.

[0073] These computer program instructions can also be stored in a computer readable memory capable of guiding a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer readable memory produce a product including instruction devices, which realize the flowcharts and / or block diagrams. Figure 1 The function of one flow or multiple flows and / or blocks Figure 1 The function of one block or multiple blocks.

[0074] These computer program instructions can also be loaded into a computer or other programmable data processing device, so that a series of operation steps are performed on the computer or other programmable device to produce a computer implemented process, so that the instructions executed on the computer or other programmable device provide steps for realizing the flowcharts and / or block diagrams. Figure 1 The function of one flow or multiple flows and / or blocks Figure 1 The function of one block or multiple blocks.

[0075] The above merely describes the preferred embodiments of the present application, and it should be pointed out that, for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims

1. A method of reducing the disk space occupied by a virtualized image file, the method comprising: The method comprises: Converting the virtual machine disk to be processed into a virtual machine disk in qcow2 format; Performing hole filling on the virtual machine disk in qcow2 format until the disk space is occupied; Judging whether the virtual machine disk can be written again, if yes, continuing to perform hole filling, otherwise, performing forensic wiping on the virtual machine disk to release the disk space, comprising: when the virtual machine disk can be written again, zero filling all the remaining space of the virtual machine disk to form one or more zero filling files; After judging that the virtual machine disk cannot be written again, recording the identification bits of the zero filling files, and performing forensic wiping on the zero filling files formed by comparing the identification bits to release the occupied disk space; Performing virtualization image file operation on the virtual machine after releasing the disk space, and exporting the virtualization image file; Performing directional conversion compression on the exported virtualization image file to obtain a virtualization image file in qcow2 format with reduced size, saving disk space; The directional conversion compression on the exported virtualization image file comprises the following steps: Marking the exported virtualization image file as an uninitialized state; Dividing the virtualization image file marked as the uninitialized state into a plurality of different virtual disk blocks according to system standards; Rewriting the sectors of the plurality of virtual disk blocks to obtain a plurality of corresponding non-compressed sectors; Generating a plurality of incremental files by performing sparse mapping on the plurality of non-compressed sectors respectively; Merging the plurality of incremental files to obtain a virtualization image file in qcow2 format with reduced size.

2. The method of claim 1, wherein, Performing hole filling on the virtual machine disk in qcow2 format comprises the following steps: Writing infinite null characters to the root directory of the virtual machine to form one or more zero filling files until the remaining disk space is occupied.

3. The method of claim 2, wherein, The remaining space of the virtual machine disk includes the idle disk space of the disk itself and the disk space still occupied by the corresponding virtual machine disk after deleting the content on the disk.

4. The method of claim 1, wherein, When performing hole filling on the virtual machine disk, the virtual machine remains in a powered-on state.

5. The method of claim 1, wherein, When performing virtualization image file operation on the virtual machine after releasing the disk space, the virtual machine remains in a powered-off state.

6. A system for reducing the disk space occupied by a virtualization image file, applied to the method for reducing the disk space occupied by a virtualization image file according to any one of claims 1 to 5, characterized in that, The system comprises: A conversion module for converting the virtual machine disk to be processed into a virtual machine disk in qcow2 format; A hole filling module for performing hole filling on the virtual machine disk in qcow2 format until the disk space is occupied; A forensic wiping module for judging whether the virtual machine disk can be written again, if yes, continuing to perform hole filling, otherwise, performing forensic wiping on the virtual machine disk to release the disk space; An export module for performing virtualization image file operation on the virtual machine after releasing the disk space, and exporting the virtualization image file; A compression module for performing directional conversion compression on the exported virtualization image file to obtain a virtualization image file in qcow2 format with reduced size, saving disk space.

7. An apparatus for reducing disk space occupied by a virtualized image file, the apparatus comprising: A processor and a storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps of a method of reducing disk space usage of a virtualized image file according to any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, which when executed by a processor, implements the steps of a method of reducing disk space usage of a virtualized image file according to any one of claims 1 to 5.

Images