Intelligently defragment file systems inside VMs for fast recovery in the cloud and efficient deduplication for the cloud

Abstract

An exemplary method includes: partitioning a corresponding disk of each of a plurality of virtual machines (VMs) to create a corresponding plurality of blocks associated with each of the VMs; creating a cluster including one or more of the VMs based on the partitioning process; creating a VM template whose data and disk structure match the corresponding data and disk structure of each of the VMs in the cluster; and in response to a file operation involving a first VM disk among the VM disks, defragmenting the first VM disk so that the disk structure of the first VM disk is the same as the disk structure of the VM template.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to data protection. More specifically, at least some embodiments of the present invention relate to systems, hardware, software, computer-readable media, and methods for providing cloud DR protection for VMs having file systems that are logically similar or identical but physically stored on disk in different ways. Background Art

[0002] Businesses may generate a large amount of important data, which is usually kept in some type of data protection environment. Such data protection environment helps ensure that the data is recoverable in the event of a disaster that destroys the data or prevents access to the data. Therefore, disaster recovery (DR) is a critical process for organizations.

[0003] Generally speaking, DR for cloud systems allows users to failover or perform DR testing of virtual machines from a local environment to a public cloud environment. To improve efficiency and save costs, data is stored in relatively cheap object storage in the public cloud and is restored to block storage as needed only when a failover or DR test is initiated. Due to the large amount of metadata required and the expected slowdown during the virtual machine (VM) reconstruction process, the system also does not save data in a deduplicated form, because in this case, the reconstruction requires reading data from a significantly larger number of objects, and the total cost of ownership (TCO) will be higher.

[0004] Disaster recovery planning for VMs presents some special challenges. For example, VMs in a group may have logically nearly identical file systems (FS), but the corresponding file systems may be physically stored on disk in very different ways. Furthermore, some processes may change the location of files on the VM's file system, so that otherwise nearly identical VMs may have substantial differences in their corresponding VMDK binary images. Furthermore, in many cases, files deleted from the VM file system are not zeroed out. Therefore, even if the VM file system is mostly empty, there may still be a lot of garbage on disk. These situations are problematic.

[0005] For example, when replicating a VM to the cloud for DR purposes and then restoring multiple nearly identical VMs, it may be difficult or impossible to use only a single block device as a backup template and then retain only the differences associated with each VM. Similarly, if there is a process that changes the VM file system structure without actually changing the files, that process may create unnecessary large differences between two VM versions that are actually identical. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] In order to describe the manner in which at least some of the advantages and features of the present invention can be obtained, a more specific description of embodiments of the present invention will be presented by reference to specific embodiments of the present invention shown in the accompanying drawings. It should be understood that these drawings depict only typical embodiments of the present invention and are not therefore to be considered as limiting the scope thereof, and embodiments of the present invention will be described and explained with additional specificity and detail through the use of the accompanying drawings.

[0007] Figure 1 Aspects of an exemplary operating environment for some embodiments of the invention are disclosed.

[0008] Figure 2 Aspects of exemplary host configurations are disclosed.

[0009] Figure 2 Aspects of an example method for creating a VM template are disclosed.

[0010] Figure 4 Aspects of a method for defragmenting a VM disk are disclosed.

[0011] Figure 5 Aspects of a method for restoring one or more VMs using a VM template are disclosed. DETAILED DESCRIPTION

[0012] Embodiments of the present invention generally relate to data protection. More specifically, at least some embodiments of the present invention relate to systems, hardware, software, computer-readable media, and methods for providing cloud DR protection for VMs having file systems that are logically similar or identical but physically stored on disk in different ways. At least some embodiments of the present invention are implemented in conjunction with a production site comprising one or more VMs and a cloud storage environment (such as a cloud data center), but the scope of the present invention is not limited to this exemplary environment. In general, embodiments of the present invention include a method that will allow the disk layout of similar VMs to remain as similar as possible after changes, thereby allowing customers to save cloud storage space and keep a small number of templates in the cloud, as well as achieve faster recovery times in cloud DR use cases such as multi-VM failover or DR testing.

[0013] In more detail, an exemplary embodiment of the present invention relates to providing cloud DR protection for VMs that are actually similar and have very small differences. To this end, a similarity algorithm and a distributed defragmentation (or "defrag") algorithm are provided. The similarity algorithm attempts to identify VMs that are similar to each other in terms of corresponding data and file systems, and the distributed defragmentation (or "defrag") algorithm (such as in the form of an agent) uses a predefined consistency logic to arrange data on similar VMs and attempts to place the same data at the same offset of the disk. More specifically, the defragmentation agent is installed on the production site VM client that needs to be protected. The defragmentation agent places the data on the corresponding VM so that the VM disks of the protected machines with almost the same data are also almost the same in terms of their disk layout.

[0014] It is important to note that due to file system properties and data movement, even identical file systems containing the same files and directories can have very different on-disk structures due to different placement of files on disk. Thus, versions of the same virtual machine can have significantly different on-disk binary layouts even if they differ only slightly from each other logically.

[0015] In more detail, the defragmentation agent installed on each production site VM will perform various functions related to empty disk space and data on the disk. For example, the defragmentation agent will operate so that when a file is deleted, the defragmentation agent will zero the data, that is, fill the space left by the deleted file with zeros, and then move these zero areas to the end of the disk.

[0016] Likewise, the defragmentation agent will try to place identical data in the same location on the corresponding VM disks in different but similar VMs. To this end, the similarity algorithm will provide a sample VM for the VM cluster, and the defragmentation agent operates to ensure that the data of the other VMs will be arranged in a similar manner as in the sample VM. For example, if in a new version of a VM, a process simply moves data in the file system to a new location (such as, for example, rewriting a file by using a small change), the defragmentation agent will move the data to the same location as before the file was overwritten.

[0017] Advantageously, then, some embodiments of the present invention may provide various benefits and improvements over the configuration and operation of conventional hardware, software, systems, and methods. To illustrate, one embodiment of the present invention includes a process for establishing and maintaining a shared disk configuration as much as possible between a group of VMs to be protected. In addition, one embodiment of the present invention provides a single VM recovery template, which may be in the form of a full VM image stored in block storage at a cloud site and may be used to restore similar VMs in a group of VMs. Since only a single VM recovery template is stored, a relatively small amount of block storage is employed compared to arrangements using multiple recovery images, thereby reducing costs, since block storage tends to be more expensive than, for example, object storage. One embodiment of the present invention provides a relatively fast DR process because a single VM recovery template is used to restore multiple similar VMs.

[0018] It should be noted that the aforementioned advantages of various embodiments are presented by way of example only, and various other advantages of the exemplary embodiments of the present invention will be apparent from this disclosure. It should be further noted that it is not necessary for any embodiment to achieve or enable any such advantages disclosed herein.

[0019] A. Exemplary Operating Environment and Aspects of Operation

[0020] The following is a discussion of various aspects of exemplary operating environments and associated operations for various embodiments of the present invention. This discussion is not intended to limit the scope of the present invention or the applicability of the embodiments in any way.

[0021] Generally speaking, embodiments of the present invention may be implemented in conjunction with systems, software, and components that implement and / or cause data management operations individually and / or collectively. Such data management operations may include, but are not limited to, data read / write / delete operations, data backup operations, data restore operations, data clone operations, data archiving operations, and disaster recovery operations. Therefore, while the discussion herein may be directed in some aspects to a discussion of data protection environments and operations, the scope of the present invention is not limited thereto. More generally, then, the scope of the present invention encompasses any operating environment in which the disclosed concepts may be useful. By way of illustration and not limitation, embodiments of the present invention may be employed in conjunction with data backup and restore platforms such as Dell-EMC NetWorker and Avamar platforms and Dell-EMC RecoverPoint and VMWare SRM (Site Recovery Manager) platforms.

[0022] The data protection environment may take the form of a public or private cloud storage environment, a customer local storage environment, and a hybrid storage environment including public and private elements, but the scope of the present invention also extends to any other type of data protection environment. Any of these exemplary storage environments may be partially or completely virtualized. The storage environment may include or consist of a data center that is operable to provide services for read and write operations initiated by one or more clients and perform disaster recovery (DR) operations, including creating and writing to recovery disks.

[0023] In addition to the storage environment, the operating environment may also include one or more host devices, such as, for example, clients, each of which hosts one or more applications. Thus, a particular client may employ one or more instances of each of the one or more applications, or otherwise be associated with them. Generally speaking, the applications employed by the client are not limited to any particular function or type of function. For example, some exemplary applications and data include email applications (such as MS Exchange), file systems, and databases (such as Oracle databases and SQL Server databases). Applications on the client may generate new and / or modified data that needs to be protected.

[0024] Any device or entity disclosed herein may be protected by one or more data protection policies according to various embodiments of the present invention. Other examples of devices that may be protected by data protection policies according to embodiments of the present invention include, but are not limited to, containers and VMs.

[0025] Any device in the operating environment (including clients, servers, and hosts) may take the form of software, physical machines, or virtual machines (VMs), or any combination of these, but no implementation requires a specific device implementation or configuration. Similarly, data protection system components (such as, for example, databases, storage servers, storage volumes (LUNs), storage disks, replication services, backup servers, restore servers, backup clients, and restore clients) may also take the form of software, physical machines, or virtual machines (VMs), but no implementation requires a specific component implementation. In the case of using VMs, a hypervisor or other virtual machine monitor (VMM) may be used to create and control the VMs.

[0026] As used herein, the term "data" is very broad. Thus, the term includes, by way of example and not limitation, data segments (or simply "segments") that may be generated by a data stream segmentation process, data blocks, data chunks, atomic data, emails, any type of object, files, address books, directories, subdirectories, volumes, and any combination of one or more of the above.

[0027] Exemplary embodiments of the present invention are applicable to any system capable of storing and processing various types of objects in analog, digital or other forms. Although terms such as documents, files, blocks or objects may be used by way of example, the principles of the present disclosure are not limited to any particular form of representing and storing data or other information. Instead, such principles are equally applicable to any object capable of representing information.

[0028] Pay special attention now Figure 1 , operating environment 100 may include or consist of a data protection environment. The data protection environment may include an enterprise data center, a cloud data center, or both. The data protection environment may support various data protection processes, including, for example, data replication, deduplication, cloning, data backup, and data restoration. As used herein, the term backup is intended to be interpreted broadly and includes, but is not limited to, partial backup, incremental backup, full backup, clone, snapshot, continuous replication, and any other type of data copy, and any combination of the above. Any of the above may or may not be deduplicated.

[0029] Generally speaking, Figure 1 The exemplary operating environment 100 in FIG. 1 includes a production site 200 and a data center 300, which may be a cloud storage data center, but this is not required. The production site 200 (which may take the form of, for example, an enterprise or organizational site) may include any number "n" of production VMs 202. Likewise, the data center 300 may be, but is not required to be, a public cloud site, such as Amazon AWS, Microsoft Azure, or Google Cloud with Amazon Elastic Block Storage (Amazon EBS).

[0030] It should be noted that, as used herein, the term "VM" is broad and generally refers to a virtualized representation of any combination of hardware and / or software. The disclosed VMs are not limited to the execution of any particular function or group of functions. Some exemplary VMs that can be used to implement embodiments of the present invention are virtualizations of components including, but not limited to, file systems, applications, databases, disks, processors, storage elements, memory elements, software, hardware, and any combination of these.

[0031] Generally speaking, production VMs 202 may participate in the operation of computing systems and other systems at production site 200, and production VMs 202 may collectively form a cluster. Figure 1 204) can be used as a template or sample for configuring corresponding file systems and / or data of other production VMs 202. In some embodiments, a sample VM 204 is provided that is not a production VM.

[0032] The production site 200 or alternatively the data center 300 may additionally include a similarity module 206. Generally speaking, the similarity module 206 identifies those production VMs 202 that are similar to each other in, for example, their respective data and / or disk layouts. The similarity module 206 may be a stand-alone module, but this is not necessarily required. In terms of its operation, the similarity module 206 may automatically or at the initiation of a user (such as an administrator) query or poll a group of production VMs 202 to determine whether and where any such similarities exist between those production VMs 202. The degree to which a group of production VMs 202 have sufficiently similar data and disk layouts to be designated or deemed "similar" to each other by the similarity module 206 is not necessarily fixed, but such a degree may be fixed and may be defined as desired, for example based on the requirements of a particular application or operating environment. For example, an administrator or other user may define what constitutes sufficient similarity between two or more production VMs 202.

[0033] There are multiple ways to determine or decide whether production VMs 202 are similar to each other. One aspect to examine when determining similarity is different backup copies of the same VM. However, in at least some cases, most of the data may be placed in the same disk location on other production VMs 202. Another way to evaluate similarity is to have users manually indicate which VMs were created from a template. In some operating environments, for example, management tools such as Dell-EMC VMware vRealize Automation (vRA) can also be used to retrieve this data. This approach may be effective in situations where there are also VMs (e.g., user VMs) that are very similar to each other due to their creation from templates (where the customer created all VMs using similar software).

[0034] Likewise, the similarity module 206 may include or consist of a clustering algorithm, embodied as, for example, a clustering module at the production site 202, which may be used in conjunction with the chunking of the disks of the production site VMs 202. The clustering module may include or consist of a similarity module 208 and a chunk module 208.

[0035] For example, the production site 200 or alternatively the data center 300 may include a chunk module 208 for chunking the disks of the production VMs 202 into relatively large chunks, such as 4MB. Then, for each production VM 202, the chunk module 208 creates a hash vector and runs a process for clustering the production VMs 202 into several groups. Specifically, the clustering process may be performed using a clustering algorithm, such as, for example, a k-means clustering algorithm, wherein the distance between two or more production VMs 202, i.e., the relative similarity of these VMs 202 to each other, is determined by identifying the amount of chunks or chunks that are not common to the production VMs 202 under consideration. Finally, with respect to the production site 200, and as discussed in more detail below, each of the production VMs 202 may include a corresponding instance of a defragmentation agent 202a.

[0036] Continue to refer Figure 1 In at least some embodiments of the present invention, data center 300 is used as a DR target. That is, one or more of production VMs 202 are backed up at data center 300. In one particular embodiment discussed in more detail elsewhere herein, a single production VM 202 representing each production VM 202 in a group of production VMs 202 is backed up at data center 300 and then used to restore the group of production VMs 202 as part of a DR process.

[0037] The data center 300 may include both block storage 302 and object storage 304. As noted herein, block storage 302 may be relatively more expensive than object storage 304 in terms of money. For example, as part of a backup process, a full image 306 of a production VM 202 (such as VM1 202) may be created and stored in object storage 304. As part of a DR process, the full image 306 is then used to restore the production VM1 202 at the data center 300, and then the restored VM1 308 is used in conjunction with any differential VM backups 310 to restore one or more of the production VMs 202 at the data center 300. Figure 1 In the example of , a single restored production VM2 312 is indicated, but multiple production VMs 202 may be restored at the data center 300 based on the restored VM1 308 .

[0038] Finally, a recovery algorithm 314 is provided that may be hosted at the data center 300. In other embodiments, the recovery algorithm 314 may be hosted on a separate server that is in communication with the production site 200 and the data center 300. In other embodiments, the recovery algorithm 314 may be hosted at the production site 200. Generally speaking, the recovery algorithm implements and / or causes a DR process for the production VM 202 by creating a restored VM1 308 and then restoring one or more production VMs 202 at the data center 300 using the restored VM1 308 and the differential VM backup 310.

[0039] B. Aspects of an Exemplary Defragmentation Agent

[0040] As previously noted, each production VM 202 may include a corresponding instance of a defragmentation agent 202a. When present, the template VM 204 is also provided with a defragmentation agent 204a. Although no specific arrangement is required, instances of the defragmentation agent 202a may be pushed to their corresponding production VMs 202, such as by, for example, an administrator. Alternatively, the defragmentation agent 202a may be built into the production VM 202 when the production VM 202 is created. As discussed in more detail below, the defragmentation agent 202a on the production VM 202 will place data on respective VM disks, so that the VM disks of the protected machines (i.e., the production VM 202) having almost identical data to each other will also be almost identical to each other in their corresponding disk layouts. That is, the exemplary defragmentation agent 202a includes a distributed defragmentation algorithm that uses a predefined consistent logic to arrange data on similar production VMs 202, and attempts to place identical data at the same offset of the corresponding disks of the production VM 202.

[0041] In more detail, the exemplary defragmentation agent 202a performs at least the following processes, wherein the first process involves the empty space on the disk of the production VM 202. Specifically, when a file is deleted from the disk of the production VM 202, the defragmentation agent 202a zeroes the data and moves the zero area to the end of the disk. That is, the free space obtained by deleting the file is filled with zeros, and then these zero areas will be moved to the end of the disk of the production VM 202. Therefore, in a group of similar production VMs 202, not only will the disks of the production VM 202 have the same data, but the data will be arranged in the same way on each of these disks.

[0042] That is, the similarity algorithm will provide the cluster with a sample VM 204, and the data of other production VMs 202 will be arranged in a similar manner as in the sample VM 204, and then the defragmentation agent 202a will try to put the same data in the same location on the disk in different production VMs 202. For example, if in a new version of a production VM 202, a process simply moves data in the file system of the production VM 202 to a new location on the disk of the production VM 202 (such as, for example, by rewriting the file with small changes), the defragmentation agent 202a will move the data to the same disk location before the file overwrite occurred.

[0043] C. Aspects of an Exemplary Recovery Process

[0044] Continuing with reference to the exemplary defragmentation agent 202a, it can be seen that, utilizing the defragmentation agent 202a instances in the production VMs 202 forming the cluster, identical disk regions on different production VMs 202 can be detected, and then a single template (such as, for example, VM1 204) can be created, stored, and used to create EBS or other disks at the data center 300. When restoring one or more production VMs 202 from this single cloud VM1 308, a snapshot of the EBS or other disk is created, and any changes between the snapshot and the production VM(s) 202 to be recovered are applied to the disk to generate the restored VM(s) 2312.

[0045] It is noted that embodiments of the present invention may employ template VMs 204 to restore as few as a single production VM 202. However, identifying a larger set of production VMs 202 that are similar to one another will enable the use of relatively fewer template VM1 204, and therefore fewer full VM images to be stored in EBS or the data center 300, which in turn will reduce the amount of relatively expensive block storage space 302 required and also improve DR recovery time. Since a single template (such as, for example, VM1 204) can be used to restore multiple VM2 312, there is no need to create and store multiple templates and / or full VM1 images at the data center 300, and thus embodiments of the present invention are effective in achieving data deduplication. This data deduplication saves customers money because it reduces the amount of block storage 302 consumed at the data center 300.

[0046] Turning now to aspects of some exemplary DR recovery processes, embodiments of the present invention operate to create a temporary recovery VM1 308 having a disk attached that is the same size as the disk of the VM to be restored 202. Objects are read from the object store 304, specifically, from the VM1 full image 306, and written to the attached disk. Next, the disk file system (FS) is mounted, and driver modifications are installed that allow the disk to be booted on the cloud (such as, for example, at the data center 300). Once the disk is ready, the temporary recovery VM1 308 is shut down, the disk is attached to the new VM as a boot disk, and the new VM is restored to VM2 312. Note that the same process applies to non-boot disks, but no drivers need to be installed for the non-boot disk.

[0047] D. Example Host and Server Configuration

[0048] Now briefly refer to Figure 2 , Figure 1 Any one or more of the components disclosed in the embodiment may take the form of a physical computing device, may include a physical computing device, may be implemented on a physical computing device, or may be hosted by a physical computing device, an example of which is represented by 400. Similarly, in the case where any of the aforementioned elements includes a virtual machine (VM) or consists of a VM, the VM may constitute Figure 2 Virtualization of any combination of physical components disclosed in.

[0049] exist Figure 2 In the example of , the physical computing device 400 includes: memory 402, which may include one, some or all of the following items: random access memory (RAM), non-volatile random access memory (NVRAM) 404, read-only memory (ROM) and permanent memory; one or more hardware processors 406; non-transitory storage media 408; I / O devices 410 and data storage devices 412. The one or more memory components 402 of the physical computing device 400 may take the form of solid-state device (SSD) memory. Likewise, one or more application programs 414 including executable instructions are provided.

[0050] Such executable instructions may take various forms, including, for example, instructions that are executable to perform any method disclosed herein or a portion thereof, and / or instructions that can be executed by / at any of a storage site (whether local to the enterprise or a cloud storage site), a client, a production site, a data center, a backup and restore server to perform the functions disclosed herein. Similarly, such instructions may be executable to perform any other operations disclosed herein, including but not limited to read operations, write operations, backup operations, storage operations, snapshot operations, block operations, similarity evaluation operations, recovery operations (including disaster recovery operations), deduplication operations, and restore operations.

[0051] E. Aspects of Some Exemplary Methods

[0052] Next reference Figure 3 , provides details about a method for creating a VM template that can be used as a basis for defragmenting the disks of a VM population, wherein an example of such a method is generally indicated at 500. In some embodiments, the method 500 can be performed in cooperation with a block module and a similarity module, examples of which are disclosed herein. However, this allocation of functions is provided by way of example only and is not intended to limit the scope of the present invention in any way.

[0053] Method 500 may begin when corresponding disks of one or more production VMs are chunked 502, such as by a chunk module. The disks may be chunked 502 into chunks of any suitable size. In some embodiments, the chunk size is 4MB, but the disks may be chunked into larger or smaller chunks. Typically (but not necessarily), each chunked disk is chunked into chunks of the same size as the other disks.

[0054] Next, a hash vector of the blocks is created 504 for each VM. For example, the similarity module then evaluates the vectors based on a comparison of the vectors to each other and clusters 506 the VMs into one or more similarity groups. Specifically, the clustering process 506 can be performed using a clustering algorithm (such as, for example, a k-means clustering algorithm), in which the distance between two or more production VMs 202, i.e., the relative similarity of these VMs 202 to each other, is determined by identifying the blocks or amount of blocks that are not common to the production VMs 202 under consideration. The more blocks that two or more VMs have in common, the more similar these VMs are.

[0055] Once a similar production VM has been identified, a VM template is created 508, such as by a similarity module, wherein the disk structure and data of the VM template are similar or identical to the disk structure and data of the production VM. Like the production VM, the VM template includes an instance of a defragmentation agent. When the corresponding defragmentation agent makes a change (in conjunction with Figure 4As discussed further, a full image of the VM template is obtained 510 and backed up 512 in a data center or other data protection environment.

[0056] Reference now Figure 4 , provides details about some exemplary methods according to various embodiments of the present invention, one of which is generally indicated at 600. In general, exemplary method 600 relates to intelligent defragmentation of a portion or all of a VM, but other embodiments of method 600 may be applied in whole or in part to other devices. In at least some embodiments, the portion of a VM or other device that is defragmented is part of a file system of the VM or other device.

[0057] Embodiments of the method illustrated by method 600 may be performed in whole or in part at a production site. In certain embodiments, method 600 is performed in whole or in part by a defragmentation agent or other device of a VM, such as, for example, at a production site. Likewise, embodiments of the method illustrated by method 600 may be performed in conjunction with a sample or template VM.

[0058] The exemplary method 600 may begin when a file operation involving a file in a VM file system is detected 602. The file operation may be, for example, a write or delete operation. In the case of a write operation, the data identified in the write request is written 604 to the VM disk location specified by the VM template.

[0059] When the detected file operation 602 is a file delete operation, the file data is zeroed 606 and the zeroed area is moved 608 to a specific portion of the VM disk (such as the end of the disk). The specific destination specified for the zeroed area on the VM disk can be specified in a template VM, which can be accessed, for example, by an instance of a defragmentation agent resident on the VM where the file deletion occurred. The VM template can be created by a similarity algorithm (examples of which are disclosed herein), and the VM template can reside at the production site with the VM whose data is being moved.

[0060] As noted, the scope of method 600 is not limited to delete operations, but may additionally or alternatively apply to file write and file modification operations that may be detected 602 or notifications may be received about. To illustrate, if in a new version of a VM, a process simply moves data in the file system of the VM to a new location (such as, for example, by rewriting a file with a small change), the defragmentation agent instance at the VM will move the data to the same location as before the file was overwritten. More generally, in response to data modification operations on the VM disks, the defragmentation agent will move 608 the data on the VM disks as directed by the VM template.

[0061] Thus, each defragmentation agent instance on a group of VMs operates to, among other things, defragment the disks of the associated VMs and store common data in the same respective locations on each VM disk. In this manner, the defragmentation agents operate collectively to ensure similar or identical disk structures and data on each VM in a group or cluster of similar VMs. The basis for the disk structure and data to be implemented on each VM disk is provided by a VM template, which is accessed by the defragmentation agent.

[0062] Reference now Figure 5 , provides details about an exemplary method for restoring one or more VMs at a target site (such as, for example, a cloud data center) using a full image of a VM template. An example of such a method is described in Figure 5 700. In at least some embodiments, part or all of method 700 may be performed at a data center (such as, for example, a cloud storage site), but this is not required.

[0063] Method 700 may begin by receiving 702 one or more full images for backup. The full images may be full images of VM templates, which may be located at a production site, but this is not required. The full images may be received 702 ad hoc, periodically, or on any other basis. After the full images have been received, they are stored at a data center 704, such as in an object store.

[0064] At some point after the storage 704 of at least one full image of the VM template, a disaster recovery (DR) request is received 706. The DR request may be generated and received 706 in relation to a problem at a production site. The problem may be one or more events that result in corruption or loss of data, such as, for example, one or more production VMs. The DR request, which may be received from a production site, an administrator, or other entity, may specify restoring one or more production VMs at a data center and / or another DR target site.

[0065] After receiving the DR request 706, a temporary recovery VM is then created 708. The temporary recovery VM is created 708 based on a full image of a VM template. The full image of the VM template may reside at a data center. In some cases, there may be multiple full images of the VM template at the data center, and the creation of the recovery VM may be based on any of these images. The particular full image used to create the temporary recovery VM 708 may be selected based on a particular restore point or time identified in the DR request.

[0066] After the temporary recovery VM has been created 708, the temporary recovery VM is then used to restore 710 the VM(s) identified in the DR request. Advantageously, a single temporary recovery VM created based on a VM template can be used to restore multiple production VMs due to the similarities established between these production VMs.

[0067] When the production VM(s) have been restored 710 at the data center, the temporary recovery VM can then be shut down 712. If any incremental backups of the VM full image have been generated and stored after the VM full image was created, these incremental backups or differences can be applied 714 to the restored production VM so that the restored production VM accurately reflects the configuration of the production VM before it was affected by the DR event, both with respect to its stored data and disk structures. Once the restored production VMs are fully updated, they can be put into use and run 716. In at least some embodiments, the restored production VMs reside at the data center and are accessible at the data center. However, the restored production VMs can reside on any other DR target and are not necessarily hosted at the data center.

[0068] F. Exemplary Computing Devices and Associated Media

[0069] The embodiments disclosed herein may include the use of a special or general purpose computer including various computer hardware or software modules, as discussed in more detail below. The computer may include a processor and a computer storage medium carrying instructions that, when executed and / or caused to be executed by the processor, perform any one or more of the methods disclosed herein.

[0070] As noted above, embodiments within the scope of the present invention also include computer storage media, which are physical media for carrying or having computer-executable instructions or data structures stored thereon. Such computer storage media can be any available physical media that can be accessed by a general or special purpose computer.

[0071] By way of example and not limitation, such computer storage media may include hardware storage devices such as solid state disks / devices (SSDs), RAM, ROM, EEPROM, CD-ROMs, flash memory, phase change memory (“PCM”) or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other hardware storage devices that can be used to store program code in the form of computer-executable instructions or data structures that can be accessed and executed by general or special purpose computer systems to implement the functions disclosed in the present invention. Combinations of the above should also be included within the scope of computer storage media. Such media are also examples of non-transitory storage media, and non-transitory storage media also cover cloud-based storage systems and structures, but the scope of the present invention is not limited to these examples of non-transitory storage media.

[0072] Computer executable instructions include, for example, instructions and data that cause a general purpose computer, a special purpose computer, or a special purpose processing device to perform a specific function or group of functions. Although the subject matter has been described in language specific to structural features and / or methodological actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Instead, the specific features and actions disclosed herein are disclosed as exemplary forms of implementing the claims.

[0073] As used herein, the term "module" or "component" may refer to a software object or routine executed on a computing system. The various components, modules, engines, and services described herein may be implemented as objects or processes (e.g., as separate threads) executed on a computing system. Although the systems and methods described herein may be implemented in software, it is possible and contemplated to implement them in hardware or a combination of software and hardware. In the present disclosure, a "computing entity" may be any computing system as previously defined herein, or any module or combination of modules running on a computing system.

[0074] In at least some instances, a hardware processor is provided that is operable to execute executable instructions for performing a method or process, such as the methods and processes disclosed herein. The hardware processor may or may not include elements of other hardware, such as the computing devices and systems disclosed herein.

[0075] In terms of computing environment, embodiments of the present invention may be performed in a client-server environment, whether a network environment or a local environment, or in any other suitable environment. Suitable operating environments for at least some embodiments of the present invention include cloud computing environments, in which one or more of the client, server, or other machines may reside in and operate in the cloud environment.

[0076] The present invention may be implemented in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects as illustrative only and not restrictive. Therefore, the scope of the present invention is indicated by the appended claims rather than the foregoing description. All changes within the equivalent meaning and scope of the claims are included within the scope of the claims.

Claims

1. A method for intelligently defragmenting a file system inside a virtual machine to quickly restore it in the cloud and efficiently delete duplicate data for the cloud, comprising: One or more hardware processors are used to perform operations including: chunking a respective disk of each of a plurality of virtual machines (VMs) to create a respective plurality of chunks associated with each of the VMs; Creating, for each VM, a vector of the plurality of blocks associated with the VM; creating a cluster including one or more of the VMs based on the chunking process, wherein the cluster is created based on an evaluation of the vectors by a similarity module, and the VMs in the cluster are included in the cluster based on a comparison of their respective vectors; creating a VM template using the similarity module, the data and disk structure of the VM template matching the corresponding data and disk structure of each of the VMs in the cluster, wherein the VM template is created based on the VMs in the cluster; and In response to a file operation involving a first VM disk in a VM disk, the first VM disk is defragmented so that a disk structure of the first VM disk is the same as a disk structure of the VM template, wherein, when the file operation is a file rewrite operation, the operation also includes moving the file data to the same location where the file data was located before the rewrite operation was performed.

2. The method according to claim 1, wherein: Defragmentation of the first VM disk is performed by a defragmentation agent residing at the VM whose VM disk is the subject of the file operation.

3. The method according to claim 1, wherein: After the first VM disk has been defragmented, the data stored on the first VM disk is identical to the data stored on the disk of the VM template.

4. The method according to claim 1, wherein: The creation of the cluster is based on identifying an amount of chunks that are not common to the VMs.

5. The method according to claim 4, wherein: When the amount of the chunks not in common with the VMs is below a threshold, those VMs are included in the cluster.

6. The method according to claim 1, wherein: Defragmenting the first VM disk includes: zeroing data associated with the file operation such that the first VM disk includes zeros where the data previously resided; and The zeros are moved to a disk location of the first VM disk specified by the VM template.

7. A non-transitory storage medium having computer executable instructions stored therein, wherein the computer executable instructions, when executed by one or more hardware processors, perform operations, the operations comprising: chunking a respective disk of each of a plurality of virtual machines (VMs) to create a respective plurality of chunks associated with each of the VMs; Creating, for each VM, a vector of the plurality of blocks associated with the VM; creating a cluster including one or more of the VMs based on the chunking process, wherein the cluster is created based on an evaluation of the vectors by a similarity module, and the VMs in the cluster are included in the cluster based on a comparison of their respective vectors; creating a VM template using the similarity module, the data and disk structure of the VM template matching the corresponding data and disk structure of each of the VMs in the cluster, wherein the VM template is created based on the VMs in the cluster; as well as In response to a file operation involving a first VM disk in a VM disk, the first VM disk is defragmented so that a disk structure of the first VM disk is the same as a disk structure of the VM template, wherein, when the file operation is a file rewrite operation, the operation also includes moving the file data to the same location where the file data was located before the rewrite operation was performed.

8. The non-transitory storage medium according to claim 7, wherein: Defragmentation of the first VM disk is performed by a defragmentation agent residing at the VM whose VM disk is the subject of the file operation.

9. The non-transitory storage medium according to claim 7, wherein: After the first VM disk has been defragmented, the data stored on the first VM disk is identical to the data stored on the disk of the VM template.

10. The non-transitory storage medium according to claim 7, wherein: The creation of the cluster is based on identifying an amount of chunks that are not common to the VMs.

11. The non-transitory storage medium according to claim 10, wherein: When the amount of the chunks not in common with the VMs is below a threshold, those VMs are included in the cluster.

12. The non-transitory storage medium according to claim 7, wherein: Defragmenting the first VM disk includes: zeroing data associated with the file operation such that the first VM disk includes zeros where the data previously resided; and The zeros are moved to a disk location of the first VM disk specified by the VM template.

Images