VM Disk Defragmentation for Cloud Storage Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current disaster recovery systems for virtual machines (VMs) face challenges in efficiently managing VMs with logically similar but physically different filesystems, leading to increased storage costs and slower recovery times due to the need for multiple block device backups and metadata overhead.
Innovation Solution
Implementing a similarity algorithm and distributed defragmentation algorithm to identify and align similar VMs' data, ensuring identical data is placed at the same disk offsets, reducing physical differences and allowing for a single VM recovery template to be used for multiple VMs, thereby minimizing cloud storage usage and enhancing recovery speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If data is stored in object storage in the cloud for cost efficiency, then storage costs are reduced, but recovery speed slows down due to the need to restore from object storage to block storage upon demand
Solution Approach 1:
The patent pre-organizes data in object storage by creating a mapping between object storage locations and block storage locations during normal operations. This preliminary organization allows the system to know exactly where data resides and how to reconstruct it, enabling fast recovery without requiring full restoration from object storage to block storage during disaster recovery events
2Adaptability or versatility
If VMs with logically similar filesystems are physically stored differently on disk, then filesystem flexibility is maintained, but storage efficiency decreases due to inability to use single block device backup template
Solution Approach 1:
The patent segments the filesystem into logical components that can be independently mapped to physical storage locations. By separating logical filesystem structure from physical storage layout, the system maintains filesystem flexibility while enabling physical consolidation of identical data across multiple VMs, allowing use of single backup templates for storage efficiency
Solution Approach 2:
The patent introduces a mapping layer that acts as an intermediary between the logical filesystem and physical storage. This mapping mechanism translates logical filesystem operations into physical storage operations, allowing VMs to maintain flexible filesystems while their actual data is stored in a consolidated, deduplicated manner that enables single backup template usage
3Loss of energy
If deduplication is implemented to reduce storage space, then storage costs are reduced, but rebuild time increases due to reading data from larger amount of objects
Solution Approach 1:
The patent pre-creates and stores metadata mappings that record the relationship between deduplicated data objects and their original locations. This preliminary organization of metadata allows the system to quickly reconstruct data during rebuild operations by using the pre-computed mappings, avoiding the need to read through all deduplicated objects during recovery
4Productivity
If files are not zeroed upon deletion to save write operations, then disk write operations are reduced, but data integrity is compromised due to presence of garbage data on disk
Solution Approach 1:
The patent introduces a mapping layer that acts as an intermediary between the filesystem and physical disk. When files are deleted, the mapping is updated to mark those locations as free or invalid, while the actual disk data can remain until overwritten. This allows the system to maintain data integrity through mapping validation without requiring immediate zeroing of deleted file spaces, improving disk write efficiency
Data Source
AI summary
One example method includes chunking a respective disk of each of a plurality of virtual machines (VM) to create a respective plurality of chunks associated with each of the VMs, creating, based on the chunking process, a cluster comprising one or more of the VMs, creating a VM template whose data and disk structure match respective data and disk structures of each of the VMs in the cluster, and in response to a file operation involving a first one of the VM disks, defragmenting the first VM disk so that a disk structure of the first VM disk is the same as a disk structure of the VM template.