Inline Snapshot Deduplication for Virtual Machine Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data management systems (DMS) face challenges in efficiently managing snapshots of virtual machines (VMs), as deduplication processes often consume high I/O and processing resources and require significant storage space, especially when performed after snapshots are obtained and stored.
Innovation Solution
The implementation of inline snapshot deduplication, where a data management system generates a composite hash for a VM during snapshot creation and compares it to previously generated hashes to identify suitable deduplication bases, allowing for real-time deduplication and reduced storage needs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If post-storage deduplication is performed after snapshots are obtained and stored, then storage space is saved, but I/O and processing resources are consumed highly
Solution Approach 1:
The patent applies preliminary action by performing deduplication during the snapshot creation process itself, before the snapshot is stored. The system calculates hashes of data blocks and compares them with existing snapshots in real-time, determining which blocks need to be stored and which can be skipped. This preliminary deduplication action eliminates the need for subsequent heavy I/O operations while still achieving storage space savings.
2Quantity of substance
If post-storage deduplication is performed after snapshots are obtained and stored, then storage space is saved, but processing time is increased
Solution Approach 1:
The patent performs deduplication calculations and comparisons during the snapshot creation process, before storage occurs. By calculating hashes and comparing data blocks in real-time as the snapshot is being created, the system eliminates the need for separate post-storage deduplication processing, thereby reducing overall processing time while maintaining storage space efficiency.
3Reliability
If full snapshots are stored for all VMs, then data reliability is maintained, but storage capacity is consumed rapidly
Solution Approach 1:
The patent applies local quality by differentiating the treatment of data blocks based on their uniqueness. Instead of storing all data blocks uniformly, the system analyzes each block's hash and determines locally whether it needs to be stored in the snapshot. Unique blocks are stored, while duplicate blocks are skipped, achieving storage capacity optimization while maintaining data reliability through selective storage.
Solution Approach 2:
The system performs preliminary hash calculation and comparison during snapshot creation to determine which data blocks are unique and need storage. This preliminary action enables the system to store only necessary blocks, reducing storage capacity consumption while ensuring that all unique data required for reliability is captured.
Data Source
AI summary
A data management system (DMS) may select, prior to obtaining a first snapshot of a first virtual machine (VM) and from among one or more snapshots previously obtained by the DMS, a second snapshot to use for deduplication of the first snapshot. The DMS may obtain the first snapshot after selecting the second snapshot. Obtaining the first snapshot may include writing a first subset of data blocks from the first VM to a snapshot file for the first snapshot based on the first subset of the data blocks from the first VM being different from a first corresponding subset of the second snapshot and refraining from writing a second subset of the data blocks from the first VM to the snapshot file for the first snapshot based on the second subset of the data blocks from the first VM matching a second corresponding subset of the second snapshot.