Multilayer De-duplication for Virtual Machine Snapshot Backup
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional virtual machine snapshot backup systems face challenges in achieving high data processing speed, effective de-duplication, and low resource utilization, particularly in handling large-scale data backup requests and detecting redundant data across multiple users, which affects storage efficiency and user experience.
Innovation Solution
The method and system implement multilayer de-duplication by dividing virtual machine snapshots into child data blocks and segments, applying de-duplication at multiple levels to exclude redundant data, using data fingerprints and public data sets to identify and store frequently repeated data segments, and optimizing resource usage through periodic scanning and indexing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional backup techniques (such as Amazon EBS snapshot or ECM technique) are used, then data backup can be performed, but storage efficiency is poor and redundant data cannot be effectively excluded
Solution Approach 1:
The patent segments the virtual machine disk into blocks, and each block into pages, creating a multi-level hierarchical structure. This segmentation enables fine-grained de-duplication at different levels (block level and page level), allowing the system to identify and exclude redundant data more effectively while maintaining the ability to restore complete disk images when needed.
Solution Approach 2:
The patent introduces a new dimension to de-duplication by implementing a three-layer hierarchical structure (disk blocks, pages, and a public data set). This multi-dimensional approach allows the system to track data redundancy across different levels and scopes, improving both storage efficiency and de-duplication reliability compared to conventional single-level approaches.
2Quantity of substance
If high de-duplication ratio is achieved by excluding most redundant data, then storage efficiency improves, but data processing speed decreases
Solution Approach 1:
The patent performs preliminary actions by pre-dividing the disk into blocks and pages before backup, and pre-establishing the hierarchical de-duplication structure. During the backup process, the system can quickly identify which blocks and pages need to be processed versus which can be directly referenced from previous backups or the public data set, significantly improving processing speed while maintaining high storage efficiency.
3Quantity of substance
If multilayer de-duplication is implemented to exclude redundant data, then storage efficiency improves, but system complexity increases
Solution Approach 1:
The patent manages complexity by segmenting the de-duplication process into three distinct layers: block-level de-duplication, page-level de-duplication, and public data set de-duplication. Each layer handles specific aspects of redundancy removal, making the overall complex task more manageable and implementable through standardized procedures at each level.
Solution Approach 2:
The patent creates a public data set that serves multiple functions: it stores frequently repeated data blocks, serves as a reference for de-duplication operations, and enables efficient restoration of common data. This multi-functional component simplifies the overall system by consolidating multiple de-duplication operations into a unified mechanism.
4Reliability
If complete snapshot backup is performed to ensure data safety, then reliability improves, but resource utilization increases
Solution Approach 1:
The patent applies partial de-duplication by selectively processing only the necessary blocks and pages at each level. Instead of treating the entire disk as a single unit, the system identifies and processes only the changed or potentially redundant portions, reducing resource consumption while maintaining adequate data safety through the hierarchical reference structure.
Solution Approach 2:
The patent uses references and pointers throughout the hierarchical structure to represent data blocks and pages. Rather than physically copying all data, the system creates a compact representation where unchanged data is referenced from previous backups or the public data set, significantly reducing resource utilization while maintaining data integrity and safety.
Data Source
AI summary
The present disclosure provides an example method and system for virtual machine backup based on multilayer de-duplication. A virtual machine snapshot is divided into multiple child data blocks. Each child data block is divided into multiple data segments. Multilayer de-duplication is applied to the virtual machine snapshot to exclude data causing duplicate backup in the virtual machine snapshot. The remaining virtual machine snapshot data after the processing of the multilayer de-duplication is stored.