Global Data Deduplication Across Storage Volumes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data deduplication technologies are limited to comparing and eliminating redundant data within the same storage volume, failing to perform deduplication across different storage volumes, which leads to inefficient use of disk space and increased costs due to the need for additional storage as backups are made.
Innovation Solution
A storage management computing device that performs global data deduplication by identifying and comparing data blocks across different storage volumes using checksums and bit-by-bit comparisons, creating or updating references in a metadata file to share and manage redundant data blocks, thereby optimizing storage usage across multiple volumes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data deduplication is performed only within the same storage volume, then the deduplication process is simple and fast, but storage space cannot be efficiently shared across different volumes leading to redundant data storage
Solution Approach 1:
The patent merges multiple storage volumes into a unified deduplication namespace, allowing data blocks from different volumes to be compared and deduplicated together. This is achieved by creating a global deduplication index that spans across volume boundaries while maintaining individual volume metadata structures, thus enabling cross-volume deduplication without complete system redesign
Solution Approach 2:
The patent introduces a volume aggregate metadata structure that acts as an intermediary between individual volume metadata and the global deduplication process. This aggregate metadata layer coordinates cross-volume deduplication operations while preserving volume-level isolation and management capabilities, solving the complexity issue by providing a structured mediation layer
2Quantity of substance
If additional storage disks are added to accommodate backup data, then more backup capacity is available, but costs increase including disk purchase, power, cooling, and physical storage
Solution Approach 1:
The patent identifies and eliminates redundant backup data across volumes by detecting duplicate data blocks and replacing them with references to existing copies. This discards unnecessary duplicate storage while recovering valuable disk space for storing new backup data, directly reducing the need for additional physical disks and associated energy costs
Solution Approach 2:
The patent performs deduplication operations proactively during data ingestion and backup processes, identifying and eliminating redundant data before it consumes additional storage capacity. This preliminary action prevents the accumulation of duplicate data that would otherwise require additional disks, power, and cooling infrastructure
3Productivity
If cross-volume data deduplication is implemented, then storage efficiency improves, but the complexity of data management and metadata tracking increases
Solution Approach 1:
The patent segments the metadata management system into distinct layers: volume-level metadata for individual volume tracking, aggregate-level metadata for cross-volume coordination, and global deduplication indexes for duplicate detection. This segmentation allows complex cross-volume operations to be broken down into manageable, modular components that can be processed independently
Solution Approach 2:
The patent adds a new dimension to metadata management by introducing volume aggregates that operate at an intermediate level between individual volumes and the global system. This dimensional addition provides a structured hierarchy that simplifies cross-volume operations while maintaining volume-level independence, managing complexity through hierarchical organization
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method, non-transitory computer readable medium, and device that assists with performing global data deduplication on data blocks across different volumes includes identifying at least two data blocks stored in two or more storage volumes. It is determined whether the at least two data blocks are classified as a shared data block. A new data volume signature is created when the at least two data blocks are determined to be shared. One of the at least two data blocks that are determined to be shared is deleted and the other one of the at least two data blocks and the created signature in one of the two or more storage volumes is stored.