Deduplicated Universal Share Backup via Segment Fingerprints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Backup agents burden production servers with significant overhead, including CPU load and memory usage, and require substantial administrative effort, while existing deduplication methods require reassembly of data segments for backup creation, which is resource-intensive.
Innovation Solution
A method that creates a deduplicated copy of data objects by hashing segments, comparing fingerprints, and storing identifiers to locate matching segments in deduplication storage, allowing for the creation of a deduplicated backup without reassembling the data, thereby reducing resource usage and administrative burden.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If backup agents are used to perform backup operations on production servers, then backup functionality is achieved, but CPU load and memory usage on production servers increase significantly
Solution Approach 1:
The backup agent is extracted from the production server and placed on the backup storage system. The production server only needs to provide data access through standard file system interfaces, while all backup processing, deduplication, and data movement operations are performed by the backup agent on the backup storage system, eliminating the resource overhead on production servers.
Solution Approach 2:
A backup agent running on the backup storage system acts as an intermediary between the production server and the backup storage. This agent performs all backup operations including data reading, deduplication, compression, and storage management, allowing the production server to remain unaffected while achieving comprehensive backup functionality.
2Reliability
If traditional deduplication methods are used that reassemble data segments for backup creation, then complete backup copies are created, but resource-intensive reassembly operations are required
Solution Approach 1:
Data is pre-segmented and hashed during the backup process, with fingerprints stored in a catalog. When creating backup copies, the system queries the catalog for existing segment fingerprints and retrieves only the necessary segment identifiers, avoiding the need to reassemble entire data sets. This preliminary organization of data in segmented form enables efficient backup creation without full reassembly operations.
Solution Approach 2:
Instead of reassembling complete data copies, the system creates backup copies by copying segment identifiers from the catalog. The backup storage system stores references to deduplicated segments and uses these references to reconstruct data when needed, significantly reducing the computational resources required for backup creation while maintaining backup completeness.
Data Source
AI summary
A method includes creating a deduplicated universal share (US) of data objects, which in turn includes receiving a US of the data objects, deduplicating the US, wherein deduplicating the US includes: hashing segments of the US to generate respective US segment fingerprints; comparing US segment fingerprints to fingerprints for respective segments held in deduplication storage in order to identify segments in the deduplication storage that equate to the US segments, respectively, of the US; storing identifiers that directly or indirectly identify locations, respectively, of the segments, respectively, in the deduplication storage that equate to the US segments, respectively, of the US. After creating the deduplicated universal share, a deduplicated backup of the US is created without reassembling the US from segments held in the deduplication storage, the creating the deduplicated backup including: creating a list that comprises copies of the stored identifiers, and storing the list.