Library Server for Deduplication Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deduplication systems incur additional storage costs, redundant data transfer, and reference management overhead due to treating static and dynamic data similarly, which leads to inefficient data sharing between deduplication systems.
Innovation Solution
Implementing a library server that allows deduplication systems to share static data by virtually segmenting and generating metadata for shared files, enabling identification of segments that can be excluded from backup and deduplication operations, thereby reducing storage and transfer requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If existing deduplication systems treat static and dynamic data similarly using reference counts and reference lists, then data can be managed uniformly, but additional storage costs are incurred, redundant data transfer occurs, and reference management overhead increases
Solution Approach 1:
The patent segments data into static and dynamic categories, applying different management strategies to each. Static data that can be shared between deduplication systems is identified and handled separately from dynamic, user-specific data. This segmentation allows the system to avoid redundant storage and transfer of static data while maintaining uniform management for dynamic data through reference counts and lists.
2Device complexity
If existing deduplication systems manage static data without differentiation, then implementation is simpler, but redundant data transfer between backup server and deduplication systems occurs
Solution Approach 1:
The system performs preliminary identification of static data that can be shared between deduplication systems before backup operations. By pre-categorizing data as static or dynamic and establishing sharing rules in advance, the system avoids redundant transfer of static data while maintaining simple implementation through clear classification criteria and predefined management protocols.
3Reliability
If existing deduplication systems use reference counts and reference lists for all data, then data integrity is maintained, but reference management overhead increases
Solution Approach 1:
The patent applies different management qualities to different data types: static data that can be shared is managed with sharing awareness and exclusion from redundant reference tracking, while dynamic data maintains full reference count and reference list management for integrity. This local quality differentiation reduces reference management overhead for static data while preserving data integrity through appropriate management strategies for each data type.
Data Source
AI summary
Disclosed herein are methods, systems, and processes to share data storage-related services between multiple deduplication systems. In one embodiment, the method comprises receiving an indication that a local file corresponds to a shared file; virtually segmenting the shared file into a plurality of data segments, where virtually segmenting the shared file generates data segment fingerprints and library virtual mapping metadata, and the library virtual mapping metadata comprises fingerprint information corresponding to each of the plurality of data segments, and at least one of a unique file identifier, an offset of each data segment, and a size of each data segment; and transmitting the library virtual mapping metadata, where the library virtual mapping metadata is configured to be used in a subsequent deduplication operation.