Object Storage Deduplication Through Structural Fragment Linking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cloud-object-storage (COS) systems inefficiently store multiple versions of documents, leading to excessive storage resource usage due to the lack of effective deduplication at a structural level, as slight changes result in non-duplicate data blocks.
Innovation Solution
Decompress data objects into structural fragments, identify identical fragments across versions, replace them with links, and store the revised data object containing these links, thereby reducing storage requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional cloud-object-storage systems store multiple versions of documents as complete copies, then all versioning information is maintained, but storage resource usage becomes excessive
Solution Approach 1:
The patent segments documents into structural fragments (e.g., paragraphs, sections, headers) rather than treating them as monolithic blocks. This segmentation enables selective deduplication where only unique fragments are stored, while common fragments across versions are deduplicated. The system maintains versioning by tracking which fragments belong to which versions, resolving the contradiction between maintaining complete versioning information and reducing storage usage.
2Quantity of substance
If data is deduplicated at a data block level, then some storage reduction is achieved, but duplicate detection fails when only slight changes exist between versions
Solution Approach 1:
By segmenting documents into larger structural units (paragraphs, sections, headers) rather than small data blocks, the system increases the likelihood that identical or near-identical content will be recognized as duplicates. This structural segmentation maintains storage reduction benefits while improving duplicate detection accuracy for documents with slight changes.
Solution Approach 2:
The patent applies different treatment to different parts of the document structure. Structural elements like headers, paragraphs, and sections are identified and processed separately, allowing the system to recognize duplicates at the appropriate structural level while maintaining precision in detecting actual content differences.
Data Source
AI summary
Storing compressed data objects by decompressing a first data object, identifying, differences between the first and second data objects, creating a new version of the second data object according to the differences, the first data object, and a reference object, and storing the first data object and the new version of the second data object.