Two-Level Delta Resemblance Data Storage Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As data volume increases, existing data backup methods face performance degradation due to increased segments and index entries, leading to reduced access performance and storage efficiency, as larger segments result in fewer identical occurrences and less efficient storage.
Innovation Solution
The method employs resemblance-based compression by breaking data streams into logical segments, representing them as composites of base and delta segments, using one-level or two-level delta coding, and combining deltas to maintain performance and scalability while optimizing storage efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If the number of segments is increased to improve storage efficiency, then storage efficiency is improved, but access performance deteriorates
Solution Approach 1:
The patent divides data into segments and further segments the index into multiple partitions. This segmentation allows the system to maintain fine-grained storage efficiency while improving access performance by limiting the search space to specific partitions rather than scanning the entire index.
Solution Approach 2:
The patent introduces a new dimension by creating multiple index partitions with different granularity levels. This multi-dimensional indexing structure allows the system to optimize for both storage efficiency and access performance simultaneously by selecting appropriate partition levels based on access patterns.
2Speed
If the size of segments is increased to maintain access performance, then access performance is maintained, but storage efficiency deteriorates
Solution Approach 1:
The patent applies segmentation at multiple levels: data is divided into segments, and the index is divided into multiple partitions. This multi-level segmentation allows smaller effective segment sizes for indexing purposes while maintaining larger physical segment sizes for storage efficiency, resolving the contradiction between the two requirements.
Solution Approach 2:
Different index partitions can have different characteristics and granularity levels tailored to specific access patterns. This local optimization allows the system to maintain access performance for frequently accessed data while achieving storage efficiency for less frequently accessed data.
Data Source
AI summary
Storage using resemblance of data segments is disclosed. It is determined that a new segment resembles a second prior stored segment wherein the second prior stored segment is represented as a first stored delta and a first prior stored segment. A second delta between the new segment and the prior stored segment is determined. A representation of the new segment based at least in part on the second delta is stored.