Version Chain Clustering for Faster Retrieval and Deletion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data processing systems using linear and binary tree version chains face inefficiencies in accessing earlier versions, data loss due to corruption, and suboptimal storage compression, particularly when dealing with long version chains and deletion operations.
Innovation Solution
Implement a version chain clustering method that groups similar data segments into clusters, reducing the length of the version chain and storing only representative delta-compressed data, which allows for faster retrieval and deletion of versions, and minimizes data loss by reducing the number of error-free deltas required for restoration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a linear version chain is used to store file versions, then storage space is reduced through delta compression, but processing time for accessing earlier versions increases proportionally with the distance from the most recent version
Solution Approach 1:
The patent divides the linear version chain into multiple clusters, where each cluster contains a subset of versions. This segmentation allows the system to access versions within the same cluster more quickly without requiring traversal of the entire chain, thus reducing processing time while maintaining storage efficiency through delta compression within each cluster.
Solution Approach 2:
The patent introduces a hierarchical dimension to the version chain structure by organizing versions into clusters arranged in a tree-like hierarchy. This adds a vertical dimension to the traditionally horizontal linear chain, enabling faster access through hierarchical navigation rather than sequential traversal, thereby reducing processing time while preserving compression benefits.
2Device complexity
If a linear version chain is used, then data structure simplicity is maintained, but deletion of a delta version requires decompressing all more recent versions which is time and processing intensive
Solution Approach 1:
By segmenting the version chain into independent clusters, the patent allows deletion operations to be confined to local clusters rather than affecting the entire chain. When a version is deleted, only the specific cluster needs to be reorganized, not all subsequent versions, dramatically improving deletion efficiency while maintaining overall structural simplicity.
Solution Approach 2:
The patent extracts the deletion operation from the global version chain context and localizes it to specific clusters. This extraction allows deletion to be performed independently within clusters without requiring decompression and reorganization of the entire chain, thus improving productivity while preserving data structure simplicity.
3Device complexity
If a linear version chain is used, then implementation simplicity is maintained, but data loss risk increases because corruption of any delta renders all earlier versions unavailable
Solution Approach 1:
The patent segments the version chain into isolated clusters where each cluster is independently stored and referenced. This segmentation creates redundancy at the cluster level, so that corruption in one cluster does not propagate to other clusters. The system can still access and reconstruct versions from unaffected clusters, improving reliability while maintaining implementation simplicity through modular cluster management.
Solution Approach 2:
The patent implements a form of prior cushioning by organizing versions into clusters that provide built-in protection against corruption. The cluster structure acts as a cushion that isolates the impact of corruption, ensuring that even if some deltas are corrupted, the overall version history remains recoverable through alternative paths within the clustered structure, thus improving reliability without complicating implementation.
4Speed
If a binary tree version chain is used to reduce access time, then processing speed improves, but storage compression becomes suboptimal because versions separated by larger time periods produce larger delta files
Solution Approach 1:
The patent segments the version chain into clusters that maintain temporal proximity within each cluster, similar to the linear structure. This segmentation allows delta compression to work effectively on closely spaced versions within clusters while the hierarchical arrangement of clusters provides fast access paths, thus achieving both good compression and fast access without the suboptimal compression issues of a full binary tree structure.
Solution Approach 2:
The patent applies local quality by ensuring that within each cluster, versions are arranged to maximize compression efficiency through temporal proximity. Each cluster locally optimizes for compression while the overall hierarchical structure provides fast access. This local optimization approach avoids the global compression degradation that occurs in binary tree structures where distant versions are compared.
Data Source
AI summary
A system, a method and a computer program product for storing data, which include receiving a data stream having a plurality of transactions that include at least one portion of data, determining whether at least one portion of data within at least one transaction is substantially similar to at least another portion of data within at least one transaction, clustering together at least one portion of data and at least another portion of data within at least one transaction, selecting one of at least one portion of data and at least another portion of data as a representative of at least one portion of data and at least another portion of data in the received data stream, and storing each representative of a portion of data from each transaction in the plurality of transactions, wherein a plurality of representatives is configured to form a chain representing the received data stream.