Storage Synchronization via Block Fingerprint Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face inefficiencies in reverting data to a previous state, as simply recopying entire data sets is complex and slow, especially when log files grow unmanageable and cannot track older changes effectively.
Innovation Solution
Implementing a system that uses log files and snapshots, along with fingerprint comparisons of blocks, to identify and transfer only changed data segments, allowing for efficient data reconstruction and synchronization without the need for full data set recopying.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire stored contents of an originating storage system are recopied from a replica system, then data restoration is achieved, but the process becomes very complex and slow
Solution Approach 1:
The patent segments the data restoration process by dividing the storage system into blocks and using fingerprints to identify which specific blocks need restoration. Instead of recopying entire data sets, only the necessary blocks are transferred, significantly reducing complexity and time while maintaining data restoration reliability
Solution Approach 2:
The patent extracts only the essential information needed for restoration by using fingerprints (hash values) of blocks to identify changed data. This extraction approach allows the system to determine exactly which blocks need to be restored without processing or transferring unnecessary data, reducing complexity while achieving reliable restoration
2Reliability
If the entire stored contents of an originating storage system are recopied from a replica system, then data restoration is achieved, but the process becomes slow
Solution Approach 1:
The patent segments the data restoration process by dividing the storage system into blocks and using fingerprints to identify which specific blocks need restoration. Instead of recopying entire data sets, only the necessary blocks are transferred, significantly reducing complexity and time while maintaining data restoration reliability
Solution Approach 2:
The patent applies partial action by transferring only the specific blocks that have changed or need restoration, rather than performing the excessive action of copying entire data sets. This partial approach dramatically improves restoration speed while ensuring complete data recovery for the necessary blocks
3Productivity
If log files are used to track data changes, then data synchronization is improved, but log files grow unmanageable and cannot track older changes effectively
Solution Approach 1:
The patent extracts only the essential information needed for synchronization by using fingerprints (hash values) of blocks to identify changed data. This extraction approach allows the system to determine exactly which blocks need to be synchronized without processing or transferring unnecessary data, reducing complexity while achieving reliable restoration
Solution Approach 2:
The patent changes the parameter representation from storing complete log entries to storing compact fingerprints (hash values) of data blocks. This parameter transformation reduces the quantity of information that needs to be tracked while maintaining the ability to identify changes, preventing log files from growing unmanageable
Data Source
AI summary
A system for processing data includes a data storage device and a processor. The data storage device stores a set of data. The processor is configured to divide the set of data in the data storage system into a set of segments; compute a set of fingerprints, wherein the set of fingerprints comprises a fingerprint for each segment of the set of segments; store the set of fingerprints in a new snapshot; identify a second set of fingerprints in the new snapshot that are not already in a fingerprint index; cause a second set of segments associated with the second set of fingerprints to be stored in a backup data storage system; and cause the second set of fingerprints to be added to the fingerprint index.