Partial Data Synchronization via Block-Level Change Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods for synchronizing data across multiple users and storage devices are costly, inefficient, and can cause network bottlenecks, especially when dealing with large datasets, as they often require copying entire files, which is impractical and unsuitable for multi-party updates, affecting production environments.
Innovation Solution
The implementation of a partial synchronization method that designates specific portions of primary data files for synchronization, using enhanced storage managers and data agents to auto-restore changes from backup data, minimizing network communication and storage resource usage, and enabling more frequent updates with reduced resource impact.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional full file copying is used to synchronize data across multiple users, then data consistency is maintained, but network bandwidth consumption and storage resource usage increase significantly
Solution Approach 1:
The patent segments files into multiple blocks and only synchronizes the specific blocks that have changed, rather than copying entire files. This is achieved through block-level change detection and selective synchronization of only the modified portions across multiple users, significantly reducing network bandwidth consumption while maintaining data consistency.
Solution Approach 2:
The patent extracts and identifies only the specific changed portions (blocks) from files that need to be synchronized. By using change detection mechanisms to extract only the modified blocks rather than entire files, the system reduces unnecessary data transmission while ensuring that all users receive the necessary updates for consistency.
2Loss of time
If frequent full file synchronization is performed, then data currency is improved, but impact on production environment performance increases
Solution Approach 1:
The patent performs partial synchronization by only copying changed blocks rather than entire files. This allows frequent synchronization operations to be executed with minimal resource consumption, improving data currency while having negligible impact on production environment performance since only necessary data blocks are transmitted.
3Loss of energy
If incremental change propagation is used, then network resource usage is reduced, but the scope of changes synchronized may be broader than necessary
Solution Approach 1:
The patent applies local quality by treating different blocks of files differently based on their change status. Changed blocks are identified and synchronized with high priority, while unchanged blocks are not transmitted. This block-level granularity allows precise control over synchronization scope, ensuring that only necessary changes are propagated while minimizing network resource usage.
Data Source
AI summary
Disclosed methods and systems leverage resources in a storage management system to partially synchronize primary data files based on synchronizing selected portions thereof without regard to changes that may be occurring in other non-synchronized portions. Accordingly, a number of primary data files may be partially synchronized by synchronizing designated portions thereof via auto-restore operations from backup data. This approach relies on storage management resources to designate portions of source data that is to be kept synchronized across any number of targets; detect changes to the designated portions; back up changes to secondary storage; and distribute the changes from secondary storage to the associated targets, with minimal impact to the primary data environment. The approach may be mutually applied, so that changes in any one of an associated group of source data files may be likewise detected, backed up, and distributed to the other members of the group.