Partial Data Synchronization via Block-Level Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods for synchronizing data across multiple users and storage devices are costly, inefficient, and can cause network bottlenecks, especially when dealing with large datasets, as they often require copying entire files, which is impractical and unsuitable for multi-party updates, affecting production environment performance.
Innovation Solution
The implementation of a partial synchronization method that designates specific portions of primary data files for synchronization, using enhanced storage managers and data agents to auto-restore changes from backup data, minimizing network communication and storage resources, and enabling more frequent updates with reduced resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional full-file copying is used to synchronize data across multiple users, then data consistency is maintained, but network bandwidth consumption and storage resource usage increase significantly
Solution Approach 1:
The patent segments files into multiple blocks and only synchronizes the specific blocks that have changed, rather than copying entire files. This is achieved through block-level backup and restore operations, where the storage manager identifies and transfers only the modified blocks between clients, dramatically reducing network bandwidth consumption while maintaining data consistency.
Solution Approach 2:
The patent extracts and synchronizes only the changed portions (blocks) from files rather than transferring complete files. The storage manager identifies modified blocks and extracts only those specific blocks for synchronization, eliminating the need to transmit unchanged data and reducing network resource usage.
2Reliability
If traditional full-file copying is used to synchronize data, then all users receive complete data updates, but storage resource consumption increases
Solution Approach 1:
The patent divides files into blocks and maintains backup copies of individual blocks rather than complete files. When synchronization is needed, only the specific changed blocks are restored and transferred, reducing storage resource consumption while ensuring data completeness through selective block-level operations.
Solution Approach 2:
The patent performs partial synchronization by transferring only the necessary changed blocks rather than complete files. This partial action approach synchronizes sufficient data to maintain consistency without the excessive resource consumption of full-file copying, transferring only what is needed for data completeness.
3Loss of time
If frequent data synchronization is performed to keep users current with changing data, then data currency is improved, but network bottlenecks and performance impact on production environment increase
Solution Approach 1:
The patent enables frequent synchronization by segmenting data into blocks and only transferring changed blocks rather than complete files. This reduces the time and network resources required for each synchronization operation, allowing more frequent updates to occur without creating network bottlenecks or impacting production environment performance.
Solution Approach 2:
The patent implements periodic change detection and synchronization at the block level, where the storage manager periodically identifies changed blocks and triggers selective synchronization only when changes are detected. This periodic action maintains data currency while minimizing unnecessary network traffic and performance impact during periods when no changes occur.
Data Source
AI summary
Disclosed methods and systems leverage resources in a storage management system to partially synchronize primary data files based on synchronizing selected portions thereof without regard to changes that may be occurring in other non-synchronized portions. Accordingly, a number of primary data files may be partially synchronized by synchronizing designated portions thereof via auto-restore operations from backup data. This approach relies on storage management resources to designate portions of source data that is to be kept synchronized across any number of targets; detect changes to the designated portions; back up changes to secondary storage; and distribute the changes from secondary storage to the associated targets, with minimal impact to the primary data environment. The approach may be mutually applied, so that changes in any one of an associated group of source data files may be likewise detected, backed up, and distributed to the other members of the group.