Resuming Steady-State Synchronization After Service Restarts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cloud storage systems face performance degradation due to the need for time-consuming rescan synchronizations after service restarts, especially for large namespaces.
Innovation Solution
A method to restart steady-state synchronization services without invoking a follow-on rescan synchronization by saving an indicator of the progress of the prior synchronization and using it to resume synchronization from the interrupted point.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If rescan synchronization is performed after service restarts, then the local and cloud file systems are brought back into synchronization, but the system experiences performance degradation and long synchronization times
Solution Approach 1:
The system performs preliminary actions by saving the progress indicator of steady-state synchronization before a service restart. This progress indicator captures the state of synchronization at the time of interruption, allowing the system to resume from the exact point where it left off rather than starting over, thereby avoiding time-consuming full rescan synchronizations after restarts
Solution Approach 2:
The system creates a copy of the synchronization progress state by persisting the progress indicator to non-volatile memory. This copied state information enables the synchronization service to reconstruct the exact position in the synchronization sequence after restart, eliminating the need to re-process already synchronized objects and significantly reducing synchronization time
2Reliability
If rescan synchronization is performed after service restarts, then data consistency between local and cloud file systems is ensured, but memory and processor resources are consumed excessively
Solution Approach 1:
The system performs the preliminary action of capturing and persisting the synchronization progress indicator before service restart. This pre-saved state information allows the system to skip processing of objects that were already synchronized, thereby reducing processor and memory resource consumption during post-restart synchronization while still ensuring complete data consistency
Solution Approach 2:
The system discards the need to re-process already synchronized objects by recovering the progress indicator from non-volatile memory. This recovered progress information allows the system to discard redundant processing efforts and focus only on objects that changed during the service interruption, significantly reducing resource consumption while maintaining data consistency
3Ease of manufacture
If steady-state synchronization is paused for software upgrading, then system maintenance can be performed, but a follow-up rescan synchronization is invoked causing performance degradation
Solution Approach 1:
The system performs preliminary action by saving the progress indicator of steady-state synchronization before pausing for software upgrades. This pre-saved progress state allows the synchronization service to resume exactly where it left off after the upgrade, eliminating the need for time-consuming full rescan synchronizations and maintaining high productivity during the maintenance window
Data Source
AI summary
A novel method for restarting a steady-state synchronization process includes persisting synchronization data in non-volatile memory before restarting, and utilizing the persisted synchronization data to restart the synchronization where the prior steady-state synchronization process left off. In a particular embodiment, the synchronization data is indicative of a particular point of the synchronization process and is utilized to begin synchronizing from that point after the service is restarted. In a more particular embodiment, the synchronization data is a particular operation of a sequence of operations performed as part of the synchronization. In another particular embodiment, operations associated with a batch are applied before stopping the synchronization.