Filesystem Scan Checkpointing for Concurrent Depth-First Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Scanning large filesystems for backup or restore operations places a heavy load on storage servers and networks, leading to potential crashes or the need for full scans from the start, which is inefficient and resource-intensive.
Innovation Solution
Implement a depth-first concurrent scan method with checkpoints to divide the scan workload into concurrent tracks, allowing for restarts from the latest checkpoint in case of crashes or terminations, using a processor-executable routine to manage scan workers and create snapshots at predefined intervals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a full filesystem scan is performed for backup or restore operations, then complete data coverage is achieved, but heavy load is placed on storage servers and network resources
Solution Approach 1:
The patent divides the filesystem scan into multiple concurrent depth-first tracks, allowing the scan workload to be segmented across multiple processing threads. This segmentation reduces the load on storage servers and network resources while maintaining complete data coverage by ensuring all filesystem portions are scanned across the distributed tracks.
2Reliability
If a full filesystem scan is performed, then complete backup data is obtained, but the scan may crash or require termination due to resource constraints
Solution Approach 1:
The patent creates checkpoints at predefined intervals during the scan process as a preliminary action. These checkpoints serve as predetermined recovery points that allow the scan to resume from the latest checkpoint if a crash or termination occurs, rather than requiring a complete restart from the beginning.
Solution Approach 2:
The system implements feedback by monitoring scan progress and resource conditions, then using checkpoint information to determine whether to continue or restart the scan from a saved state. This feedback mechanism improves scan completion reliability by automatically recovering from failures without losing progress.
3Reliability
If the scan is restarted from the beginning after a crash, then data integrity is maintained, but significant time is lost due to rescanning already processed portions
Solution Approach 1:
The patent creates checkpoints at predefined intervals during the scan process as a preliminary action. These checkpoints serve as predetermined recovery points that allow the scan to resume from the latest checkpoint if a crash or termination occurs, rather than requiring a complete restart from the beginning.
4Productivity
If incremental scans with change tracking are used, then scan time is reduced, but the entire filesystem hierarchy still requires scanning for full backup or restore
Solution Approach 1:
The patent divides the filesystem scan into multiple concurrent depth-first tracks, allowing the scan workload to be segmented across multiple processing threads. This segmentation reduces the load on storage servers and network resources while maintaining complete data coverage by ensuring all filesystem portions are scanned across the distributed tracks.
Data Source
AI summary
A system for scanning a file system is presented. The system includes a memory storing one or more processor-executable routines; and a processor communicatively coupled to the memory. The processor is configured to execute the one or more processor-executable routines to execute a file system scan using a depth-first concurrent scan method; create one or more checkpoints during the file system scan based on one or more predefined time intervals; and restart a scan from a latest checkpoint of the plurality of checkpoints. A related method is also presented.