File System Metadata Capture and Restore via Parallel Crawling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Maintaining and updating file system metadata in real-time is challenging due to the complexities of large file systems, where changes may not be atomic and can lead to undesirable conditions, and existing methods require the file system to be offline for scanning and restoration, causing inconvenience and inefficiency.
Innovation Solution
A dual process approach that includes multi-threaded, systematic scanning and on-demand, transactional actions to create, maintain, and restore file system metadata, allowing the file system to remain active and enabling parallel operations for metadata capture and restore, including inode management and defragmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the file system is taken offline to create and maintain a metadump, then the metadata can be accurately captured without missing changes, but the file system becomes unavailable causing downtime and inconvenience
Solution Approach 1:
The patent applies preliminary action by creating a metadump before file system changes occur, then using a crawler to systematically traverse the file system and capture metadata changes. The system prepares the metadata capture infrastructure in advance, allowing the file system to remain online while metadata is being collected and synchronized.
Solution Approach 2:
The patent introduces a crawler as an intermediary component that systematically traverses the file system and captures metadata changes. This intermediary enables the metadump to be updated without taking the file system offline, as the crawler acts as a mediator between the active file system and the metadata capture process.
2Ease of operation
If the file system is scanned to create a metadump while remaining active, then file system availability is maintained, but metadata changes may be missed leading to inconsistent state
Solution Approach 1:
The patent implements continuity of useful action by having the crawler systematically and continuously traverse the file system to capture metadata changes. Rather than performing a single snapshot, the crawler continuously monitors and captures changes, ensuring that metadata is updated progressively without interrupting file system operations.
Solution Approach 2:
The patent uses feedback mechanisms where the crawler detects file system changes and triggers updates to the metadump accordingly. The system monitors for changes in the file system state and feeds this information back into the metadata capture process, ensuring that the metadump remains synchronized with the actual file system state while the file system remains online.
3Adaptability or versatility
If multiple metadata updates are performed as separate operations, then flexibility in handling individual updates is improved, but atomicity cannot be guaranteed leading to undesirable partial states
Solution Approach 1:
The patent merges multiple separate metadata update operations into a single atomic transaction. By combining the updates that would otherwise be performed separately into one unified operation, the system ensures that either all updates succeed together or none are applied, maintaining atomicity while still allowing flexible handling of different update scenarios.
Solution Approach 2:
The patent segments the metadata update process into distinct components that can be independently managed but executed atomically. The update logic is divided into separate steps (allocation, creation, modification, deletion) that can be individually reasoned about, yet are executed as a unified atomic transaction to ensure consistency.
Data Source
AI summary
Example apparatus and methods create and maintain a functional mirror of file system metadata for an active file system. The functional mirror may be implemented as a set of database tables that allow a journal functionality for information received from the active file system. Example apparatus and methods also restore the file system metadata from the functional mirror. Mirror creation and maintenance can operate in parallel. Mirror maintenance and metadata restore can also operate in parallel. The functional mirror may be created primarily using a systematic, system-wide, multi-threaded crawler approach whose results are updated using an event-driven approach that accounts for changes that may make obsolete information acquired during the crawler approach. The metadata may be restored using a systematic, system-wide, multi-threaded crawler approach acting together with an event-driven approach that restores certain files out of order as they are needed.