File System Snapshot Indexing for Resource-Efficient Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional backup systems face challenges in efficiently searching for specific files across multiple file systems and snapshots due to high resource consumption and slow performance, especially when using lightweight virtual machines.
Innovation Solution
A device and method that index file system snapshots using a database with nodes and edges files, allowing for efficient searching with regular expressions across all file systems and snapshots while conserving memory and CPU resources, and enabling efficient replication to remote sites.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If elastic search is used to search across multiple snapshots, then search capability is improved, but resource consumption (memory and CPU) increases significantly and performance becomes too slow
Solution Approach 1:
The patent pre-generates and stores indexes of file metadata (names, paths, sizes, timestamps) in a database before search operations are performed. This preliminary indexing allows rapid retrieval during search operations without requiring heavy real-time processing, thus improving search speed while maintaining resource efficiency.
Solution Approach 2:
The patent extracts only the necessary file metadata (names, paths, sizes, timestamps) from the complete file system snapshots and stores them in a separate database index. This extraction allows search operations to work with a lightweight subset of data rather than scanning entire snapshots, dramatically improving search performance while reducing resource consumption.
2Ease of operation
If elastic search is used to search across multiple snapshots, then search capability is improved, but memory consumption increases significantly
Solution Approach 1:
The patent extracts only the necessary file metadata (names, paths, sizes, timestamps) from the complete file system snapshots and stores them in a separate database index. This extraction allows search operations to work with a lightweight subset of data rather than loading entire snapshots into memory, dramatically reducing memory consumption while maintaining search capability.
Solution Approach 2:
The patent creates a simplified copy of file system data in the form of metadata indexes stored in a database. Instead of working with complete snapshot copies that consume large memory, the system uses these lightweight metadata copies for search operations, reducing memory requirements while preserving essential search functionality.
3Loss of information
If conventional scanning is performed on mounted file system images, then file information can be collected, but the process consumes too much memory and CPU resources in lightweight virtual machines
Solution Approach 1:
The patent extracts only the necessary file metadata (names, paths, sizes, timestamps) from file system snapshots during the backup process and stores them in a database index. This extraction eliminates the need for heavy post-backup scanning operations in lightweight virtual machines, as the metadata is already available in a searchable format, thus reducing CPU and memory resource consumption while maintaining complete file information collection.
Data Source
AI summary
A device is configured to index file system names and paths of a single chain of backed-up snapshots. The device is configured to obtain a first snapshot of the file system. Further, the device is configured to scan the first snapshot of the file system to obtain a first scan of the nodes and the tree structure at a first time point. The device is further configured to insert the first scan into a database. Then the device is configured to index the nodes in the database.