Incremental Content Indexing for Block-Based Backups
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional file system backups are inefficient due to the need to read and check millions of file records for modifications, while block-based backups are faster but require time-consuming content indexing to identify changed files and map them to their corresponding files, making them resource-intensive and impractical for large-scale data centers.
Innovation Solution
A method for facilitating forever incremental content indexing that reads only the changed blocks of a current backup, eliminating the need for a database of intermediate results and focusing on the number of changed files, thereby reducing indexing overhead by up to 100 times, without requiring the reading of all inode/MFT records.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional file system backup methods are used to check every file record for modifications, then complete backup accuracy is achieved, but backup time becomes extremely lengthy and resource consumption increases
Solution Approach 1:
The patent extracts only the changed blocks from the entire file system using change block tracking, rather than processing all file records. This selective extraction of modified data blocks enables incremental backups that achieve complete backup accuracy for changed files while dramatically reducing backup time by excluding unchanged blocks from the backup process
Solution Approach 2:
The patent applies partial action by performing content indexing only on the subset of changed blocks rather than the entire file system. This partial indexing approach maintains sufficient accuracy for backup verification while reducing the time and computational resources required, accepting that not every single file record needs to be fully examined when block-level changes are tracked
2Productivity
If block-based backup is used to track modified blocks, then backup speed is improved, but content indexing becomes time-consuming and resource-intensive
Solution Approach 1:
The patent extracts and processes only the changed blocks identified by change block tracking, rather than indexing the entire backup set. This selective extraction approach maintains backup speed by limiting indexing operations to only those blocks that actually changed, thereby reducing indexing time and resource consumption while still achieving complete content indexing for the modified portions
Solution Approach 2:
The patent performs change block tracking as a preliminary action before the backup and indexing processes. By pre-identifying which blocks have changed, the system prepares a targeted list of blocks that need content indexing, thereby speeding up the overall process by avoiding unnecessary indexing of unchanged blocks while maintaining complete indexing accuracy for changed content
3Loss of information
If content indexing is performed on all file records to discover file mappings, then complete file discovery is achieved, but resource consumption and processing time increase significantly
Solution Approach 1:
The patent extracts and processes only the changed blocks for content indexing, rather than examining all file records in the file system. This selective extraction approach achieves sufficient file discovery completeness for the incremental backup by focusing on changed content, while dramatically reducing processing resources and energy consumption by excluding unchanged blocks from the indexing process
Solution Approach 2:
The patent applies partial action by performing content indexing only on the subset of changed blocks identified through change block tracking. This partial indexing achieves adequate file discovery completeness for incremental backups by capturing all changed files, while reducing resource consumption by avoiding redundant indexing of unchanged files that would not affect the incremental backup content
Data Source
AI summary
A first list is received including locations of blocks of a file system that changed between current and previous backups. A superblock of the file system is read to generate a second list including locations of metadata records representing file objects. The first and second lists are intersected to return a third list including locations listed in the first and second lists. The locations in the third list are changed metadata records. A metadata record as of the current backup and identified in the third list is read. A previous version of the metadata record as of the previous backup is read. The current and previous metadata record versions are compared to detect potential conditions occurring between the backups. Upon detection of a first type of condition, information about the condition is recorded. Upon detection of a second type of condition, information about the condition is not recorded.