Incremental Backup Using Directory Attributes to Skip Unchanged Trees
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing size of file systems and user behavior of retaining old, infrequently accessed files due to decreasing storage costs lead to inefficiencies in incremental backups, causing longer scanning times and increased processing loads, which interfere with concurrent client access.
Innovation Solution
Implementing a top-down search method with directory attributes such as tree modification attributes and lists of changed files to accelerate the identification and exclusion of unchanged files and directories during the incremental backup process, reducing the need for a full scan of the file system tree.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a conventional depth-first scan of the entire file system tree is performed to identify changed files, then all changed files are guaranteed to be found, but the scanning time increases significantly as file systems grow in size
Solution Approach 1:
The patent segments the file system tree into multiple directory trees rooted at different top-level directories. Instead of scanning the entire file system as one monolithic tree, the backup process divides it into manageable segments (e.g., /home, /var, /etc as separate trees). This segmentation allows parallel processing and reduces the time each individual scan must traverse, while still ensuring all changed files are identified across the complete file system.
Solution Approach 2:
The patent implements preliminary action by maintaining directory attributes (such as modification timestamps) that are updated whenever files are changed. Before performing an incremental backup scan, the system has already prepared these attributes in the background, so that during the actual backup window, the scan can quickly compare current file attributes against the stored attributes without performing heavy analysis. This preliminary preparation eliminates the need to re-scan unchanged directories during the backup process.
2Reliability
If the file system is scanned frequently to ensure changed files are captured, then backup completeness is maintained, but concurrent client access to the file system is interfered with
Solution Approach 1:
The patent implements dynamics by allowing the backup process to adapt its behavior based on system conditions. The backup system can dynamically adjust the scan frequency, pause or resume operations based on client activity levels, and prioritize client access when heavy file system activity is detected. This dynamic approach ensures backup reliability is maintained while minimizing interference with concurrent client operations.
Solution Approach 2:
The patent employs periodic action by scheduling incremental backups at predetermined intervals rather than continuously scanning. Between scheduled backup operations, the file system operates normally without backup interference. This periodic approach balances the need for regular backups with the need to minimize disruption to client access, allowing the system to switch between backup mode and normal operation mode.
3Measurement precision
If the entire file system tree is scanned to ensure all changed files are identified, then no changed files are missed, but the processing load increases due to handling files changed during scanning
Solution Approach 1:
The patent extracts and excludes unchanged directories from the scan process by using directory attributes to identify directories that have not been modified since the last backup. When a directory's attribute indicates no changes, the entire directory tree is extracted from the scan scope, eliminating the processing load of traversing and checking each file within those directories. This extraction approach maintains accuracy for changed files while dramatically reducing overall processing load.
4Quantity of substance
If storage capacity is increased to retain old files, then user data retention is improved, but incremental backup efficiency deteriorates due to more files to scan
Solution Approach 1:
The patent applies segmentation to handle large file systems by dividing them into multiple directory trees. This allows the backup process to work with smaller, more manageable segments rather than treating the entire multi-terabyte file system as one unit. The segmentation approach enables parallel processing of different directory trees and reduces the time required to scan each individual segment, thereby improving overall backup productivity despite the large total file system size.
Solution Approach 2:
The patent uses preliminary action by pre-computing and storing directory attributes that indicate whether directories contain changed files. This preliminary preparation allows the incremental backup process to quickly skip over large portions of the file system that haven't changed, even when the total file system size is very large. The preliminary attributes enable the system to focus processing only on relevant portions of the file system, maintaining backup speed independence from overall file system growth.
Data Source
AI summary
Directory attributes are provided so that the time for creating an incremental backup of a file system by a scan of the file system tree is proportional generally to the number of files that change between backups instead of the number of files in the file system. A tree modification attribute indicates whether or not any file in a directory tree has changed since the last backup. If no file has changed in the directory tree, then the entire tree is skipped during the scan for changed files. In addition, a list is compiled of the files in the directory that represent branches having at least one changed file, so that the list is scanned instead of the directory entries. When a file is changed for the first time since the last backup, the file is queued so that the directory attributes are updated in a background process.