Inode-Based Change Log for Faster Incremental Backup
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional backup systems face performance degradation due to the need to trawl entire file systems for incremental backups, processing of transactional change logs, and managing large change logs, which leads to unnecessary I/O operations and disk access issues.
Innovation Solution
Implementing a change log with a bounded size using inode numbers to index records, allowing for efficient identification of file changes and reducing unnecessary I/O operations by associating changes with inodes, thus enabling quicker backup processing and minimizing disk access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire file system is trawled to generate a list of modified files for incremental backup, then all modified files can be identified, but significant computing resources are consumed and backup performance degrades
Solution Approach 1:
The system pre-processes and records file change information in a change log data structure before backup operations are needed. This preliminary action stores file inode numbers, change timestamps, and change types in an organized manner, allowing the backup system to quickly retrieve only the necessary change information without trawling the entire file system during the actual backup process.
Solution Approach 2:
The invention extracts only the essential change information (inode numbers, timestamps, change types) from the file system and stores it in a dedicated change log. This extraction eliminates the need to examine entire files or traverse the complete file system structure during backup, as only the extracted change metadata is processed.
2Loss of information
If a transactional change log records all file changes in temporal order, then complete change history is maintained, but the change log becomes very large requiring frequent purging or wrapping
Solution Approach 1:
The change log is segmented by organizing entries according to inode numbers rather than strict temporal order. This segmentation groups all changes related to the same file together, allowing the system to efficiently query changes for specific files without processing the entire chronological sequence. The segmentation also enables selective retention of change logs based on inode activity patterns.
Solution Approach 2:
The system changes the organizational parameter of the change log from temporal ordering to inode-based grouping. This parameter change allows the change log to be structured in a way that facilitates efficient querying and reduces the need for frequent purging, as changes can be selectively managed by inode rather than requiring uniform temporal processing.
3Reliability
If file changes are recorded in temporal order in the change log, then all changes are captured chronologically, but unnecessary I/O operations occur when files are created and deleted frequently
Solution Approach 1:
The system performs preliminary grouping of file changes by inode number, so that when a backup operation queries for changes, all changes for a given file are already organized together. This preliminary organization eliminates the need for sequential I/O operations to retrieve scattered change records, as the inode-based structure allows for efficient batch retrieval of all changes related to active files.
4Loss of information
If conventional change logs are used without inode indexing, then all file changes are recorded, but backup operations cannot efficiently identify which files have actually changed
Solution Approach 1:
The invention introduces an inode-based indexing structure as an intermediary between the raw change log and the backup processing system. This intermediary layer organizes change information by inode number, allowing the backup system to efficiently query for changes without directly processing the raw temporal change log. The inode index acts as a mediator that translates complete change information into an efficiently queryable format.
Data Source
AI summary
Systems and methods for performing backups including incremental backups are provided. A change log is provided. The change log includes a map file that includes a bit for each inode in a file system. The bits are set on certain file events. The map file allows the files to be included in the next backup to be identified without having to trawl the entire file system. The change log also includes a data file that lists actual changes to the various files in corresponding records that are indexed by the inode number. The change log can enable faster incremental backups in part by eliminating unnecessary disk access requests.