Delta Catalogs for Backup Storage Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional file system backup methods require significant storage space due to the need for large catalogs in incremental backups, making it inefficient and costly for enterprises to utilize features like true image restore and synthetic backups.
Innovation Solution
A delta catalog system that maintains a backed up object table and extent map, recording only changes and deletions between backups, reducing the need to store every object and thus minimizing storage requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If incremental backups are used to backup only changed data, then backup efficiency is improved, but catalog storage space increases significantly
Solution Approach 1:
The catalog is segmented into two distinct components: a compact delta catalog that stores only changes (additions, modifications, deletions) between backups, and a full backup catalog that stores complete object lists at specific points in time. This segmentation allows the system to maintain efficient incremental backup tracking while reducing the storage burden of change tracking data.
Solution Approach 2:
The patent extracts and stores only the essential change information (object IDs, operation types, timestamps) in the delta catalog, separating this from the complete object metadata that would otherwise be required in traditional incremental backup catalogs. This extraction of minimal necessary information dramatically reduces catalog storage requirements.
2Reliability
If full backups are performed periodically, then data completeness is ensured, but backup time and storage space increase
Solution Approach 1:
The system performs preliminary incremental backups that capture only changes since the last full backup, using the delta catalog to track modifications. This preliminary action of capturing only changes allows the system to maintain data completeness while minimizing the time and storage required for each backup operation.
Solution Approach 2:
Instead of copying all data in full backups, the system copies only the delta catalog entries representing changes. The full backup catalog serves as a reference that is updated incrementally, allowing the system to maintain complete data recovery capability while performing much smaller, faster backup operations.
3Measurement precision
If catalogs store all objects for true image restore, then restore accuracy is improved, but storage requirements become prohibitive
Solution Approach 1:
The delta catalog acts as an intermediary between the full backup catalog and the restore operation. It provides the precise change information needed to reconstruct the file system state at any point in time by applying changes sequentially to the base full backup, achieving accurate restoration without storing complete object catalogs for every backup point.
Solution Approach 2:
The system changes the parameters of catalog storage from storing complete object metadata for every backup point to storing only change operations (add, modify, delete) with minimal identifiers. This parameter change from comprehensive object tracking to differential operation tracking maintains restore accuracy while dramatically reducing storage requirements.
Data Source
AI summary
A method and mechanism for performing data backups in a computing system. A “delta” catalog is utilized for the maintenance of data backups. The delta catalog includes a backed up object table and an extent map. For a given backup, the backed up object table update stores entries which identify only those objects backed up during that backup. The backed up object table is further configured to store entries which identify objects which were deleted between the time of a prior backup procedure and a current backup procedure. The extent map is configured identify all objects present in the system at the time of a particular backup. The extent map identifies the objects which were present as one or more sequences of entries in the backed up object table.