Granular Data Extent Tracking for Differential Backup Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data management systems inefficiently handle changes to file metadata, leading to unnecessary backups of entire files, and fail to differentiate between metadata and payload data changes, resulting in resource wastage.
Innovation Solution
An information management system that organizes data into granular units, tracking changes at different levels of granularity, allowing for differential backup of only changed extents, thereby distinguishing between metadata and payload changes and optimizing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the system backs up entire files when any change is detected, then data protection completeness is improved, but resource consumption and backup time increase significantly
Solution Approach 1:
The patent segments files into multiple extents of fixed size (e.g., 4MB each). When a change occurs, only the affected extent is identified and backed up, rather than the entire file. This segmentation allows the system to protect data completely while minimizing the amount of data actually processed during backups, resolving the contradiction between reliability and productivity.
2Reliability
If the system treats metadata changes as file changes, then data integrity is maintained, but unnecessary data transfers and storage operations occur
Solution Approach 1:
The patent extracts metadata from the file content and stores it separately in a catalog data structure. When metadata changes occur (such as permission modifications), only the metadata record in the catalog is updated, not the entire file. This extraction allows the system to maintain data integrity while eliminating unnecessary data transfers and storage operations associated with backing up unchanged file content.
3Reliability
If the system performs full backups frequently, then data recovery reliability is improved, but system performance and productivity deteriorate
Solution Approach 1:
The patent performs preliminary actions by maintaining a catalog that tracks the location and state of each extent within every file. This catalog is updated continuously as changes occur. When a backup is needed, the system uses this pre-established catalog information to quickly identify only the changed extents that need to be backed up, rather than performing expensive full scans of entire files. This preliminary tracking action enables reliable data recovery while minimizing the performance impact of backup operations.
Data Source
AI summary
According to certain aspects, a client computing system can be configured to map stored database data into a plurality of second data units having a second level of granularity larger than first data units; store a mapping of the second data units to the first data units; monitor writes to the database data by the database application; identify of the writes changed first data units of the first data units which have changed as a result of the writes; use the mapping to identify changed second data units of the second data units which each include at least one of the changed first data units; maintain a table listing the changed second data units; and in response to instructions to create a secondary copy of the database data, consult the table to identify the changed second data units, and forward the changed second data units to secondary storage controller computer(s).