Auto Retention Locking for Deduplication Backup Files
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In deduplicated backup storage systems, the automatic retention locking mechanism often locks backup files prematurely due to the lack of regular updates to the file modification time (mtime) during distributed segment processing, leading to broken backup files that cannot be cleaned up until their retention duration expires, causing unnecessary overhead and space occupancy.
Innovation Solution
Implement an efficient method for auto retention locking by setting the ARL flag and cooling off period (COP) when a file is created, resetting COP only when the application indicates complete writing, and allowing re-opening for modification within COP, enabling cleanup of incomplete ingest files and preventing premature locking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If automatic retention locking is triggered after a preconfigured delay from file mtime, then files are protected from deletion or modification, but backup files may be locked prematurely before deduplication processing completes
Solution Approach 1:
The system sets a flag on the backup file at creation time to indicate that the file should be automatically retention locked after a cooling off period. This preliminary action allows the system to prepare for automatic locking while waiting for deduplication to complete, rather than relying solely on mtime updates during processing.
Solution Approach 2:
The system monitors write operations to the backup file during the cooling off period. When writes are detected, the cooling off period is reset, providing feedback that prevents premature locking. This feedback mechanism ensures the file is only locked after deduplication processing is complete.
2Reliability
If the cooling off period is extended to allow complete deduplication processing, then premature locking is prevented, but storage space is occupied longer by incomplete backup files
Solution Approach 1:
By setting the automatic retention lock flag at file creation, the system establishes a predetermined locking mechanism that will activate after the cooling off period, rather than requiring manual intervention or waiting for mtime updates.
Solution Approach 2:
The system automatically monitors write operations and resets the cooling off period when writes are detected, eliminating the need for manual monitoring or intervention. The file automatically becomes eligible for locking once deduplication completes.
3Reliability
If manual retention locking is required for each backup file, then precise control over locking timing is achieved, but administrative burden increases significantly
Solution Approach 1:
The system automatically applies retention locking to backup files after the cooling off period expires without manual intervention. The automatic retention lock mechanism monitors files and applies locks autonomously, eliminating the need for administrators to manually lock each file.
Solution Approach 2:
The system provides feedback through the cooling off period mechanism, automatically detecting when deduplication is complete and triggering the retention lock at the appropriate time, replacing manual administrative control with automated feedback-driven locking.
Data Source
AI summary
A first flag is set on a backup file to which data to be backed up is written. The first flag indicates that the backup file should be automatically retention locked after a cooling off period is over. Before the cooling off period is over, a request is received to open the backup file for writes. The request is allowed and upon allowing the request, the first flag is cleared and a second flag is set on the backup file indicating that writes are in progress. The clearing of the first flag excludes the backup file from being automatically retention locked after the cooling off period is over.