Deduplication System Retention for Backup Retry
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deduplicated data systems face issues where new unique data segments become unreferenced and deleted during backup-job failures, leading to retransmission and inefficiencies when the backup job is retried.
Innovation Solution
Implementing a system that retains new data segments until the backup job is retried by setting a minimum retention period equal to or greater than the reclamation period, retry period, or user-defined duration, ensuring the data segments remain available for deduplication when the backup job is reattempted.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the deduplicated data system deletes unreferenced data segments to reclaim storage space, then storage space efficiency is improved, but data segments become unavailable for retry backup jobs
Solution Approach 1:
The system performs preliminary action by retaining newly uploaded data segments for a minimum retention period before allowing them to be deleted. This ensures that if a backup job fails partway through, the data segments remain available for retry operations. The retention period is configured to be at least as long as the maximum retry period, guaranteeing availability throughout the entire retry window.
2Reliability
If the deduplicated data system retains all data segments indefinitely, then data segment availability is improved, but storage space efficiency deteriorates
Solution Approach 1:
The system changes the parameter of data segment retention by introducing a minimum retention period that dynamically adjusts based on backup job status. Instead of indefinite retention or immediate deletion, the system maintains a time-based parameter that balances availability and space efficiency. The retention period is extended when backup jobs are in progress or have failed, and normal reclamation policies apply when jobs are complete.
3Reliability
If the backup system retries failed backup jobs, then data completeness is improved, but backup time increases due to retransmission
Solution Approach 1:
The system performs preliminary action by pre-retaining the data segments that were successfully uploaded during the initial backup attempt. When the backup job fails, these segments are already in place and ready for immediate reuse during the retry operation, eliminating the need for retransmission and significantly reducing backup time.
Solution Approach 2:
The system recovers the data segments that would otherwise be discarded or deleted. By retaining segments during the minimum retention period, the system recovers them for use in retry operations, transforming what would be wasted storage into useful reusable data, thereby improving efficiency and reducing redundant work.
Data Source
AI summary
A computer-implemented method for preserving deduplication efforts after backup-job failures may include (1) identifying a deduplicated data system that reduces redundant data storage by storing and referencing a plurality of deduplicated data segments and reclaims storage space by deleting unreferenced data segments from the deduplicated data system, (2) identifying a backup job that backs up data to the deduplicated data system, causes the deduplicated data system to store at least one new data segment available to be referenced within the deduplicated data system, and fails after the deduplicated data system stores the new data segment within the deduplicated data system causing the new data segment to be unreferenced within the deduplicated data system, and (3) causing the deduplicated data system to retain the new data segment until the backup job is retried despite the new data segment being unreferenced. Various other methods, systems, and computer-readable media are also disclosed.