Deleting Orphan Archived Files via Time-Based Decision Algorithm
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems face challenges in managing orphan archived files, leading to potential data loss when these files are deleted, as they lack an effective time-based decision algorithm for determining when to delete cloud objects associated with stub files.
Innovation Solution
The proposed solution involves traversing a database of a local storage system to identify records associated with stub files, determining the time of client operations, and using this information to identify snapshots, thereby providing an indication to delete cloud objects only if the client operation time precedes the snapshot time, ensuring data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If orphan archived files are deleted from storage array, then storage space is freed and data redundancy is reduced, but data loss may occur when files are restored from snapshots
Solution Approach 1:
The system performs preliminary actions by creating snapshots of the file system before deleting orphan archived files. The snapshot creation occurs prior to the deletion operation, preserving the state of files at the time of snapshotting. This allows subsequent restoration from snapshots without data loss, as the snapshot captures the file system state before the orphan file deletion.
Solution Approach 2:
The system implements feedback mechanisms by monitoring the relationship between stub file deletion times and snapshot times. The decision to delete a cloud object is based on feedback from comparing these timestamps - only cloud objects associated with stub files deleted before the earliest snapshot are removed. This feedback loop ensures data integrity while enabling storage space optimization.
2Quantity of substance
If cloud objects are deleted without time-based checks, then cloud storage space is reduced, but data consistency is compromised when files are restored from snapshots
Solution Approach 1:
The system changes the parameter of deletion decision from simple presence/absence to time-based evaluation. By introducing time as a critical parameter and comparing stub file deletion timestamps with snapshot timestamps, the system determines whether cloud objects should be deleted. This parameter change enables selective deletion that preserves data consistency while optimizing storage space.
3Reliability
If a time-based decision algorithm is implemented for deleting orphan archived files, then data loss risk is reduced, but system complexity increases due to snapshot management
Solution Approach 1:
The system performs self-service by automatically managing snapshots and orphan file deletion without requiring manual intervention. The virtualization software autonomously creates snapshots, identifies orphan archived files, compares timestamps, and executes deletions based on predefined policies. This automation reduces operational complexity while maintaining data integrity through consistent application of the time-based decision algorithm.
Data Source
AI summary
Methods, apparatus, and processor-readable storage media for deleting orphan archived files from a storage array using a time-based decision algorithm are provided herein. An example computer-implemented method includes traversing a database of a local storage system to identify a record associated with a stub file, wherein the record is indicative of a time of a client operation, involving the stub file, on a file system of the local storage system; identifying a particular snapshot in a set of available snapshots of the file system; and providing an indication to a cloud storage platform to delete a cloud object corresponding to the stub file in response to determining that the time of the client operation occurred earlier than a snapshot time associated with the particular snapshot in the set.