File Immutability via Deduplication Filesystem Redirection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deduplication filesystems in public clouds face challenges in achieving file immutability due to the complexity of managing retention locks across shared deduplicated objects, leading to inefficiencies and increased storage costs, as well as vulnerabilities from frequent lock extensions and garbage collection cycles.
Innovation Solution
Implementing a system that locks new cloud-based data objects to a fixed future date for a protection duration, using cloud provider APIs to ensure compliance mode locking, and periodically extending locks or spawning new filesystems to replicate data, thereby reducing the need for frequent lock extensions and garbage collection operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deduplication filesystem locks data blocks corresponding to locked files, then file immutability is achieved, but the algorithm to map and lock data blocks becomes much harder and storage overhead increases
Solution Approach 1:
The patent segments the filesystem into multiple namespaces, with each namespace containing a subset of data blocks. This segmentation allows the system to lock only the specific namespace containing the locked file's data blocks, rather than managing locks across the entire filesystem. The segmentation reduces the complexity of the locking algorithm by limiting the scope of block mapping and locking operations to a manageable subset of the total data blocks.
2Duration of action of stationary object
If the filesystem writes deduplicated block sized objects and maintains reference counts, then object lock extension is possible, but storage overhead and system complexity increase
Solution Approach 1:
The patent extracts the reference counting mechanism from the main filesystem operations and implements it as a separate, optimized component. By taking out the reference counting function and implementing it independently with specialized data structures, the system can track object references more efficiently without adding significant overhead to the main filesystem operations. This extraction allows for better management of lock retention periods while minimizing the impact on storage overhead.
3Reliability
If frequent lock extensions and garbage collection cycles are performed, then data protection is maintained, but storage costs increase and system performance decreases
Solution Approach 1:
The patent implements preliminary action by pre-allocating namespace resources and pre-establishing lock retention policies before they are actually needed. Namespaces are created and configured in advance with their capacity and retention parameters, eliminating the need for frequent dynamic adjustments during operation. Lock extensions are planned and executed based on predetermined policies rather than reactive frequent operations, reducing the performance impact of these maintenance activities.
Data Source
AI summary
Embodiments for providing file immutability for cloud storage data in a deduplicating filesystem and using a new filesystem that is spawned to receive redirected live data, after which the old filesystem is expired. Data objects are stored in the cloud by defining a protection duration from a first date to a fixed future date, and applying a retention lock to one or more data objects stored in cloud storage during the protection duration, the retention lock preventing unauthorized deletion, modification or movement of the data. A renew threshold date is defined within the protection duration, and a new filesystem is spawned on this date. All new data is redirected to the new filesystem and locked for the protection duration, and the first file system is destroyed.