Bucket Snapshot Filtering for Granular Object Restoration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional object storage products do not provide native support for bucket-level snapshots, placing the burden of backup and data recovery on the client application, and existing snapshot restoration methods are not instantaneous and cannot recover deleted object versions.
Innovation Solution
A storage system that natively creates bucket-level snapshots by maintaining a snapshot metafile with snapshot identifiers and time indicators, allowing instantaneous restoration of previous object versions and protecting deleted versions through hidden internal versions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional object storage products are used, then client applications must traverse all objects in a bucket to catalog object versions for backup and recovery, but this places the burden on the client application and does not provide native bucket-level snapshot support
Solution Approach 1:
The storage system performs backup and recovery operations autonomously by maintaining native bucket-level snapshots. The system automatically catalogs object versions and enables restoration without requiring client applications to traverse all objects manually, thereby reducing client complexity while improving ease of operation.
Solution Approach 2:
The storage system introduces an intermediary mechanism (native snapshot infrastructure) between the client application and the object storage bucket. This intermediary handles the complex tasks of cataloging object versions and enabling restoration, allowing client applications to simply create and restore snapshots without dealing with the underlying complexity.
2Reliability
If existing snapshot restoration methods are used, then previous object versions can be restored, but the restoration is not instantaneous and deleted object versions cannot be recovered
Solution Approach 1:
The storage system performs preliminary action by continuously maintaining bucket-level snapshots that capture the state of all objects at any given time. This preliminary snapshot creation enables instantaneous restoration, as the system already has the complete state information stored and ready for immediate recovery, including deleted object versions.
Solution Approach 2:
The storage system creates a copy of the entire bucket state at any point in time through bucket-level snapshots. This copy includes all object versions, even deleted ones, allowing instantaneous restoration of the complete state without needing to traverse or reconstruct individual objects, thereby achieving both completeness and speed.
3Productivity
If bucket-level snapshots are implemented, then instantaneous restoration and complete data recovery are enabled, but the storage system must maintain snapshot metafiles and hidden internal versions
Solution Approach 1:
The storage system implements a universal bucket-level snapshot mechanism that serves multiple functions: it enables instantaneous restoration, recovers deleted object versions, and provides complete data protection. By using a single unified approach rather than individual object-level operations, the system achieves high productivity while managing complexity through a consistent multi-functional infrastructure.
Data Source
AI summary
Systems and methods for supporting granular snapshots are provided. In one example, a storage system may limit a scope of an operation relating to a snapshot of a bucket by applying a snapshot filter associated with the snapshot in which the snapshot filter specifies one or more criteria for determining a subset of multiple objects of a bucket to which the snapshot applies. In one embodiment, the snapshot filer may represent a prefix specified as part of the operation and application of the snapshot filter may involve filtering the multiple objects based on the prefix. The operation may involve creation of a snapshot, enumeration of objects protected by the snapshot, deletion of the snapshot, or restoration of the snapshot. The association of the snapshot filter with the snapshot may be accomplished by persisting the snapshot filter to a snapshot metafile within a snapshot entry corresponding to the snapshot.