Snapshot Physical Size Tracking for Incremental Object Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Determining the physical size of snapshots stored across multiple objects in an object store is challenging due to incremental nature and shared data, making it difficult to accurately calculate the size of a snapshot in relation to prior snapshots, especially when snapshots are deleted or compressed.
Innovation Solution
A physical size API is implemented to determine the actual physical size of snapshots by using tracking objects and metadata requests to identify unique snapshot data, with serverless containers for stateless processing and checkpointing to handle crashes, and storing results in object tags for persistent access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If snapshots are backed up to object store using incremental backup, then storage cost is reduced and scalability is improved, but determining the physical size of snapshots becomes difficult and inaccurate
Solution Approach 1:
The patent introduces tracking objects as intermediary data structures that map snapshot logical blocks to physical object store locations. These tracking objects enable accurate physical size calculation by providing a clear mapping relationship between logical snapshot data and actual stored objects, resolving the measurement accuracy problem while maintaining incremental backup efficiency
Solution Approach 2:
The patent segments snapshot data into discrete objects in the object store, with each object representing a specific data range. By dividing the snapshot into manageable object segments and tracking them individually, the system can accurately calculate the physical size of each snapshot by summing the sizes of its constituent objects, rather than treating the snapshot as a monolithic entity
2Reliability
If multiple snapshots are stored in the object store, then more historical data is preserved, but storage consumption increases significantly
Solution Approach 1:
The patent implements snapshot deletion by identifying and removing only the objects that are unique to the deleted snapshot, rather than deleting entire snapshot copies. The system uses tracking objects to identify which objects can be safely discarded when a snapshot is deleted, enabling efficient storage reclamation while preserving other snapshots that may share common data blocks
Solution Approach 2:
The patent merges multiple snapshots by allowing them to share common data blocks in the object store. Instead of storing duplicate copies of identical data across multiple snapshots, the system uses tracking objects to reference shared objects, reducing overall storage consumption while maintaining the integrity of each snapshot's data set
3Measurement precision
If tracking objects are used to determine physical size, then measurement accuracy is improved, but system complexity increases
Solution Approach 1:
The patent creates simplified tracking objects that contain only the essential mapping information between logical blocks and physical objects, rather than maintaining complex metadata structures. Each tracking object is a straightforward data structure that lists object references and their sizes, making the system easier to implement and manage while still providing accurate physical size measurement capabilities
Data Source
AI summary
Techniques are provided for determining a physical size of a snapshot backed up to an object store. Snapshot data of the snapshot may be backed up into objects that are stored from a node to the object store, such as a cloud computing environment. A tracking object is created to identify which objects within the object store comprise the snapshot data of the snapshot. In order to determine the physical size of the snapshot, the tracking object and/or tracking objects of other snapshots such as a prior snapshot are evaluated to identify a set of objects comprising snapshot data unique to the snapshot and not shared with the prior snapshot. The physical sizes of the set of objects are combined with a metadata size of metadata of the snapshot to determine the physical size of the snapshot.