Snapshot Physical Size API for Shared Object-Store Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Determining the physical size of snapshots stored across multiple objects in an object store is challenging due to incremental nature and shared data, making it difficult to accurately calculate the size of a specific snapshot.
Innovation Solution
A physical size API is implemented to determine the actual physical size of snapshots by using tracking objects and metadata requests to identify unique snapshot data, with serverless containers for efficient processing and checkpointing to handle crashes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If snapshots are backed up to object store using incremental storage, then storage cost efficiency is improved, but determining the physical size of snapshots becomes difficult
Solution Approach 1:
The patent introduces tracking objects as intermediary data structures that map snapshot logical blocks to physical object store locations. These tracking objects enable the system to determine the actual physical size of snapshots by aggregating unique object references, resolving the contradiction between incremental storage efficiency and accurate size measurement.
Solution Approach 2:
The patent segments snapshot data into discrete objects within the object store, where each object represents a specific data segment. By tracking which objects are referenced by which snapshots and calculating unique object contributions, the system can accurately determine snapshot physical sizes while maintaining incremental storage benefits.
2Quantity of substance
If multiple snapshots share common data objects, then storage space is optimized, but calculating unique snapshot size becomes complex
Solution Approach 1:
Tracking objects serve as intermediaries that record the relationship between snapshots and their constituent data objects. By maintaining these tracking structures, the system can efficiently calculate unique snapshot sizes by comparing object references between snapshots, avoiding the need for complex direct analysis of shared data relationships.
Solution Approach 2:
Instead of physically copying shared data objects between snapshots, the patent uses references and tracking metadata to represent snapshot contents. This allows multiple snapshots to share underlying objects while maintaining accurate size calculations through metadata aggregation, reducing both storage overhead and calculation complexity.
Data Source
AI summary
Techniques are provided for determining a physical size of a snapshot backed up to an object store. Snapshot data of the snapshot may be backed up into objects that are stored from a node to the object store, such as a cloud computing environment. A tracking object is created to identify which objects within the object store comprise the snapshot data of the snapshot. In order to determine the physical size of the snapshot, the tracking object and/or tracking objects of other snapshots such as a prior snapshot are evaluated to identify a set of objects comprising snapshot data unique to the snapshot and not shared with the prior snapshot. The physical sizes of the set of objects are combined with a metadata size of metadata of the snapshot to determine the physical size of the snapshot.