Any Point in Time Snapshots via Metadata Volume
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems, particularly those using VSAN, often store data only representing the latest point in time, limiting the ability to create snapshots in hindsight and thus preventing any point in time functionality.
Innovation Solution
Incorporating a metadata volume or representation into UPiT snapshots, which stores a metadata stream containing a record of all inputs and outputs, allowing snapshots to be generated for any point in time by combining this metadata with existing snapshots.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in blobs representing only the latest point in time, then storage efficiency is improved, but the ability to create snapshots in hindsight is lost
Solution Approach 1:
The patent segments data storage into two distinct components: blobs that store the latest point in time data efficiently, and a metadata stream that records historical information. This segmentation allows each component to serve its specific function optimally while together they enable both storage efficiency and snapshot capability.
Solution Approach 2:
The metadata stream acts as an intermediary between the blob storage and the snapshot functionality. It mediates by recording all input/output operations and their associated metadata, which can then be used to reconstruct any point in time snapshots without affecting the efficient blob-based storage structure.
2Device complexity
If snapshots are created only for specific points in time, then system complexity is reduced, but any point in time functionality is unavailable
Solution Approach 1:
The system performs preliminary action by continuously recording the metadata stream of all input/output operations as they occur. This ongoing recording prepares the necessary information in advance, enabling any point in time snapshots to be created later without complex real-time processing when the snapshot is actually needed.
Solution Approach 2:
The patent uses copying by creating snapshots as copies of data at specific points in time, reconstructed by combining the blob data with the relevant portion of the metadata stream. This copying approach allows multiple snapshots to be created from the same underlying data without duplicating the entire storage structure.
3Adaptability or versatility
If metadata stream is recorded for all inputs and outputs, then any point in time functionality is enabled, but data processing overhead increases
Solution Approach 1:
The patent extracts only the necessary metadata from input/output operations into the metadata stream, rather than copying entire data sets. This extraction approach enables any point in time functionality by recording essential information (identifiers, offsets, timestamps) while minimizing the processing overhead associated with handling large volumes of data.
Solution Approach 2:
The system performs partial action by recording only the metadata portion of input/output operations rather than the complete data. This partial recording provides sufficient information for snapshot reconstruction while avoiding the excessive processing burden of duplicating all data movements.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Systems and methods for providing any point in time functionality. With a storage system such as a VSAN, any point in time protection is achieved by combining a metadata stream with snapshots of the storage system. This allows snapshots to be generated in hindsight such that any point in time functionality is provided.