Continuous Data Protection via Metadata Journal and Hash Reference Counting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional snapshot technologies provide coarse granularity, allowing users to restore volumes only to specific snapshot points in time, and store a limited number of snapshots, making it impossible to restore the exact state of a volume at any point in time between snapshots.
Innovation Solution
A continuous data protection (CDP) mechanism is implemented on a storage array, which includes a metadata journal and hash reference count table, allowing for efficient native CDP by incrementing hash reference counts for each I/O operation, generating periodic snapshots, and adding metadata with timestamps, enabling restoration to any point in time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional snapshot technology is used, then storage space is conserved, but the ability to restore to any point in time is lost
Solution Approach 1:
The patent segments the continuous data protection functionality into two parts: (1) periodic snapshots that provide coarse-grained restore points, and (2) a metadata journal that records fine-grained I/O operations between snapshots. This segmentation allows the system to achieve fine-grained restore capability without storing complete volume images at every point in time, thus conserving storage space while enabling precise restoration.
Solution Approach 2:
The patent adds a temporal dimension to the metadata journal by recording timestamps with each I/O operation. This temporal dimension allows the system to reconstruct the volume state at any point in time by filtering and applying metadata entries based on their timestamps, enabling fine-grained restoration without requiring storage space for every possible restore point.
2Measurement precision
If a metadata journal is maintained for each I/O operation, then fine-grained restore capability is achieved, but system complexity increases
Solution Approach 1:
The metadata journal structure is designed to be universal and multi-functional. It serves multiple purposes: (1) enabling fine-grained restoration by recording all I/O operations with timestamps, (2) supporting snapshot creation by providing the data needed to reconstruct volume states, and (3) facilitating data deduplication through hash reference counting. This multi-functionality reduces the need for separate mechanisms, thereby managing system complexity while achieving fine-grained restore capability.
Solution Approach 2:
The metadata journal acts as an intermediary between the I/O operations and the snapshot/restoration processes. Instead of directly managing complex restore operations for every I/O change, the system uses the metadata journal as a mediator that records changes in a standardized format, which can then be processed efficiently during restoration. This intermediary layer simplifies the overall system architecture.
3Quantity of substance
If hash reference counting is implemented, then storage efficiency is improved, but processing overhead increases
Solution Approach 1:
The patent implements hash reference counting selectively rather than for all data operations. The mechanism is activated specifically when data is written to the volume, and the reference count is incremented only for the hash of the written data. This partial application of hash reference counting provides storage efficiency benefits for deduplication while minimizing the processing overhead associated with hash calculations and reference count management.
Data Source
AI summary
In one aspect, a method includes receiving an I/O to write data to a volume stored, increasing a hash reference count for a hash of the data in response to receiving the I/O, periodically generating snapshots of the volume, adding metadata on the I/O and a timestamp to a metadata journal and increasing the reference count value in response to adding the metadata.