Cloud Data Replication via Metadata Stream for Any Point in Time Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data replication methods, such as snapshot-based systems and tier-1 mirroring, have unsatisfactory recovery point objectives (RPO) and require significant compute resources, limiting the ability to restore data to any point in time and increasing storage and processing demands.
Innovation Solution
A system that replicates production data to the cloud using a metadata stream, allowing continuous replication without maintaining a replica volume, and only requiring compute resources during restore operations, achieving near-zero RPO by storing data as objects in a data bucket and generating a metadata stream that stores relationships between the replicated and production data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If snapshot-based replication is used to store data in the cloud, then storage costs are reduced, but the recovery point objective (RPO) is limited and restores can only be performed to the most recent snapshot
Solution Approach 1:
The patent segments data into discrete objects stored in a data bucket, with a separate metadata stream tracking each object's version and timestamp. This segmentation allows selective restoration of individual objects to any point in time without requiring full snapshot storage, achieving precise RPO with reduced storage.
Solution Approach 2:
The patent introduces a metadata stream as an intermediary between the stored data objects and the restoration process. This metadata stream contains version information and timestamps that enable precise point-in-time recovery without maintaining full replica volumes, resolving the contradiction between storage efficiency and recovery precision.
2Measurement precision
If tier-1 mirroring is used to replicate data, then any point in time restoration is enabled, but compute resources are required continuously to maintain the replica volume
Solution Approach 1:
The patent extracts the compute-intensive replication maintenance function from the cloud environment and relocates it to the source system. Only metadata is transmitted to the cloud, while actual data replication occurs locally, eliminating continuous compute resource requirements in the cloud while maintaining any-point-in-time restoration capability.
Solution Approach 2:
The source system performs self-service replication by maintaining the replica volume locally and only transmitting metadata changes to the cloud. This self-service approach eliminates the need for cloud-based compute resources to maintain replicas, reducing energy consumption while preserving full restoration capability.
3Reliability
If conventional mirroring is used to maintain replica volume in the cloud, then data restoration capability is improved, but storage requirements and device complexity increase
Solution Approach 1:
The patent extracts only the essential metadata from the full replication system, storing minimal information (object identifiers, versions, timestamps) in the cloud while maintaining full restoration capability. This extraction approach dramatically reduces system complexity and storage requirements compared to conventional mirroring.
Solution Approach 2:
The patent creates a lightweight metadata copy in the cloud that references the source data, rather than maintaining full data copies. This copying strategy enables data restoration through metadata-driven reconstruction while minimizing cloud storage requirements and system complexity.
Data Source
AI summary
Systems, apparatus, and methods for any point in time replication to the cloud. Data is replicated by replicating data to a remote storage or a data bucket in the cloud. At the same time, a metadata stream is generated and stored. The metadata stream establishes a relationship between the data and offsets of the data in the production volume. This allows continuous replication without having to maintain a replica volume. The replica volume can be generated during a rehydration operation that uses the metadata stream to construct the production volume from the cloud data.