Change-Aware Snapshot Replication for Fresh Data Without Unnecessary Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Snapshot replication in data replication systems often results in outdated data at the target system due to infrequent updates, leading to unnecessary computing resource consumption and inefficient data management.
Innovation Solution
Implement a process to check for data changes in the source system before performing a snapshot update by comparing summary data or metadata of the replica with the remote system, triggering an update only when changes are detected.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If snapshot replication is performed periodically to keep target data updated, then data freshness is improved, but computing resource consumption increases and update efficiency deteriorates
Solution Approach 1:
The patent applies preliminary action by checking for data changes before performing snapshot replication. The system compares metadata or generates digests of source and target data objects beforehand to determine if replication is necessary, avoiding unnecessary full data transfers and improving update efficiency while maintaining data freshness
Solution Approach 2:
The patent implements feedback mechanisms by continuously monitoring and comparing data change indicators (metadata, digests) between source and target systems. This feedback loop enables the system to make informed decisions about when replication is needed, optimizing the balance between data freshness and resource consumption
2Reliability
If snapshot replication is performed frequently to maintain data synchronization, then data freshness is improved, but computing overhead and resource consumption increase
Solution Approach 1:
The patent extracts and compares only critical change indicators (metadata, digests) rather than transferring or comparing entire data objects. This selective extraction approach maintains synchronization reliability while significantly reducing computing overhead and resource consumption
Solution Approach 2:
The patent changes the parameter being monitored from complete data content to simplified representations (metadata, digests). This parameter transformation enables efficient change detection with minimal computing resources while maintaining the ability to detect all actual data changes
3Reliability
If all data contents are transferred during snapshot update to ensure completeness, then data completeness is improved, but transfer time and network resources increase
Solution Approach 1:
The patent applies partial action by transferring only the data changes that are necessary rather than performing complete full snapshots. By using change detection mechanisms, the system transfers minimal data while ensuring completeness of the replicated information, reducing transfer time and network resource consumption
Data Source
AI summary
Disclosed techniques and solutions can provide improved snapshot replication. Typically, an initial replica obtained using snapshot replication is periodically updated. However, the update process can unnecessarily consume computing resources if data in a source data object has not changed with respect to data in a replica data object. Disclosed techniques check to determine whether a snapshot replica is out of date before obtaining a new snapshot. The checks can be performed on manual request or on the occurrence of triggers, such as receiving a query that accesses the replica data object or according to a schedule. Information for current and prior versions of the remote data object can be compared to determine whether a replica is out of date, such as digest values of contents of the remote data object or timestamps associated with the remote data object.