Transactional Consistent Backup of Partitioned Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed systems face challenges in maintaining transactional consistency during backups and restores across partitioned storage, leading to potential inconsistencies and errors due to differences in backup times and transaction sequences across partitions.
Innovation Solution
The implementation of an optimistic marking process during backups, followed by log repair to ensure transactional consistency, where markers are adjusted to maintain the integrity of transactional boundaries across partitions, allowing for transactionally consistent restores without requiring a write outage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If backups are performed across partitioned storage without write outage, then availability is improved, but transactional consistency may be violated
Solution Approach 1:
The system performs preliminary actions by writing all backup data to partition-specific archive logs and placing optimistic markers before verifying transactional consistency. This allows the backup process to proceed without write outages, improving availability while maintaining a mechanism to ensure consistency afterward through marker validation and log repair operations.
2Productivity
If optimistic marking is used during backup, then backup performance is improved, but transactional boundaries may be violated
Solution Approach 1:
The system uses feedback by validating transactional boundaries after backup completion. Optimistic markers are placed during backup without blocking performance, then the system checks whether these markers violate transactional boundaries by examining the sequence of changes in archive logs. If violations are detected, log repair operations adjust marker positions to maintain boundary integrity.
3Reliability
If markers are moved to repair logs, then transactional consistency is improved, but additional processing time is required
Solution Approach 1:
The system performs log repair operations as a preliminary action before restore operations. By detecting and repairing transactional boundary violations in archive logs before restoration begins, the system ensures transactional consistency is established upfront, preventing potential restore failures and reducing overall processing time despite the additional marker adjustment step.
Data Source
AI summary
Methods, systems, and computer-readable media for transactionally consistent backup of partitioned storage are disclosed. Backup operations are initiated for a first and second partition of one or more data stores. The position of a first marker is determined in a first log associated with the first partition. The first log indicates a sequence of changes for the first partition. The position of a second marker is determined in a second log associated with the second partition. The second log indicates a sequence of changes for the second partition. The first or second marker is determined to violate a transactional boundary across the first and second partitions. A position of at least one of the markers is modified such that the markers do not violate the transactional boundary.