Multi-Volume Database Recovery Using Logical Replay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In modern distributed cloud services, the I/O bottleneck has shifted from individual disks and nodes to network traffic between the database processing layer and storage layer, leading to performance issues and scalability challenges, particularly when handling redo logs and change logs.
Innovation Solution
Implementing a distributed database system that segregates redo log data into one set of volumes and change log data into another, utilizing efficient distributed storage, allowing for reduced network traffic and improved failure recovery through asynchronous log processing and replication across multiple storage nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If redo logs and change logs are stored together in the same volume, then storage management is simplified, but network traffic increases and recovery time increases
Solution Approach 1:
The patent segments log storage by creating separate volumes for redo logs and change logs. The redo log volume stores transaction logs while the change log volume stores data modifications. This segmentation reduces network traffic during recovery operations since only the necessary log volume needs to be accessed, and allows parallel processing of recovery operations without contention.
2Reliability
If logs are processed synchronously to ensure data consistency, then data integrity is maintained, but recovery time increases
Solution Approach 1:
The patent implements preliminary actions by pre-processing and pre-positioning logs in separate volumes during normal operation. Change logs are captured and stored independently before recovery is needed. During recovery, this preliminary preparation allows the system to immediately apply pre-positioned change logs without waiting for synchronous processing, significantly reducing recovery time while maintaining data consistency through the use of commit points and transaction boundaries.
3Reliability
If database instances are replicated across multiple regions for high availability, then system reliability improves, but network traffic and coordination overhead increase
Solution Approach 1:
The patent segments the database system into independent database instances, each with its own redo log volume and change log volume. This segmentation allows each instance to operate independently with local log processing, reducing cross-region network traffic. Replication between regions becomes more efficient since only essential coordination data needs to be synchronized, not entire log volumes.
4Productivity
If parallel write operations are issued to storage fleet for performance, then throughput increases, but network bottleneck amplification occurs
Solution Approach 1:
The patent segments log write operations by directing redo logs and change logs to different storage volumes. This segmentation allows parallel write operations to proceed simultaneously to separate volumes without competing for the same network bandwidth, thereby maintaining high throughput while reducing network traffic amplification. Each volume can be managed independently with optimized I/O patterns.
Data Source
AI summary
Techniques for lossless database recovery using logical replay in a multi-volume database environment are described. Techniques for fast database recovery in a multi-volume database environment via transactional awareness are described. In the event of a failure associated with a first volume storing database page data, the first volume can be restored to a point in time and transactions occurring at/after that point in time can be identified and replayed via use of a second volume that tracks logical change data for the database.