Multi-Volume Durable Log Tracking for Faster Database Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional database systems face I/O bottlenecks due to network traffic between the database processing layer and storage layer, especially in distributed cloud environments, leading to performance degradation and scalability issues.
Innovation Solution
Implementing a distributed database system that utilizes split volume types, where redo log data is stored in one set of volumes and change log data in another, allowing asynchronous writes and parallel processing, reducing network traffic and enabling faster recovery and scalability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If traditional database systems use a single volume for both redo log and change log data, then device complexity is reduced, but network traffic increases and performance degrades due to I/O bottlenecks
Solution Approach 1:
The patent divides the storage volume into separate redo log volumes and change log volumes. This segmentation allows independent management and optimization of each log type, reducing network traffic by allowing parallel asynchronous writes to different volumes, thereby improving database performance while maintaining manageable complexity through clear separation of concerns
2Productivity
If asynchronous writes are implemented across multiple volumes, then network traffic is reduced and performance improves, but device complexity increases due to multi-volume management
Solution Approach 1:
The system performs preliminary actions by pre-configuring multiple volumes with appropriate characteristics (redo log volumes for durability, change log volumes for scalability) before write operations begin. This allows the database to immediately utilize parallel asynchronous writes without runtime complexity, as the volume architecture is already optimized for the intended workload patterns
3Reliability
If redo log and change log are stored separately, then failure recovery time is reduced and data loss is minimized, but system complexity increases
Solution Approach 1:
The patent extracts change log data from the traditional redo log volume and stores it in separate change log volumes. This extraction enables independent recovery processes where the database can recover from redo logs without being blocked by change log availability, significantly reducing failure recovery time and minimizing data loss while maintaining manageable complexity through standardized extraction patterns
Data Source
AI summary
Techniques for volume durable log sequence number movement in a multi-volume database environment are described. Blocks of database updates are persisted in an atomic and durable manner, where the blocks may include updates to a first volume, a second volume, or both. Backlinks from one block of updates to the previous block of updates may be set, in addition to backlinks between updates to the first volume and backlinks between updates to the second volume. Upon durably persisting a block, the block backlinks can be followed to help verify that all contiguous blocks—of changes to one or both volumes—have all been persisted. Thereafter, a volume durable log sequence number (VDL) pointer value can be updated to point to a known “last” durably persisted log update.