Partitioned Change Logs for Reliable Point-in-Time Restoration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database partitioning leads to inefficiencies in tracking changes over time, as change logs for split partitions remain dependent on pre-split logs, complicating point-in-time restoration.
Innovation Solution
A system that splits change logs into child logs upon partition splitting, deletes log portions after complete snapshots are stored for each child partition, and manages dependencies using reference counters and retention windows.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If change logs are retained for all database partitions including split partitions, then point-in-time restoration reliability is maintained, but storage space consumption increases
Solution Approach 1:
The patent segments change logs by creating child change logs for each child database partition when a partition split occurs. The parent change log is divided into multiple child change logs that are stored separately, allowing selective retention based on restoration needs for specific partitions without maintaining redundant copies for all partitions.
Solution Approach 2:
The patent enables discarding of parent change logs after child change logs are created and properly linked. The system tracks dependencies using reference counters that are decremented when child partitions are restored, allowing safe deletion of parent change logs once they are no longer needed for restoration, thus recovering storage space while maintaining restoration capability.
2Quantity of substance
If parent change logs are deleted after partition splitting, then storage efficiency improves, but restoration reliability deteriorates
Solution Approach 1:
The patent implements a reference counter mechanism that provides feedback on the dependency status of change logs. The counter tracks how many child partitions depend on a parent change log, and restoration operations decrement these counters. This feedback system ensures parent change logs are only deleted when no longer needed, maintaining restoration reliability while enabling storage optimization.
Solution Approach 2:
The patent introduces child change logs as intermediaries between the parent change log and child database partitions. These child change logs capture the split point and enable restoration of child partitions without requiring the parent change log to be retained, acting as a mediator that breaks the direct dependency chain and enables safe deletion of parent logs.
3Device complexity
If change logs are not split when database partitions are split, then device complexity is reduced, but tracking precision deteriorates
Solution Approach 1:
The patent segments the change log into child change logs that correspond to child database partitions. Each child change log contains only the changes relevant to its specific partition, providing precise tracking for each partition while maintaining a simple overall structure through automated segmentation triggered by partition split events.
Data Source
AI summary
Various systems and methods of tracking database partition change log dependencies are described herein. A system is disclosed herein, according to some embodiments. A change log for a database partition of the plurality of database partitions is stored to a log store. An indication that a split of the database partition into two or more child database partitions has occurred is received. Responsive to the split of the database partition, the change log is split into two or more corresponding child change logs. A portion of the change log is deleted responsive to a determination that a respective complete snapshot is stored for each child database partition corresponding to a child change log of the change log and a determination that a retention window for the portion of the change log has passed.