Distributed Database Partition Replica Bootstrapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current distributed database systems require extensive processing and time to build a replica partition by re-executing the entire changelog, which is inefficient and data-intensive when replacing a failed active or replica partition.
Innovation Solution
A method is introduced where a new partition copy is generated with an empty database, and the database from an existing partition copy is copied to it, designating the new partition as either active or replica based on the required replica scheme, thereby reducing data usage and speeding up the replica creation process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a new replica partition is built by re-executing the entire changelog, then the replica partition can be completely reconstructed, but the processing time and data requirements become extensive and inefficient
Solution Approach 1:
The patent extracts only the necessary subset of changelog events required to reconstruct the replica partition state, rather than processing the entire changelog history. This is achieved by identifying and applying only those events that occurred after the source partition's last checkpoint, significantly reducing processing time while maintaining replica completeness.
Solution Approach 2:
The system performs preliminary actions by maintaining checkpoints in the changelog that mark specific states of the source partition. These pre-established checkpoints allow the replica creation process to quickly determine where to start applying events, avoiding the need to re-process the entire changelog from the beginning and thus reducing creation time.
2Reliability
If the entire changelog is maintained and re-executed, then complete data reconstruction is achieved, but extensive data storage and processing resources are consumed
Solution Approach 1:
The patent extracts and retains only the essential subset of changelog data needed for replica reconstruction - specifically, events occurring after the source partition's checkpoint. This extraction approach maintains data reconstruction accuracy while significantly reducing the quantity of changelog data that must be stored and processed.
Solution Approach 2:
The system changes the parameter of data retention by transitioning from maintaining the entire changelog history to maintaining only the post-checkpoint events. This parameter change in data scope reduces storage requirements while preserving the essential information needed for accurate replica reconstruction.
3Productivity
If a new partition copy is created with empty database and copied from existing partition, then replica creation is accelerated and data usage is reduced, but the system must manage partition failure detection and recovery
Solution Approach 1:
The system implements self-service mechanisms where the distributed database automatically detects partition failures, identifies suitable source partitions for replication, and initiates the replica creation process without manual intervention. The automated failure detection and recovery management reduces the operational complexity despite the accelerated replica creation process.
Solution Approach 2:
The system employs feedback mechanisms to monitor partition health and status, automatically triggering replica creation when failures are detected. This feedback loop manages the complexity of failure detection and recovery by systematically responding to partition events, enabling fast replica creation while maintaining system reliability.
Data Source
AI summary
A system and method of bootstrapping replicas from existing partitions is disclosed. A distributed database is maintained by a system. The distributed database includes a first partition copy and a second partition copy each containing a database. A failure of the first partition copy is detected and a third partition copy including an empty database is generated. The database of the second partition copy is copied to the empty database of the third partition copy and the third partition copy is designated as one of an active partition copy or a replica partition copy.