Log Fragmentation Using Match Tree for Database Replay Concurrency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current log fragmentation methods in database systems lead to unbalanced distribution of data logs among fragments, resulting in low concurrency and inefficient log replay, especially when a large number of user connections generate a high volume of logs, causing delays in service continuity during master database server failures.
Innovation Solution
A method and apparatus for log fragmentation using a match tree structure that balances the distribution of data logs by determining a match tree based on data table identifiers, fragmenting logs according to this tree, and adjusting sub-match values to ensure a consistent range of data logs across fragments, thereby increasing concurrency and efficiency during log replay.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If log replay is performed sequentially on the slave database server, then the system implementation is simple, but the log replay speed is excessively low causing data logs to pile up
Solution Approach 1:
The patent divides the log replay process into multiple parallel segments by fragmenting incoming data logs into multiple fragments and assigning them to different replay threads. This segmentation enables concurrent log replay, significantly improving log replay speed and preventing data log accumulation while maintaining manageable system complexity through structured thread management.
2Reliability
If the slave database server waits to finish log replay before providing services, then data consistency is maintained, but service continuity is affected when master database server goes down
Solution Approach 1:
The patent implements preliminary log fragmentation and parallel replay preparation before failover is needed. By pre-processing logs into fragments and maintaining ready-to-execute replay threads, the system can quickly switch to slave service provision when master failure occurs, minimizing service interruption while ensuring data consistency through controlled fragment replay.
3Productivity
If logs are fragmented into many small fragments, then concurrency is increased, but the distribution balance among fragments deteriorates
Solution Approach 1:
The patent implements dynamic fragment distribution mechanisms that adaptively balance log fragments across multiple replay threads. The system monitors fragment processing status and dynamically adjusts distribution to maintain balance, enabling high concurrency while preventing workload imbalance that would reduce overall replay efficiency.
Data Source
AI summary
A log fragmentation method is disclosed. A slave database server receives a first log sent by a master database server, wherein the first log includes a data table identifier and match identifier. The slave database server determines a match tree based on the data table identifier of the first log, and fragments the first log based on the match tree. Using a match tree that has been set up, a tree structure of the match tree can be effectively utilized to fragment a log, and the fragmented log has a high degree of concurrency, thus improving the efficiency of log replay.