Parallel Transaction Log Grouping for Database Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database replication solutions suffer from low efficiency due to the need for serial processing of transaction logs, which is constrained by dependency relationships between logs.
Innovation Solution
A database replication system that allows parallel processing of transaction logs by grouping them based on generation time points and recording dependency relationships within each group, enabling independent replay at the destination end.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If transaction logs are processed serially to maintain dependency relationships, then data consistency is ensured, but replication efficiency deteriorates
Solution Approach 1:
The patent segments transaction logs into multiple groups based on their dependency relationships. Each group can be processed independently in parallel, while maintaining the internal sequential order within each group. This segmentation allows the system to break the single serial processing chain into multiple parallel processing streams, thereby improving replication efficiency while ensuring data consistency within each segment.
Solution Approach 2:
The patent implements dynamic processing by allowing different groups of transaction logs to be processed in parallel based on their dependency characteristics. The system dynamically determines which groups can be processed simultaneously and which must be processed sequentially, optimizing the replication process adaptively rather than using a fixed serial or parallel approach.
2Productivity
If parallel processing is implemented to improve replication efficiency, then productivity is improved, but processing complexity increases
Solution Approach 1:
By segmenting transaction logs into distinct groups with clear boundaries based on dependency relationships, the patent simplifies the parallel processing management. Each group can be handled by independent processing threads or processes, reducing the complexity of coordinating and managing parallel operations compared to a fully interconnected parallel processing system.
Solution Approach 2:
The patent performs preliminary analysis and grouping of transaction logs before parallel processing begins. By pre-identifying dependency relationships and organizing logs into processable groups in advance, the system reduces the complexity of real-time dependency management during parallel execution, making the overall processing more manageable and efficient.
Data Source
AI summary
In a database replication operation, a source end device obtains at least two groups of transaction logs from a log file of a source end database in parallel, where the at least two groups of transaction logs include a first group of transaction logs and a second group of transaction logs. The first group of transaction logs includes at least a first transaction log and a second transaction log that are adjacent to each other, and the second group of transaction logs includes at least a third transaction log and a fourth transaction log that are adjacent to each other, and a generation time point of the second transaction log is earlier than a generation time point of the third transaction log. The source end device then sends the at least two groups of transaction logs to a destination end device.