Relational Database Slave Replay Parallelism
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Relational databases face challenges in maintaining data integrity and reducing latency during slave database replay, especially under high write loads, where low replay throughput can cause the slave database to lag behind the master database.
Innovation Solution
A method and system for replaying transactions on a slave database by identifying rows with unique sets of values and replaying them in parallel, while replaying conflicting events serially, to improve throughput and reduce latency while preserving atomicity and data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If events are replayed serially on the slave database, then data integrity is maintained, but replay throughput is low and latency increases
Solution Approach 1:
The patent segments the replay process into two distinct modes: serial replay for events affecting the same row to maintain data integrity, and parallel replay for events affecting different rows to improve throughput. The replay system analyzes each event's target row and dynamically selects the appropriate replay mode, thus resolving the contradiction between reliability and productivity.
Solution Approach 2:
The patent implements a dynamic replay mechanism that adapts the replay mode based on the characteristics of each event. The system continuously monitors row identifiers and transaction states, switching between serial and parallel replay modes in real-time. This dynamic approach allows the system to optimize replay throughput while maintaining data integrity where required.
2Productivity
If parallel replay is used to improve throughput, then latency is reduced, but data integrity may be compromised
Solution Approach 1:
The patent segments the replay operations based on row identifiers, creating isolated replay streams for different rows. Events targeting the same row are replayed serially within their specific stream, while events targeting different rows can be replayed in parallel across different streams. This segmentation ensures data integrity is maintained for each row while enabling parallel processing across multiple rows.
Solution Approach 2:
The patent introduces a row identifier analysis mechanism as an intermediary that determines the appropriate replay mode for each event. This intermediary analyzes the row identifiers of incoming events and routes them to appropriate replay queues, ensuring that events affecting the same row are processed serially while events affecting different rows can proceed in parallel, thus maintaining integrity without sacrificing throughput.
3Reliability
If the slave database replays all events serially, then atomicity is preserved, but the slave database lags behind the master database under high write loads
Solution Approach 1:
The patent segments the replay process into multiple parallel streams based on row identifiers, allowing independent replay of events affecting different rows. This segmentation enables the slave database to process multiple events simultaneously without compromising atomicity within each row's event stream, thereby reducing overall replay latency while maintaining the required reliability guarantees.
Solution Approach 2:
The patent implements a dynamic replay system that adjusts the degree of parallelization based on the characteristics of incoming events and the current state of the slave database. Under high write loads, the system dynamically increases parallel replay where safe, while maintaining serial replay where atomicity requirements demand it, thus optimizing the balance between preserving atomicity and reducing replay latency.
Data Source
AI summary
Systems and methods for replication replay in a relational database are disclosed. In one embodiment, a relational database includes a master database and a slave database, and events performed on the master database are stored in a log. A portion of the log is provided to a slave replay system associated with the slave database, and the slave replay system identifies a first table row associated with a first event of the portion of the log and a second table row associated with a second event in the portion of the log. The slave replay system replays the first and second events on the slave database in parallel if the first and second rows are different rows with unique sets of values, and otherwise replays the first and second events serially.