Multi-path Database Replication for High Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional database replication architectures using a single replication path struggle with high throughput at primary databases, leading to increased replication latency and data accumulation, as they cannot match data generation/transaction speeds and lack the ability to prioritize transactions effectively.
Innovation Solution
Implementing a multi-path database replication system that instantiates multiple replicate connections and transaction streams, allowing transactions to be applied in parallel and prioritized based on their replication priority, with data bound to specific replication paths to distribute the replication load across multiple servers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single replication path is used to maintain transaction commit order, then transaction integrity is preserved, but replication latency increases significantly during high throughput periods
Solution Approach 1:
The patent segments the single replication path into multiple parallel replication paths (stream 1, stream 2, etc.), allowing transactions to be replicated simultaneously through different paths. This segmentation enables high throughput while maintaining integrity through controlled parallel processing of transactions across multiple streams.
Solution Approach 2:
The patent implements dynamic transaction routing where transactions are assigned to different replication paths based on their characteristics and priorities. The system dynamically adjusts which path a transaction takes depending on current load conditions and transaction attributes, optimizing replication performance while maintaining order where required.
2Stability of the object's composition
If transactions are replicated serially in commit order, then transaction order is maintained, but data accumulation and latency increase during high throughput activity
Solution Approach 1:
The patent divides the sequential transaction stream into multiple parallel streams, allowing different transactions to be replicated simultaneously. This segmentation maintains stability for transactions requiring order while dramatically improving productivity through parallel processing of independent transactions.
Solution Approach 2:
The patent transitions from single-dimensional sequential replication to multi-dimensional parallel replication by introducing additional replication paths. This dimensional change allows transactions to be processed in multiple directions simultaneously, increasing throughput while maintaining order through proper stream management.
3Device complexity
If all transactions are replicated through a single path, then system complexity is low, but the system cannot prioritize transactions with different replication priorities
Solution Approach 1:
The patent segments the replication architecture into multiple paths with distinct characteristics, enabling different transaction types to be routed to appropriate paths. This segmentation provides adaptability for prioritization while keeping individual path complexity manageable through modular design.
Solution Approach 2:
The patent applies local quality by assigning different replication paths with different qualities (priorities, latency characteristics) to different transaction types. Critical transactions use high-priority paths while non-critical transactions use standard paths, enabling fine-grained control without system-wide complexity.
4Device complexity
If a single replication server handles all transactions, then system simplicity is maintained, but replication performance degrades during high throughput periods
Solution Approach 1:
The patent segments the replication workload across multiple servers, with each server handling a specific replication stream. This segmentation increases throughput by distributing processing load while maintaining simplicity through standardized server templates that can be deployed and managed uniformly.
Data Source
AI summary
Systems, methods and computer program products for multi-path replication in databases are described herein. A non-limiting embodiment includes instantiating a plurality of replicate connections to a replicate database, providing transactions to the replicate database through a plurality of transaction streams, where the transaction streams associated with respective instantiated replicate connections. Transactions occurring in distinct transaction streams are applied, in parallel, at the replicate database. Another non-limiting embodiment includes instantiating a plurality of replication sender threads, where the replication sender threads communicate in parallel with respective replication servers over respective replication paths. Data sharing one or more attributes can be bound to a replication path in communication with the respective replication servers.