Peer-to-Peer Data Replication via Commit Sequence Numbers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The single-master replication topology in telecom and enterprise applications faces a network bottleneck due to the master node having to replicate data twice, resulting in increased network load and potential performance issues.
Innovation Solution
Implementing a peer-to-peer data replication method where the master node generates commit redo records with a commit sequence number and distributes them to subscriber nodes, allowing each subscriber node to replicate data independently, thereby reducing the load on the master node and optimizing data distribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the master node replicates data to multiple subscriber nodes in a single-master topology, then data consistency is maintained, but the master node becomes a network bottleneck with increased network load
Solution Approach 1:
The patent segments the replication function by introducing commit sequence numbers (CSN) that allow different subscriber nodes to receive and process replication data independently. Each subscriber node processes a subset of the master node's write operations based on CSN ranges, dividing the replication workload across multiple nodes rather than concentrating it all at the master node.
Solution Approach 2:
The patent adds a temporal dimension to the replication process by using commit sequence numbers to order and distribute write operations. Instead of all nodes receiving all data simultaneously from the master, nodes process operations in sequence based on CSN, creating a time-based distribution pattern that reduces instantaneous network load on the master node.
2Reliability
If the master node handles all replication operations centrally, then data consistency is ensured, but the master node performance degrades due to excessive replication load
Solution Approach 1:
The patent segments the replication workload by assigning different CSN ranges to different subscriber nodes. The master node generates commit redo records with incremental CSN values, and each subscriber node processes a specific range of these records, dividing the master node's replication burden across multiple independent processing streams.
Solution Approach 2:
The patent implements preliminary action by having subscriber nodes pre-process and buffer commit redo records as they arrive from the master node. Nodes maintain queues of pending replication operations and process them asynchronously, allowing the master node to continue generating new commit records without waiting for subscriber nodes to complete processing, thus improving master node throughput.
3Reliability
If the master node replicates the same data to multiple subscriber nodes, then all nodes receive consistent data, but the network load increases proportionally with the number of subscribers
Solution Approach 1:
The patent merges the replication streams by having subscriber nodes share and reuse commit redo records. When one subscriber node receives a commit redo record from the master node, it can forward this record to other subscriber nodes, eliminating the need for the master node to send duplicate copies to each node individually. This combining approach significantly reduces overall network traffic while maintaining data consistency across all nodes.
Solution Approach 2:
The patent introduces subscriber nodes as intermediaries in the replication process. Instead of direct point-to-point replication from the master node to each subscriber, the system uses a mesh topology where subscriber nodes act as intermediate relays, forwarding commit redo records to their peers. This intermediary approach reduces the master node's network burden and optimizes overall network utilization.
Data Source
AI summary
Method, device, and system for peer-to-peer data replication are provided. The method includes: generating a first commit redo record attached with a CSN when a first write operation happens on a first physical entity in a master node, wherein the master node is preset with the first physical entity and a second physical entity and is connected to a first subscriber node and a second subscriber node; pushing, the first commit redo record to a first subscriber node corresponding to the first physical entity based on mapping relationships between the physical entities and the subscriber nodes, wherein the first commit redo record is used for replicating data of the first write operation from the master node to the first subscriber node, and then to the second subscriber node through pushing, by the first subscriber node, the received first commit redo record to the second subscriber node.