Systems and methods for synchronizing between a source database cluster and a destination database cluster

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing database replication technologies are inadequate for efficient synchronization between clusters managed by different vendors or located at different geographical regions, often resulting in inconsistent and unpredictable outcomes, especially when parallel replication is employed.

Innovation Solution

A system utilizing multiple replicators, each responsible for a subset of data, executes in parallel to synchronize data between source and destination clusters, allowing for temporary index violations and maintaining shard structures while ensuring eventual consistency through index relaxation and validation, and utilizing change streams for continuous data replication.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If multiple replicators are executed in parallel for cluster synchronization, then productivity is improved, but reliability deteriorates due to index inconsistencies and unpredictable outcomes

Engineering Contradiction:

Improvesynchronization speedVSAvoiddata consistency

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system segments the data synchronization task by assigning different subsets of data to different replicators, allowing parallel execution while maintaining control over consistency through structured index management and validation mechanisms

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system implements feedback mechanisms through index validation and consistency checking that monitor the synchronization process and provide corrective actions when inconsistencies are detected, ensuring reliable outcomes from parallel replication operations

Inventive Principle:

Principle #23Feedback

2Device complexity

If existing replication techniques are used for cluster to cluster synchronization, then device complexity is reduced, but manufacturing precision deteriorates resulting in inconsistent synchronization outcomes

Engineering Contradiction:

Improvesystem simplicityVSAvoidsynchronization accuracy

Core Design Contradiction:

Device complexityVSManufacturing precision

Solution Approach 1:

The system changes critical parameters such as index uniqueness constraints and validation rules during the synchronization process, allowing flexible control over synchronization accuracy while adapting to different cluster configurations and requirements

Inventive Principle:

Principle #35Parameter changes

3Productivity

If parallel replication is employed for large-scale sharded clusters, then productivity is improved, but loss of information increases due to index violations

Engineering Contradiction:

Improvereplication efficiencyVSAvoidindex consistency

Core Design Contradiction:

ProductivityVSLoss of information

Solution Approach 1:

The system prepares for potential index violations in advance by implementing validation mechanisms and recovery procedures that prevent data loss and ensure index consistency is maintained throughout the parallel replication process

Inventive Principle:

Principle #11Beforehand cushioning (Prior cushioning)

Data Source

PatentUS20250208958A1Systems and methods for synchronizing between a source database cluster and a destination database cluster

Publication Date: 2025.06.26 MONGODB INC

AI summary

In some embodiments, a database cluster to cluster synchronization system may include multiple replicators coupled to a source database cluster and a destination database cluster, where the source and destination cluster may be shard clusters. Each of the multiple replicators may correspond to a respective subset of the source database cluster and configured to monitor changes of data on the respective subset of the source database cluster and translate the changes of data to one or more database operations to be performed on the destination cluster. The changes of data on the source database may be contained in respective change streams associated with each of the replicators.