Transactional Replication for Distributed Database Scalability

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Centralized distributed databases face scalability issues due to high bandwidth consumption and latency, while partitioned databases suffer from data inconsistencies and reliability concerns, especially in highly distributed systems.

Innovation Solution

Implementing a distributed database system that allows members to provisionally write data locally without consulting a master, with subsequent reconciliation, using semantic replication and atomic transactions to ensure data consistency across nodes, including active and passive HA pairs.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If centralized database architecture is used, then data consistency is maintained, but bandwidth consumption increases and scalability deteriorates

Engineering Contradiction:

Improvedata consistencyVSAvoidscalability

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The database is divided into multiple partitions distributed across different nodes, each with local master and members. This segmentation allows independent operation of partitions, reducing bandwidth consumption and improving scalability while maintaining consistency within each partition through local masters.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system performs preliminary actions by pre-establishing partition boundaries and local master-authority relationships before data writes occur. This allows nodes to make local writes without immediate master coordination, reducing latency and bandwidth usage while maintaining eventual consistency through background reconciliation.

Inventive Principle:

Principle #10Preliminary action

2Reliability

If centralized database architecture is used, then data consistency is maintained, but write latency increases

Engineering Contradiction:

Improvedata consistencyVSAvoidwrite latency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system dynamically transitions from synchronous master coordination to asynchronous replication model. Members can provisionally write data locally and then propagate changes to master for reconciliation, making the write operation dynamic between synchronous and asynchronous modes based on system state.

Inventive Principle:

Principle #15Dynamics

3Productivity

If partitioned database architecture is used, then scalability improves, but data consistency deteriorates

Engineering Contradiction:

ImprovescalabilityVSAvoiddata consistency

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system implements feedback mechanisms where members propagate changes to the master and receive acknowledgments. The master sends feedback about conflict resolution status to members, ensuring consistency across partitions while maintaining scalability. This feedback loop resolves data consistency issues without requiring synchronous coordination.

Inventive Principle:

Principle #23Feedback

4Adaptability or versatility

If active-passive HA pairs are used, then availability improves, but data reliability deteriorates

Engineering Contradiction:

ImproveavailabilityVSAvoiddata consistency between nodes

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The system performs preliminary actions by establishing transactional replication relationships between active and passive nodes before failures occur. Changes are replicated transactionally to passive nodes, ensuring data consistency is maintained in advance, so when failures occur, the passive node has already received and validated the data.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system replaces manual failover mechanisms with automatic transactional replication. Instead of relying on mechanical backup processes, the system uses automated transaction logging and replication protocols to ensure data consistency between active and passive nodes, improving both availability and data reliability.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS9317545B2Transactional replication

Publication Date: 2016.04.19 INFOBLOX INC

AI summary

Writing data in a distributed database having a plurality of nodes is disclosed. Writing includes receiving a write request at a node, wherein the write request is associated with one or more operations to define an atomic transaction and performing the atomic transaction based on the request. The atomic transaction includes writing to a first version of the database in the node and writing to an entity representative of a state of the first version of the database.