Database Replication Stream Merging for Concurrency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Data replication processes face reduced concurrency due to increased latency between source and target databases, leading to longer lock durations and decreased performance in handling copy and user transactions.

Innovation Solution

Merging copy and user transaction streams locally on the source database server to produce a combined stream, which is then ordered and transmitted to the target, allowing locks to be released once transactions are added to the stream, independent of latency between the source and target.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If locks are held on source database until target acknowledges receipt, then data consistency is ensured, but lock duration increases and concurrency decreases

Engineering Contradiction:

Improvedata consistencyVSAvoidconcurrency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent segments the transaction processing into two independent phases: (1) local stream merging and ordering on the source database, and (2) transmission to and acknowledgment from the target database. By segmenting the lock holding period to only cover phase 1, the patent reduces lock duration while ensuring data consistency through the ordering mechanism in phase 2.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces an intermediary ordering mechanism that acts as a mediator between the source and target databases. This intermediary ensures correct transaction ordering without requiring the source to wait for target acknowledgment, thus resolving the contradiction between consistency and concurrency.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If locks are held for longer durations to ensure correct ordering, then transaction correctness is maintained, but performance decreases due to reduced concurrency

Engineering Contradiction:

Improvetransaction correctnessVSAvoidperformance

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent performs preliminary actions on the source database by merging and ordering transactions locally before transmission. This preliminary ordering ensures transaction correctness is established in advance, eliminating the need for prolonged locks waiting for target acknowledgment, thus improving performance.

Inventive Principle:

Principle #10Preliminary action

3Reliability

If separate streams are maintained for copy and user transactions, then data integrity is preserved, but system complexity increases

Engineering Contradiction:

Improvedata integrityVSAvoidsystem complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent merges the copy stream and user transaction stream into a single combined stream on the source database. This merging maintains data integrity through the ordering mechanism while reducing system complexity by consolidating multiple streams into one, eliminating the need for separate handling pathways.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS9535931B2Data seeding optimization for database replication

Publication Date: 2017.01.03 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

Streaming database replication is provided by merging a stream of user transactions with a stream of copy transactions comprising copy data into a combined stream on a source. A target receives a single stream that includes copy transaction and concurrent user transactions in an order that enables conflicts between data being copied and user transactions to be handled correctly. Furthermore, locks applied to data subject to a copy transaction or user transaction can be released once the copy transaction or user transaction is added to the combined stream.