Net Change Mirroring for Database Replication Efficiency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data replication techniques are limited to net changes within a single transaction, failing to efficiently replicate changes across multiple transactions, which increases replication time and data volume, and resource utilization.

Innovation Solution

A method that identifies initial and latest operations within a defined window in the source database log, determines the net change, and stores it as a final operation in an optimization repository for efficient replication to a target database, reducing redundant operations and optimizing resource usage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If existing optimization techniques are used to examine the end image of a database table row, then net changes within a single transaction can be optimized, but replication time and data volume increase when multiple transactions are involved

Engineering Contradiction:

Improvereplication efficiencyVSAvoidreplication time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent merges multiple transaction logs into a single optimization window, combining operations from multiple transactions on the same row identifier. Instead of processing each transaction separately, the system consolidates all operations within a defined window (from SCN of previous replication to SCN of current replication point) and determines a single net change operation, thereby reducing total replication time across multiple transactions.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent segments the database log into discrete windows defined by system change numbers (SCNs). Each window represents a specific replication interval, allowing the system to process and optimize net changes independently for each window. This segmentation enables efficient handling of multiple transactions by breaking them into manageable, bounded units that can be processed in parallel or sequentially without interference.

Inventive Principle:

Principle #1Segmentation

2Reliability

If all operations across multiple transactions are replicated, then data accuracy is maintained, but data volume and resource utilization increase

Engineering Contradiction:

Improvedata accuracyVSAvoiddata volume

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent extracts only the essential net change operation from multiple intermediate operations. By identifying the initial operation and final operation within a transaction window, the system extracts just the necessary data transformation needed to maintain target database accuracy, discarding redundant intermediate operations that would otherwise increase data volume without adding value.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent changes the parameter of replication from transmitting all individual operations to transmitting only the net change operation. This parameter change transforms the replication process from a volume-intensive operation to a streamlined process that maintains data accuracy by replicating only the essential state change (from initial to final operation) rather than every intermediate step.

Inventive Principle:

Principle #35Parameter changes

3Productivity

If net change replication is performed across multiple transactions, then throughput is enhanced, but system complexity increases

Engineering Contradiction:

ImprovethroughputVSAvoidsystem complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent performs preliminary actions by pre-defining replication windows using system change numbers (SCNs) before actual replication occurs. The source replication engine establishes the window boundaries (from previous replication SCN to current replication SCN) in advance, allowing the target replication engine to efficiently process only the relevant operations within that predefined range, thereby increasing throughput without requiring complex real-time coordination.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces an optimization repository as an intermediary between the source and target replication engines. This intermediary component stores the determined net change operations in a standardized format, allowing the source engine to perform complex multi-transaction analysis while the target engine receives simplified, pre-processed operations. This intermediary layer manages system complexity by decoupling the complexity of net change determination from the replication execution.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS11615113B2Net change mirroring optimization across transactions in replication environment

Publication Date: 2023.03.28 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

In optimizing net change data replication across a plurality of transactions in a replication environment, a source replication engine at a source database system: obtains a window in a source database log for the net change data replication; for a given row identifier found in the source database log, identifies an initial operation and a latest operation across the plurality of transactions within the window; compares the initial operation and the latest operation for the given row identifier to determine a net change to a row associated with the given row identifier; and stores the net change as a final operation on the given row identifier in an optimization repository for replication to a target database. The operations stored in the optimization repository are then sent to a target database system for replication to the target database.