Dynamic Latency Rollover in Data Replication

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current data replication tools face challenges in accurately measuring latency from the perspective of both source and target data stores, as they unnecessarily track and replicate transactions that are not contextual to the replication process, leading to inaccurate latency values.

Innovation Solution

A method is implemented to dynamically roll over source and target latency by identifying and skipping open transactions that affect parked tables, using Access Control List (ACL) based processing to determine relevant transactions and calculating accurate latency values for in-scope transactions, thereby improving latency measurement accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If all transactions are tracked and replicated regardless of scope, then completeness of replication is improved, but latency measurement accuracy deteriorates due to inclusion of irrelevant transactions

Engineering Contradiction:

Improvecompleteness of replicationVSAvoidlatency measurement accuracy

Core Design Contradiction:

ReliabilityVSMeasurement precision

Solution Approach 1:

The patent segments transactions into in-scope and out-of-scope categories based on table parking status. The replication system processes only in-scope transactions for latency measurement, while out-of-scope transactions are excluded. This segmentation resolves the contradiction by maintaining reliability for relevant transactions while improving latency measurement accuracy by excluding irrelevant ones.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies different quality standards to different transactions: in-scope transactions receive full replication processing with latency tracking, while out-of-scope transactions are excluded from latency measurements. This local differentiation allows the system to maintain completeness where needed while improving measurement precision by focusing only on relevant transactions.

Inventive Principle:

Principle #3Local quality

2Measurement precision

If open transactions are continuously monitored and tracked, then replication accuracy is improved, but system complexity and resource consumption worsen

Engineering Contradiction:

Improvereplication accuracyVSAvoidsystem complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent extracts and excludes out-of-scope transactions from the replication process by identifying parked tables and filtering transactions that only affect these tables. This extraction reduces system complexity by removing unnecessary monitoring and processing steps for irrelevant transactions while maintaining replication accuracy for in-scope transactions.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent applies partial monitoring by continuously tracking only in-scope transactions rather than all transactions. This partial action approach maintains replication accuracy for relevant transactions while reducing system complexity by avoiding excessive monitoring of out-of-scope transactions.

Inventive Principle:

Principle #16Partial or excessive action

3Quantity of substance

If transaction logs are retained for all transactions, then data completeness is improved, but storage requirements worsen

Engineering Contradiction:

Improvedata completenessVSAvoidstorage requirements

Core Design Contradiction:

Quantity of substanceVSVolume of stationary object

Solution Approach 1:

The patent extracts and excludes out-of-scope transactions from transaction log retention by identifying transactions that only affect parked tables. These excluded transactions do not require log retention since they will not be replicated. This extraction maintains data completeness for in-scope transactions while reducing storage requirements by eliminating redundant log entries for out-of-scope transactions.

Inventive Principle:

Principle #2Taking out (Extraction)

4Reliability

If all transactions are replicated to target data store, then replication completeness is improved, but processing time worsens due to unnecessary transactions

Engineering Contradiction:

Improvereplication completenessVSAvoidprocessing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent extracts and excludes out-of-scope transactions from the replication process by identifying transactions that only affect parked tables. This extraction maintains replication completeness for in-scope transactions while reducing processing time by eliminating unnecessary replication operations for out-of-scope transactions.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent implements skipping of out-of-scope transactions by detecting parked tables and bypassing the replication process for transactions that only affect these tables. This skipping mechanism maintains replication completeness for relevant transactions while significantly reducing processing time by rushing through or entirely skipping irrelevant transactions.

Inventive Principle:

Principle #21Skipping (Rushing through)

Data Source

PatentUS11321353B2Dynamic roll-over of source and target latency in a replication environment

Publication Date: 2022.05.03 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

A system and method for performing a dynamic rollover of source and target latency in a replication environment. One or more tables of a source data store being replicated to a target data store are parked, wherein any subsequent changes for the parked tables are not replicated to the target data store. Open transactions are identified where the parked tables affect the source and target latency, wherein the open transactions are in-progress and not completed. The source and target latency are then calculated for the identified open transactions.