Distributed Data Migration With XA Double-Write Consistency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data migration methods in distributed systems require third-party tools for log data reading and replay, are prone to failure due to binlog loss, and disrupt client access during migration, affecting data consistency and user experience.

Innovation Solution

A data migration method that performs a double write operation on incremental data to both the source and target data tables using XA transactions, ensuring atomicity and consistency without relying on binlogs, allowing continuous client access.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional data migration methods using binlog and third-party tools are employed, then data migration can be achieved, but the system becomes prone to failure due to binlog loss and requires disruption of client access

Engineering Contradiction:

Improvedata migration reliabilityVSAvoidclient access continuity

Core Design Contradiction:

ReliabilityVSEase of operation

Solution Approach 1:

The patent extracts the dependency on binlog and third-party tools by implementing a double-write mechanism that directly writes to both source and target tables. This eliminates the intermediary binlog layer that causes reliability issues and the need for service disruption.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent performs preliminary actions by pre-writing data to both the source table and target table simultaneously through XA transactions. This preliminary double-write ensures data consistency is established before migration completion, allowing seamless switching without client access disruption.

Inventive Principle:

Principle #10Preliminary action

2Ease of manufacture

If binlog-based data migration is used, then data can be migrated, but the system becomes complex due to reliance on third-party tools and binlog management

Engineering Contradiction:

Improvemigration implementation simplicityVSAvoidsystem complexity

Core Design Contradiction:

Ease of manufactureVSDevice complexity

Solution Approach 1:

The patent enables the database system to perform data migration autonomously through its own XA transaction mechanism, eliminating the need for external third-party tools. The double-write process is integrated into the database's native transaction system, simplifying the overall architecture.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent merges the data migration function with the existing XA transaction mechanism of the database system. By combining these functions, the system eliminates separate binlog management and third-party tool requirements, reducing overall system complexity.

Inventive Principle:

Principle #5Merging (Combining)

3Manufacturing precision

If data migration disrupts client access to ensure consistency, then data consistency is achieved, but user experience and access efficiency deteriorate

Engineering Contradiction:

Improvedata consistencyVSAvoidclient access efficiency

Core Design Contradiction:

Manufacturing precisionVSProductivity

Solution Approach 1:

The patent maintains continuity of useful action by allowing client access to continue uninterrupted during the migration process. The double-write mechanism ensures that data consistency is maintained throughout the migration, enabling seamless switching between source and target tables without stopping client operations.

Inventive Principle:

Principle #20Continuity of useful action

4Adaptability or versatility

If reliance on third-party tools and binlog is maintained, then existing migration processes can continue, but the system lacks adaptability and increases failure risk

Engineering Contradiction:

Improvemigration method applicabilityVSAvoidmigration success rate

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent achieves universality by implementing a double-write migration mechanism that works across different database scenarios using the native XA transaction system. This approach eliminates dependency on specific binlog implementations and third-party tools, making the migration method broadly applicable while improving reliability.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS12481628B2Data migration method and apparatus, device, distributed system and storage medium

Publication Date: 2025.11.25 CLOUD INTELLIGENCE ASSETS HOLDING (SINGAPORE) PTE LTD

AI summary

A data migration method and apparatus, a device, a distributed system and a storage medium are provided. Said method is applied to each data node in a distributed system comprising a plurality of data nodes. Said method comprises: determining, in a current data migration task, a table to be migrated of a current node, and a target data node (S101); and writing incremental data in the table to be migrated into the table to be migrated, and a target data table of the target data node, the incremental data being data obtained by performing, after the data migration task is started, a change operation on data in the table to be migrated (S102).