Database Migration Service Transaction Log Transformation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Relational databases face challenges during data migration due to the time requirement for migrating data from an active first database to a second database, which can result in missed modifications if changes occur during the migration process.

Innovation Solution

A migration service identifies and applies transformation rules to modifications in the transaction log of the first database, ensuring that these changes are reflected in the second database, and transitions the active database usage once all post-migration modifications are processed.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If data migration is performed from an active first database to a second database, then data transformation and loading can be completed, but modifications made to the first database during migration may be missed causing data inconsistency

Engineering Contradiction:

Improvedata consistencyVSAvoiddowntime

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary actions by identifying all modifications in the transaction log before finalizing the migration. The migration service scans the transaction log to detect any modifications made during the migration process and applies them to the second database before switching it as the active database, ensuring data consistency is maintained.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback by continuously monitoring the transaction log during the migration process. The migration service periodically checks for new modifications and dynamically updates the second database, creating a closed-loop system that ensures the second database remains synchronized with the first database throughout the migration.

Inventive Principle:

Principle #23Feedback

2Productivity

If the migration process is accelerated to reduce downtime, then productivity improves, but the risk of missing modifications increases

Engineering Contradiction:

Improvemigration speedVSAvoidmodification capture accuracy

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system maintains continuity of useful action by continuously monitoring the transaction log throughout the entire migration process. Rather than performing discrete checks, the migration service maintains an ongoing scan of the transaction log, ensuring that no modifications are missed regardless of the migration speed.

Inventive Principle:

Principle #20Continuity of useful action

Solution Approach 2:

The migration service performs preliminary identification of modifications by scanning the transaction log ahead of time and maintaining a buffer of detected changes. This allows the system to prepare modification updates in advance and apply them systematically to the second database without interrupting the migration flow.

Inventive Principle:

Principle #10Preliminary action

3Reliability

If the system waits for complete migration before switching databases, then data consistency is ensured, but downtime increases

Engineering Contradiction:

Improvedata consistencyVSAvoidmigration duration

Core Design Contradiction:

ReliabilityVSDuration of action of moving object

Solution Approach 1:

The system performs preliminary identification of all modifications by scanning the transaction log before finalizing the database switch. The migration service detects modifications in advance, applies them to the second database, and only then completes the transition, ensuring consistency is established before the switch occurs.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The migration process is segmented into distinct phases: initial data transformation and loading, continuous modification detection and application, and final database switching. This segmentation allows the system to progressively ensure data consistency while minimizing the duration of each phase, reducing overall migration time.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11650964B2Limiting downtime associated with migrations of databases

Publication Date: 2023.05.16 VMWARE INC

AI summary

Described herein are systems, methods, and software to manage the downtime associated with updates and configuration modification to the database. In one implementation, a migration service initiates a migration of data from a first database to a second database. The migration service further identifies, in a transaction log, modifications to the data in the first database after initiating the migration of the data and, for each modification, applies one or more transformation rules to the modification to make a compliant modification and updates the second database with the compliant modification. Once the migration is complete and no more modifications exist in the transaction log, the migration service may transition from using the first database as an active database to using the second database as the active database.