Simulated Data Migration for Cloud Database Quality Assurance

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Migrating data from a legacy system to a non-relational database, such as those used in cloud infrastructure, poses challenges in ensuring data quality due to differences in data structures and the absence of primary keys in target databases.

Innovation Solution

A method is proposed that involves simulating the data migration process, applying migration rules to source data to generate simulated data structured like the target data, merging simulated and actual target data, and using a machine learning algorithm, such as Minhash LSH, to identify discrepancies and ensure data quality.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If data is migrated from relational databases to non-relational databases (NoSQL) in cloud infrastructure, then data distribution and availability are improved, but data quality assurance becomes difficult due to absence of primary keys and different data structures

Engineering Contradiction:
Improvedata distribution and availabilityVSAvoiddata quality assurance
Core Design Contradiction:
ProductivityVSReliability

Solution Approach 1:

The patent applies preliminary action by simulating the migration process before actual data migration occurs. A simulation engine creates simulated target data from source data using the same migration rules that will be applied during actual migration. This allows validation and discrepancy detection to be performed in advance, ensuring data quality before the actual migration takes place, thus resolving the contradiction between improved data distribution and maintained data quality assurance.

Inventive Principle:
Principle #10Preliminary action

2Adaptability or versatility

If migration rules are applied to transform source data to target data structure, then data transformation capability is improved, but data accuracy and consistency become harder to validate

Engineering Contradiction:
Improvedata transformation capabilityVSAvoiddata accuracy and consistency validation
Core Design Contradiction:
Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent uses copying by creating a simulated copy of the target data through the simulation engine. This simulated target data is generated by applying the same migration rules to source data, creating an identical copy of what the actual migration will produce. By comparing this simulated copy with the actual migrated data, the system can validate accuracy and consistency without compromising the transformation capability, thus resolving the contradiction between adaptability and measurement precision.

Inventive Principle:
Principle #26Copying

3Measurement precision

If machine learning algorithms are used to identify discrepancies in migrated data, then data validation accuracy is improved, but processing time and complexity increase

Engineering Contradiction:
Improvedata validation accuracyVSAvoidprocessing time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent applies preliminary action by pre-processing and pre-comparing data structures before actual migration validation. The simulation engine pre-generates expected target data structures and pre-identifies potential discrepancies using machine learning algorithms. This preliminary validation reduces the time required during actual migration, as the heavy computational work of discrepancy detection is performed in advance, thus resolving the contradiction between validation accuracy and processing time.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS12287805B2Data quality assurance for heterogenous data migration in clouds
Publication Date: 2025.04.29 BULL SA
  • US12287805B2 patent drawing
  • US12287805B2 patent drawing

AI summary

Method for controlling data quality assurance after a migration between a source repository, which includes source data associated with a source data structure, and a target repository, which includes target data associated with a target data structure, according to migration rules. The target data is organized as a set of elements. The method includes simulating the migration by applying migration rules to source data so as to output simulated data according to the target data structure, where the simulated data is organized as a set of elements. The method includes merging the simulated data and the target data by excluding identical elements, to form an input vector for a machine learning algorithm. The method includes applying the machine learning algorithm for pairing elements of the input vector according to a similarity score and determining a discrepancies indicator according to the similarity score for each element of the input vector.