Hybrid Cloud Data Migration via Intermediate Buffer

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Traditional data migration methods face challenges due to differing data environments with various formats, interfaces, and transfer protocols, leading to complex and error-prone processes that are time-consuming and require numerous user inputs.

Innovation Solution

A system that receives a migration request, captures source file metadata, transfers files through an intermediate environment, generates a parquet file set, and uses native agents to schedule operations, ensuring efficient data migration while reducing errors and user interactions by enabling automated processes and enhanced security.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If traditional data migration methods are used across different data environments, then data can be transferred between environments, but the process becomes complex and error-prone due to varying formats, interfaces, and transfer protocols

Engineering Contradiction:

Improvedata environment compatibilityVSAvoidmigration process complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent introduces an intermediate data environment as a mediator between source and target data environments. This intermediate environment acts as a buffer that receives data from the source environment and then transfers it to the target environment, thereby simplifying the migration process by breaking it into manageable stages and reducing the complexity of direct multi-environment compatibility handling.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The migration process is segmented into distinct phases: first transferring data from the source environment to an intermediate environment, then from the intermediate environment to the target environment. This segmentation allows each transfer stage to be independently managed and optimized, reducing overall process complexity.

Inventive Principle:

Principle #1Segmentation

2Productivity

If traditional data migration methods are used, then data transfer can occur, but the process is time-consuming and requires numerous user inputs

Engineering Contradiction:

Improvedata migration speedVSAvoidmigration process time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system performs preliminary actions by automatically capturing source file metadata before the actual data transfer occurs. This preliminary metadata capture enables the system to pre-configure transfer parameters and validate data characteristics in advance, reducing the need for manual user inputs during the migration process and accelerating overall productivity.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements self-service capabilities by automatically capturing metadata, validating data formats, and configuring transfer processes without requiring numerous user inputs. The automated metadata capture and validation mechanisms enable the system to self-configure migration parameters, reducing both time loss and manual intervention requirements.

Inventive Principle:

Principle #25Self-service

3Reliability

If traditional data migration methods are used, then data can be transferred, but errors may be introduced that are not reported or captured

Engineering Contradiction:

Improvedata transfer accuracyVSAvoiderror detection capability

Core Design Contradiction:

ReliabilityVSLoss of information

Solution Approach 1:

The patent implements feedback mechanisms that automatically report and capture errors during data migration. The system monitors the migration process, detects errors in data transfer between environments, and provides feedback reports that document these errors. This feedback loop ensures that data transfer accuracy is maintained and all errors are properly captured and reported for correction.

Inventive Principle:

Principle #23Feedback

4Ease of operation

If manual data migration processes are used, then user control is maintained, but the process requires numerous user inputs and is intolerant of informational faults

Engineering Contradiction:

Improveuser interaction requirementVSAvoidfault tolerance

Core Design Contradiction:

Ease of operationVSReliability

Solution Approach 1:

The system replaces manual user operations with automated self-service mechanisms that capture metadata, validate data formats, and execute migration processes automatically. This automation reduces the number of user inputs required while simultaneously improving fault tolerance through systematic error detection and reporting capabilities that are more reliable than manual fault detection.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS11775473B2Seamless data movement and metadata management in a hybrid cloud setting using a configurable micro services based architecture

Publication Date: 2023.10.03 AMERICAN EXPRESS TRAVEL RELATED SERVICES CO INC

AI summary

A system for data migration is disclosed. The system may receive a migration request comprising a source file path and a target file location. The system may capture source file metadata based on the source file path and the migration request. The system may transfer a source file from a first data environment to an intermediate data environment via a first transfer process. The system may transfer the source file from the intermediate data environment to a second data environment via a second transfer process.