Migration Engine Atomic Rename for Heterogeneous Storage

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data migration technologies face challenges in heterogeneous storage systems, including agent failure, data corruption, and loss of file metadata during migration across different operating system platforms, leading to incompatibility and non-interoperability issues.

Innovation Solution

A migration engine with a pre-installed plug-in module is integrated into a remote agent, providing a data migration capability that stores data in a temporary area, renames files atomically, and logs information intelligently to diagnose failures, ensuring data integrity and preserving file attributes across multiple operating systems.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If data is migrated directly from source to destination without temporary storage, then migration speed is improved, but data corruption risk increases due to agent failure

Engineering Contradiction:

Improvemigration speedVSAvoiddata integrity

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent implements preliminary action by creating a temporary copy of the file at the destination location before completing the migration. This temporary file serves as a safety mechanism that allows the migration to be rolled back in case of agent failure, thus preventing data corruption while maintaining efficient migration throughput.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces an intermediary mechanism in the form of a temporary file that acts as a buffer between the source and destination. This intermediary allows the migration process to be atomic - either the file is successfully migrated and renamed, or the temporary file is deleted upon failure, eliminating the risk of partial or corrupted data transfer.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If file attributes are converted during migration between heterogeneous platforms, then interoperability is improved, but loss of original file metadata occurs

Engineering Contradiction:

Improveplatform compatibilityVSAvoidfile metadata

Core Design Contradiction:

Adaptability or versatilityVSLoss of information

Solution Approach 1:

The patent uses copying to preserve file attributes by reading the original file attributes from the source location and embedding them in the temporary file at the destination. This allows the original metadata to be preserved in the temporary file, which can then be restored to the final file if migration succeeds, or recovered if migration fails.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent implements beforehand cushioning by creating a backup copy of the file attributes in the temporary file before any attribute conversion or loss might occur. This cushioning mechanism ensures that even if attribute conversion is necessary for platform compatibility, the original attributes are preserved in the temporary file for potential restoration.

Inventive Principle:

Principle #11Beforehand cushioning (Prior cushioning)

3Difficulty of detecting and measuring

If detailed logging is enabled for failure diagnosis, then diagnostic capability is improved, but system performance deteriorates due to logging overhead

Engineering Contradiction:

Improvefailure diagnosisVSAvoidmigration throughput

Core Design Contradiction:

Difficulty of detecting and measuringVSProductivity

Solution Approach 1:

The patent extracts only the most critical logging information needed for failure diagnosis, such as error codes and key migration events, rather than logging all operations. This selective extraction approach provides sufficient diagnostic capability while minimizing the performance overhead associated with extensive logging.

Inventive Principle:

Principle #2Taking out (Extraction)

4Reliability

If atomic rename operation is used to prevent data corruption, then data integrity is improved, but migration complexity increases

Engineering Contradiction:

Improvedata consistencyVSAvoidmigration process

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent leverages the operating system's built-in atomic rename capability to ensure data consistency. By using the OS-provided atomic rename function, the system achieves reliable data integrity without implementing complex custom atomicity mechanisms, thus reducing overall migration process complexity while maintaining high reliability.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS8285817B1Migration engine for use in a logical namespace of a storage system environment

Publication Date: 2012.10.09 NETAPP INC

AI summary

A migration engine is provided for use in a logical namespace of a storage system environment. A remote agent of an integrated management framework is installed on a host machine of the storage system environment and includes a pre-installed migration plug-in module that configures the agent as a migration engine adapted to provide a data migration capability. That is, the migration plug-in has the intelligence to configure the remote agent as a migration engine adapted to perform data migration from a source location to a destination location in the storage system environment.