Replica Data Migration While Keeping Storage Operations Online
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face inefficiencies in managing data migration and storage operations, particularly in flash-based storage systems, due to redundant write operations and lack of direct control by the operating system, leading to increased latency and reduced reliability.
Innovation Solution
Implementing a direct-mapped flash storage system where the operating system directly addresses data blocks without translation by storage controllers, offloading device management tasks, and utilizing non-volatile RAM for fast-write operations to improve efficiency and reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If storage controllers translate and manage data blocks, then device management is centralized, but write operations become redundant and latency increases
Solution Approach 1:
The patent extracts the data block translation function from storage controllers and assigns it directly to the operating system. The OS now directly addresses flash memory data blocks without controller intervention, eliminating redundant write operations and reducing latency while maintaining centralized management capability.
Solution Approach 2:
The patent introduces a new intermediary layer (the direct-mapped flash storage interface) between the operating system and flash memory devices. This interface allows the OS to directly manage data blocks without the traditional storage controller translation layer, reducing operational overhead and latency.
2Device complexity
If storage controllers manage data migration, then device control is simplified, but system reliability decreases due to redundant operations
Solution Approach 1:
The patent extracts data migration management from storage controllers and implements it directly in the operating system. This eliminates the redundant translation operations that previously occurred in controllers, thereby improving reliability while maintaining simplified control through direct OS management.
3Adaptability or versatility
If traditional flash storage interfaces are used, then compatibility is maintained, but write operations are inefficient and resource overhead increases
Solution Approach 1:
The patent segments the storage interface into two distinct layers: a compatibility layer that maintains traditional flash storage interfaces for external compatibility, and a direct-mapped layer that enables efficient OS-level data block addressing. This segmentation allows the system to maintain compatibility while achieving improved write efficiency through direct OS control.
Data Source
AI summary
An illustrative method includes a storage system receiving a request to migrate a dataset having a plurality of replicas stored at a set of source datastores to a set of target datastores, and incrementally migrating, based on the request, each replica of the dataset from the set of source datastores to the set of target datastores while continuing data operations on the dataset. In some embodiments, the incrementally migrating each replica comprises migrating a first replica from the source datastores to the target datastores while using a second replica stored at the source datastores to service data operations on the dataset, and after migrating the first replica from the set of source datastores to the set of target datastores, migrating the second replica from the source datastores to the target datastores while using the first replica stored at the target datastores to service data operations on the dataset.