Sparse Volume Architecture for Rapid Export via Remote Fetch
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing file systems face inefficiencies in quickly generating and exporting volumes due to the need for repeated write operations to create hole-filled volumes, which can be impractical for rapid data access, especially when data corruption occurs and bulk data restore operations are required.
Innovation Solution
A system and method for supporting sparse volumes within a file system, where absent data blocks are not stored locally but retrieved remotely using the NetApp Remote Volume (NRV) protocol, allowing for quick generation and export of volumes with minimal write operations by storing missing data on an alternate source and retrieving it via remote fetch operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If hole-filled volumes are created using traditional techniques, then data can be stored with minimal write operations, but data access speed deteriorates due to the need for repeated write operations to generate volumes
Solution Approach 1:
The patent introduces a new dimension of remote storage by storing volume data on remote storage systems instead of locally. This allows the volume to be quickly generated on the storage system without requiring repeated write operations to fill holes, while data can be retrieved remotely when needed, thus resolving the contradiction between minimal write operations and fast data access.
Solution Approach 2:
The patent uses a remote storage system as an intermediary to store volume data. The storage system maintains a reference to the remote storage location and retrieves data from there when needed, eliminating the need for repeated local write operations while ensuring fast data access through efficient remote retrieval mechanisms.
2Reliability
If bulk data restore operations are performed after data corruption, then data can be recovered, but productivity deteriorates due to the time-consuming nature of bulk restore operations
Solution Approach 1:
The patent performs preliminary actions by maintaining volume data on remote storage systems in advance. When data corruption occurs, the system can quickly retrieve the needed data from remote storage without performing time-consuming bulk restore operations, thus improving both data recovery reliability and restore operation speed.
Solution Approach 2:
The patent extracts the bulk data restore operation by storing volume data separately on remote storage systems. This allows the system to retrieve only the specific needed data blocks from remote storage rather than performing comprehensive bulk restore operations, significantly improving restore operation speed while maintaining data recovery capability.
3Productivity
If sparse volumes are created with remote data retrieval, then volume generation speed improves with minimal write operations, but device complexity increases due to the need for remote fetch operations
Solution Approach 1:
The patent makes the storage system universal by implementing a protocol that can handle both local and remote data retrieval through a unified interface. This multi-functionality allows the system to quickly generate sparse volumes with minimal write operations while managing remote fetch operations through the same file system interface, thus improving volume generation speed without proportionally increasing perceived system complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An architecture, including a file-level protocol, for supporting sparse volumes on a storage system is provided. The file-level protocol provides coherency checking for use in retrieving data stored on a backing store remote from a storage system.