Single-Instance Storage Volumes Data Locality Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Single-instance-storage volumes face inefficiencies in data retrieval due to the scattering of data segments across the storage device, leading to increased random disk accesses and decreased performance.
Innovation Solution
A method where a client provides fingerprints of data segments to a single-instance storage volume, determining if non-sequential accesses exceed a threshold, and if so, sends the entire set of data segments for contiguous storage, replacing previously stored duplicates and updating the mapping table to ensure spatial locality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data deduplication is implemented using pointers to previously stored segments, then storage space is reduced by eliminating duplicates, but data retrieval performance deteriorates due to scattered physical locations
Solution Approach 1:
The patent segments data into fixed-size segments and uses fingerprints to identify duplicate segments. By segmenting data and storing only unique segments with pointers to duplicates, the system achieves space efficiency while managing retrieval through segment-level operations rather than file-level operations.
Solution Approach 2:
The patent introduces a metadata dimension layer that sits above the physical storage layer. The metadata database stores mapping information between file offsets and segment locations, allowing logical access to be decoupled from physical storage locations. This dimensional separation enables efficient retrieval by translating logical file offsets into physical segment locations without requiring sequential disk access.
2Adaptability or versatility
If random disk accesses are used to retrieve scattered data segments, then flexibility in data storage is maintained, but access time increases significantly
Solution Approach 1:
The system performs preliminary actions by pre-computing and storing metadata that maps file offsets to segment locations in a metadata database. When data needs to be retrieved, the system first queries the metadata database to obtain the physical locations of required segments, then performs direct disk access to those specific locations. This preliminary metadata preparation eliminates the need for sequential scanning or random access speculation.
Solution Approach 2:
The metadata database acts as an intermediary layer between the logical file system and the physical storage medium. Instead of directly accessing scattered segments on disk, the system first queries the metadata intermediary to translate logical offsets into physical segment locations and offsets. This intermediary layer decouples the flexibility of data storage from the performance of data retrieval.
Data Source
AI summary
A method and system for improving performance with single-instance-storage volumes by leveraging data locality is provided. A client provides a set of fingerprints generated from data segments to be saved to a single-instance storage volume and receives the information on whether a data segment exists on the single-instance storage volume and where it is stored if a data segment exists. Based on its received information, the client determines if a number of non-sequential accesses of a computer-readable medium for the said set of segments from the single-instance-storage volume exceeds a predetermined threshold. If so, the client provides the whole set of data segments for storage within the single-instance storage volume regardless of whether or not the data segments are duplicate data segments. These sent data segments will be stored contiguously within the single-instance storage volume while the duplicates will be removed from their previous stored locations.