Storage System Deduplication Fragmentation Context Rewriting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Storage systems with deduplicated data face performance degradation due to data fragmentation, leading to increased retrieval time as new data blocks are scattered across storage devices, reducing read performance and increasing storage costs.
Innovation Solution
The implementation of a context-based rewriting algorithm that identifies and rewrites highly fragmented duplicates, aligning their disk and stream contexts to improve read performance without significantly impacting write performance or storage efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If deduplicated storage is implemented to eliminate duplicated data, then storage cost is reduced, but retrieval performance decreases due to data scattering
Solution Approach 1:
The system performs preliminary actions by storing metadata information (hash values, logical addresses, physical addresses) about data blocks before actual retrieval operations. This allows the system to quickly locate scattered data blocks without scanning the entire storage device, thereby maintaining fast retrieval performance while implementing deduplicated storage.
Solution Approach 2:
The patent introduces metadata as an intermediary layer between the retrieval request and the actual data blocks. The metadata contains indexing information that mediates the search process, allowing the system to efficiently locate scattered deduplicated data blocks without directly scanning storage devices, thus resolving the contradiction between storage efficiency and retrieval speed.
2Device complexity
If new block data are stored at the end of storage region, then storage structure is simplified, but data become scattered and retrieval performance decreases
Solution Approach 1:
The patent uses metadata as an intermediary indexing layer that records the logical-to-physical address mappings. This allows the system to maintain a simple sequential storage structure while enabling fast retrieval of scattered data blocks through the metadata index, without requiring complex storage organization.
Solution Approach 2:
The system changes the parameter of data organization by separating logical addressing from physical storage location. Data blocks can be stored sequentially at the end of regions, but the metadata parameter stores the mapping information, allowing retrieval without scanning the entire storage device.
3Quantity of substance
If data are divided into multiple block data for deduplication, then storage efficiency is improved, but retrieval requires scanning huge number of disks
Solution Approach 1:
The patent introduces metadata as an intermediary indexing mechanism that records the relationship between data content (hash values) and physical storage locations. This allows the system to efficiently locate specific block data among multiple divided blocks without scanning huge numbers of disks, thereby maintaining fast retrieval while achieving storage efficiency through deduplication.
Solution Approach 2:
The system performs preliminary indexing by storing metadata (hash values, addresses) about data blocks when they are first written. This preliminary action creates a lookup table that enables fast retrieval of divided block data without subsequent scanning of storage devices.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A storage system according to the present invention includes: a block detecting means for checking whether respective block data obtained by division are already stored in a storage device; and a data writing means for storing the respective block data obtained by division with duplicated storage eliminated into the storage device. The block detecting means detects a share rate representing a rate of a shared portion between a plurality of consecutive block data configuring a predetermined range in write target data among the block data obtained by division and a plurality of block data in a predetermined range already stored consecutively in the storage device. The data writing means, depending on the detected share rate, newly stores the block data obtained by division, into the storage device.