Stream-Based Data Allocation for Storage Fragmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing file systems face challenges in efficiently storing data with different characteristics, leading to fragmentation, increased write amplification, and inefficient use of storage resources, particularly in virtualized storage environments where data is not logically or physically separated.
Innovation Solution
The solution involves assigning data streams to different allocation areas based on the characteristics of the data, such as access frequency and access patterns, using stream identifiers to direct the storage device to process the data in specific allocation areas, thereby separating data types logically within the storage device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data with different characteristics is stored together in virtualized storage, then storage resources are consolidated and abstracted, but fragmentation increases and access performance deteriorates
Solution Approach 1:
The patent divides the storage device into multiple allocation areas (first allocation area and second allocation area) based on data characteristics. Hot data is allocated to the first allocation area while cold data is allocated to the second allocation area, preventing fragmentation by segregating data with different access patterns into distinct physical regions.
Solution Approach 2:
The patent applies different storage policies to different regions of the storage device. The first allocation area is optimized for frequently accessed data with faster access patterns, while the second allocation area is optimized for infrequently accessed data. This local differentiation ensures that each region's characteristics match the data it stores, improving overall system performance.
2Reliability
If storage space is reserved for background garbage collection, then solid state drive operations are enabled, but storage efficiency decreases due to over-provisioning
Solution Approach 1:
The patent segments the storage device into data allocation areas and a separate garbage collection allocation area. This segmentation allows garbage collection operations to be confined to a specific region, preventing them from interfering with data storage capacity. The garbage collection area is precisely sized based on workload characteristics, eliminating over-provisioning while maintaining reliable SSD operations.
3Adaptability or versatility
If data is frequently overwritten in virtualized storage, then data flexibility is improved, but write amplification increases and device wear accelerates
Solution Approach 1:
The patent separates frequently written data into a dedicated first allocation area, isolating write operations from the rest of the storage device. This segmentation confines write amplification effects to a limited region, preventing them from affecting overall device performance and lifespan. The separated structure allows aggressive write policies for hot data without compromising cold data integrity.
Solution Approach 2:
The patent optimizes the first allocation area specifically for write-intensive workloads by positioning it in a high-endurance region of the storage device. This local optimization ensures that frequently overwritten data resides in areas with higher program/erase cycle tolerance, reducing device wear while maintaining data flexibility.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Techniques are provided for selectively storing data into allocation areas using streams. A set of allocation areas (e.g., ranges of block numbers such as virtual block numbers) are defined for a storage device. Data having particular characteristics (e.g., user data, metadata, hot data, cold data, randomly accessed data, sequentially accessed data, etc.) will be sent to the storage device for selective storage in corresponding allocation areas. For example, when a file system receives a write stream of hot data, the hot data may be assigned to a stream. The stream will be tagged using a stream identifier that is used as an indicator to the storage device to process data of the stream using an allocation area defined for hot data. In this way, data having different characteristics will be stored/confined within particular allocation areas of the storage device to reduce fragmentation and write amplification.