Adaptive Compression Mapping with Segment and Hash Defrag Zones
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face inefficiencies in managing dynamic compression rates and interference between segment and hash mapping modes, leading to fragmentation and suboptimal capacity utilization in block-level storage devices.
Innovation Solution
An adaptive mapping scheme that dynamically switches between segment and hash mapping modes based on data request patterns, allocating defrag units accordingly to optimize storage capacity and avoid interference, using garbage collection mechanisms to manage changing compression rates and reallocate unused space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If segment mapping mode is used to allocate space continuously in larger units, then sequential I/O throughput is improved, but hash mapping may result in increased fragmentation when both modes are applied to the same uncompressed address space
Solution Approach 1:
The storage address space is divided into separate segment zones and hash zones. Segment zones are dedicated to segment mapping mode for sequential I/O operations, while hash zones are dedicated to hash mapping mode for random I/O operations. This segmentation prevents interference between the two mapping modes and eliminates fragmentation issues caused by mixed allocation patterns.
Solution Approach 2:
Different regions of the storage space are assigned different mapping characteristics. Segment zones use continuous allocation optimized for sequential access patterns, while hash zones use random allocation optimized for random access patterns. Each zone has specialized properties tailored to its intended access pattern, preventing cross-contamination of allocation strategies.
2Productivity
If adaptive mapping scheme dynamically switches between segment and hash mapping modes, then storage performance and capacity utilization are improved, but the complexity of managing different mapping modes and their interference increases
Solution Approach 1:
The system is divided into distinct segment zones and hash zones with dedicated mapping modes. This eliminates the need for complex dynamic switching logic, as each zone independently manages its own mapping strategy. The segmentation simplifies the overall system architecture by removing cross-mode interference and reducing management complexity.
Solution Approach 2:
The patent introduces a zone management mechanism that acts as an intermediary between the two mapping modes. Each zone is independently managed according to its specific requirements, allowing the system to maintain both mapping modes without direct interaction or interference. This intermediary approach simplifies coordination compared to dynamic switching.
3Speed
If hash mapping allocates space randomly in smaller units, then random I/O latency is reduced, but it may impact the allocation during segment mapping mode due to fragmentation
Solution Approach 1:
The storage space is segmented into hash zones and segment zones with dedicated allocation strategies. Hash zones use random allocation for fast random I/O operations, while segment zones use continuous allocation for efficient segment mapping. This segmentation ensures that random allocation in hash zones does not impact segment mapping operations in segment zones.
Solution Approach 2:
Different regions of storage space are assigned different allocation qualities. Hash zones have random allocation properties optimized for low-latency random access, while segment zones have continuous allocation properties optimized for sequential access and segment mapping. Each region's local properties are tailored to its access pattern requirements without interfering with other regions.
4Quantity of substance
If compression rates vary across different data blocks and change over time, then storage capacity efficiency is improved, but the physical capacity needed for compressed data becomes constantly changing requiring efficient capacity management
Solution Approach 1:
The system dynamically adapts to changing compression rates by allowing flexible allocation within zone structures. As compression rates vary over time and across data blocks, the zone management mechanism can reallocate capacity between hash zones and segment zones to match actual storage needs, maintaining efficiency without requiring complex per-block management.
Solution Approach 2:
The zone management mechanism serves multiple functions: it manages both segment and hash mapping modes, handles dynamic capacity allocation, and adapts to varying compression rates. This multi-functional approach simplifies capacity management compared to specialized mechanisms for each mapping mode, as the zone structure provides a unified framework for handling diverse storage requirements.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods and systems for adaptive mapping for data compression on a storage device is provided. The method includes determining a data request pattern of a workload, determining whether to use at least one of a segment mapping mode or a hash mapping mode for mapping the workload, dividing a space on the storage device into a plurality of defrag units for storing data, and assigning the plurality of defrag units as being at least one of a segment defrag unit or a hash defrag unit. The method also includes when the data request pattern is for the segment mapping mode, storing the data on at least one of the plurality of defrag units assigned as the segment defrag unit, and when the data request pattern is for the hash mapping mode, storing the data on at least one of the plurality of defrag units assigned as the hash defrag unit.