Storage System Segmentation for Fragmentation Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current mass data storage systems face inefficiencies due to data fragmentation, leading to performance penalties and potential depletion of free disk space, as data is not stored contiguously, resulting in the accumulation of 'garbage' blocks that require defragmentation and garbage collection processes to manage.
Innovation Solution
A method and system that divide logical volumes into statistical segments with predefined sizes, assign activity levels based on I/O statistics, and manage background defragmentation and garbage collection processes using these activity levels, selecting segments for processes that fit specific criteria and consolidating data portions for efficient writing to disk drives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in non-contiguous physical space, then storage capacity is increased, but reading performance deteriorates due to fragmentation
Solution Approach 1:
The storage system is divided into multiple pools with different performance characteristics. Hot data is stored in high-performance pools while cold data is stored in lower-performance pools, allowing the system to maintain high reading performance for frequently accessed data while maximizing overall storage capacity.
Solution Approach 2:
Different regions of the storage system are assigned different quality levels based on access patterns. Frequently accessed data receives higher performance treatment through placement in optimized pools, while less accessed data is placed in standard pools, creating local quality variations that optimize overall system performance.
2Speed
If defragmentation processes are implemented, then reading performance is improved, but system productivity deteriorates due to additional processing overhead
Solution Approach 1:
Data is pre-positioned in appropriate pools based on predicted access patterns before actual access occurs. The system proactively moves data between pools based on activity levels and access history, eliminating the need for reactive defragmentation operations and maintaining high performance without productivity loss.
Solution Approach 2:
The storage system automatically manages its own data placement and movement between pools based on monitored access patterns. The system self-optimizes without requiring external defragmentation processes, using background threads to continuously adjust data placement based on real-time activity levels.
3Quantity of substance
If garbage collection processes are implemented, then free disk space is maintained, but device complexity increases
Solution Approach 1:
The garbage collection function is merged with the existing data placement and pool management operations. The same background threads that manage data placement also handle garbage collection by identifying and relocating invalid data blocks, eliminating the need for separate garbage collection processes and reducing overall system complexity.
Data Source
AI summary
There are provided a mass storage system comprising a control layer operatively coupled to a physical storage space and operable to interface with one or more clients and to present to said clients a plurality of logical volumes. The method of operating the storage system comprises: dividing one or more logical volumes into a plurality of statistical segments with predefined size; assigning to each given statistical segment a corresponding activity level characterizing statistics of I/O activity with regard to data portions within the given statistical segment, said statistics collected over a plurality of cycles of fixed counting length; and managing one or more data storage processes in the storage system (e.g. a background defragmentation process, a background garbage collection process, a destage management process, etc.) using said activity level.