Modular Block Allocator for Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing block allocators in data storage systems face challenges in timely response to multiple cleaner messages in cluster network environments with increasing numbers of CPUs, leading to processing delays and inefficient storage resource utilization, as they do not allow users to select the type of disk or location for writing dirty buffers.
Innovation Solution
A modular block allocator with a front-end module for write allocation policies and a back-end module for executing these policies, utilizing a bucket cache system that includes buckets with attributes for specific storage media types or locations, allowing for efficient allocation and storage of dirty buffers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a traditional block allocator is used in a cluster network environment with increasing numbers of CPUs, then the system can handle more processors, but the response time to cleaner messages increases and processing delays occur
Solution Approach 1:
The block allocator is divided into multiple independent allocator instances, each responsible for a specific CPU or processor. Each allocator instance independently manages its own cleaner messages and bucket cache, eliminating the bottleneck of a single centralized allocator handling all processors sequentially. This segmentation allows linear scaling with processor numbers while maintaining constant response time.
Solution Approach 2:
The patent introduces a new dimensional approach by organizing allocators in a distributed architecture across multiple processors rather than a single centralized dimension. Each processor has its own allocator instance, creating a multi-dimensional distribution that parallelizes the handling of cleaner messages across multiple dimensions (processors), thereby reducing response time while scaling adaptability.
2Device complexity
If a traditional block allocator is used, then the system structure is simple, but storage resource utilization is inefficient and users cannot select disk type or location
Solution Approach 1:
The bucket cache is segmented into multiple buckets, each representing a specific storage resource type or location (e.g., different disk types, zones, or partitions). Users can selectively allocate from specific buckets based on their storage requirements, enabling flexible resource selection without complicating the overall allocator structure. Each bucket operates independently with its own allocation policies.
Solution Approach 2:
Different buckets are assigned different qualities or characteristics corresponding to specific storage media types or locations. Each bucket has local properties (such as performance level, durability, or location) that match user requirements. This allows the system to provide differentiated storage resources without requiring a completely complex unified structure.
3Device complexity
If multiple cleaner messages are processed sequentially, then the allocation logic is simple, but processing delays increase in multi-CPU environments
Solution Approach 1:
The processing of cleaner messages is segmented and distributed across multiple allocator instances, each handling messages from a specific CPU independently. This parallelization allows multiple cleaner messages to be processed simultaneously rather than sequentially, dramatically increasing throughput while keeping individual allocation logic simple and manageable.
Solution Approach 2:
By distributing cleaner message processing across multiple concurrent allocator instances, the system achieves continuous processing without idle waiting. Multiple allocation operations can proceed in parallel, ensuring that useful action (processing cleaner messages) continues without interruption or delay, thereby maximizing productivity.
Data Source
AI summary
A modular block allocator receives a cleaner message requesting dirty buffers associated with an inode be cleaned. The modular block allocator provides at least one bucket cache comprising a plurality of buckets, wherein each bucket represents a plurality of free data blocks. The dirty buffers are cleaned by allocating the data blocks of one of the buckets to the dirty buffers. The allocated data blocks are mapped to a stripe set and when the stripe set is full, the stripe set is sent to a storage system. In one embodiment of the invention, a modular block allocator includes a front end module and a back end module communicating with each other via an application programming interface (API). The front end module contains write allocation policies that define how blocks are laid out on disk. The back end module creates data structures for execution of the policies.