Mapped RAIN Storage Routing for Granular Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage techniques often result in underutilization of storage capacity in large node clusters, leading to inefficient use of processor and network resources, especially when dealing with smaller data sets, as all disks in a node are considered part of a single group, making it desirable to have more granular logical storage groups that can utilize portions of larger real groups efficiently.
Innovation Solution
The implementation of a mapped redundant array of independent nodes (RAIN) system, which allows for a logical arrangement of real storage devices into mapped clusters that provide data redundancy and flexibility in node topology, enabling granular storage allocation and efficient use of resources by allowing access to more granular storage levels without loss of data even if nodes fail, and enabling the addition or removal of nodes without disrupting access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If all disks of nodes are considered part of a single large storage group, then the storage capacity is maximized, but the storage granularity becomes too coarse leading to underutilization when storing smaller data sets
Solution Approach 1:
The patent segments the large real storage group into multiple virtual storage groups through virtualization. Each virtual storage group can be independently allocated to different users or applications, enabling fine-grained storage allocation while maintaining efficient utilization of the underlying large storage capacity. The virtual storage groups are formed by dividing the real storage resources into smaller logical units that can be dynamically assigned.
2Adaptability or versatility
If smaller groups of nodes and disks are used, then storage granularity is improved, but processor and network resources are inefficiently utilized
Solution Approach 1:
The patent creates virtual storage groups that can serve multiple purposes and users simultaneously. The same physical storage infrastructure supports multiple virtual storage groups of different sizes, enabling the system to handle diverse storage requirements (small and large datasets) while maintaining high resource utilization. This multi-functional approach allows processor and network resources to be efficiently allocated across different workloads.
3Adaptability or versatility
If a mapped RAIN system is implemented for granular storage allocation, then storage flexibility and resource efficiency are improved, but system complexity increases
Solution Approach 1:
The patent introduces a virtualization layer as an intermediary between the physical storage infrastructure and users. This virtualization layer manages the mapping between virtual storage groups and real storage resources, handling the complexity of granular allocation internally while presenting simplified interfaces to users. The intermediary abstracts the complex Mapped RAIN architecture details, allowing flexible storage allocation without exposing system complexity to end users.
Data Source
AI summary
Disk access event control for mapped nodes of a cluster storage system supporting a redundant array of independent nodes (mapped RAIN) system is disclosed. A mapped RAIN cluster can be allocated on top of one or more real data clusters. In an embodiment, disk access events can be routed via a storage service instance supporting a mapped node. In another embodiment, disk access events can be routed via another storage service instance that does not support the mapped node. Routing the disk access event via another storage service instance that does not support the mapped node can reduce the use of computing resources. Further, the routing of the disk access event can be according to a proportional disk operation value determined based on historical disk access event characteristics.