Mapped RAIN Storage for Heterogeneous Node Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage techniques face inefficiencies in utilizing storage resources, particularly in bulk data storage systems where large storage groups often result in underutilization of nodes with many disks, and apportioning smaller groups can be inefficient in terms of processor and network resources.
Innovation Solution
The implementation of a mapped redundant array of independent nodes (RAIN) allows for a logical arrangement of real storage devices, enabling more granular use of storage resources by creating smaller logical storage groups within larger real groups, providing data redundancy and allowing for the removal or addition of nodes without data loss, and distributing data across multiple nodes for increased availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If large storage groups are formed with many nodes and disks, then total storage capacity is improved, but storage utilization efficiency deteriorates due to underutilization of nodes
Solution Approach 1:
The patent divides a large real storage group into multiple smaller logical storage groups (mapped clusters). Each logical storage group is a subset of the real storage group, allowing different logical groups to have different sizes and compositions. This segmentation enables efficient utilization of storage resources by matching logical group sizes to actual storage needs, preventing underutilization of nodes while maintaining total storage capacity.
2Productivity
If smaller storage groups are created to improve storage utilization, then storage efficiency is improved, but processor and network resource efficiency deteriorates
Solution Approach 1:
The patent creates a hierarchical structure where logical storage groups are mapped onto a shared real storage group. Multiple logical storage groups can share the underlying real storage resources, allowing processor and network resources to be efficiently utilized across multiple logical groups simultaneously. This multi-functionality enables smaller logical groups to benefit from the aggregated resources of the larger real group, maintaining resource efficiency while improving storage utilization.
3Device complexity
If homogeneous storage devices are used, then system simplicity is improved, but storage space efficiency deteriorates due to inability to efficiently manage devices of different capacities
Solution Approach 1:
The patent introduces a mapping mechanism that abstracts the heterogeneity of storage devices with different capacities. The mapping layer translates logical storage requests into physical storage allocations, dynamically adjusting parameters such as block sizes and allocation strategies based on the specific characteristics of underlying storage devices. This parameter adaptation enables efficient utilization of heterogeneous storage resources while presenting a simplified homogeneous interface to users.
Data Source
AI summary
A mapped redundant array of independent nodes (mapped RAIN) for data storage is disclosed. A mapped RAIN cluster can be allocated on top of one or more real data clusters, wherein the real clusters can comprise storage devices of different storage capacities. Mapping of data storage locations for a mapped RAIN cluster to real storage devices can be based on an affinity value determined for pairs of real nodes of the real data clusters. A normalized affinity can be employed to enable allocation of real storage to mapped nodes of mapped clusters that can be based on the heterogeneous capacities of the storage devices. This can provide improved data availability and data recovery over other techniques where heterogeneity of hardware can make efficient resource allocation a non-trivial task. The disclosed subject matter can facilitate more efficient allocation of Mapped RAINs in a heterogeneous cluster storage construct.