Doubly Mapped RAID for Granular Storage Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage techniques often result in underutilization of storage resources due to the large, inflexible nature of storage groups in ECS systems, leading to inefficient use of processor, network, and storage resources, especially for smaller data sets or slower access scenarios.
Innovation Solution
The implementation of a doubly mapped redundant array of independent nodes (RAID) allows for more granular logical storage groups within a real cluster, enabling efficient use of storage resources by mapping data across multiple extents of real disks and nodes, providing redundancy and flexibility in data storage without loss of access even if nodes fail.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional large storage groups are used in ECS systems, then data redundancy and reliability are improved, but storage resource utilization deteriorates due to underutilization of capacity
Solution Approach 1:
The patent segments the storage group into multiple granular units called extents, where each extent represents a minimal allocable storage unit. This segmentation allows the storage system to allocate only the necessary storage capacity to specific workloads rather than assigning entire large storage groups, thereby improving resource utilization while maintaining redundancy through distributed extent allocation across multiple nodes.
Solution Approach 2:
The patent introduces a new dimension of granularity by creating logical storage groups that operate at the extent level rather than the node or disk level. This dimensional change enables fine-grained control over storage allocation, allowing multiple logical storage groups to share the same physical storage infrastructure with different redundancy configurations, thus optimizing both reliability and resource utilization.
2Loss of energy
If smaller real groups are created to improve storage granularity, then storage resource efficiency is improved, but processor and network resource efficiency deteriorates
Solution Approach 1:
The patent makes the extent a universal allocable unit that can be shared across multiple logical storage groups and workloads. Instead of creating separate small storage groups for each workload, extents can be dynamically allocated and reassigned, allowing the same physical storage resources to serve multiple purposes. This multi-functionality improves storage efficiency without requiring proportional increases in processor and network resources.
Solution Approach 2:
The patent merges multiple logical storage group requirements into a single shared pool of extents. By combining storage resources at the extent level rather than maintaining separate small storage groups, the system reduces overhead and improves processor and network resource efficiency while still providing granular storage allocation capabilities.
3Reliability
If large storage groups are used, then data redundancy is improved, but adaptability to different data storage needs deteriorates
Solution Approach 1:
The patent implements dynamic extent allocation where storage capacity can be flexibly assigned and reassigned based on changing workload requirements. Logical storage groups can be created, modified, or deleted at the extent level without affecting the underlying physical storage infrastructure. This dynamic approach allows the system to adapt to different data storage needs while maintaining redundancy through configurable extent distribution and replication strategies.
Solution Approach 2:
The patent enables parameter changes in storage allocation by allowing logical storage groups to specify different redundancy levels, extent sizes, and allocation policies. These parameter changes can be made without reconfiguring the entire storage infrastructure, providing adaptability to different data storage needs while maintaining data redundancy through configurable parameters rather than fixed group structures.
Data Source
AI summary
A doubly mapped redundant array of independent nodes (doubly mapped RAIN) for data storage is disclosed. A doubly mapped RAIN cluster can be allocated on top of one or more real data clusters. The real data cluster can be N nodes by M disks by L extents. A doubly mapped RAIN cluster can be N′ nodes by M′ disks by L′ extents, where N′ is less than, or equal to N. Mapping of data storage locations in a doubly mapped RAIN cluster can facilitate use of a real cluster at a different granularity than in either a conventionally administered real cluster or in a singly mapped RAIN. The topology of a doubly mapped RAIN can be constrained via rules related to correlation of real disks to mapped nodes, correlation of real extents to mapped nodes, correlation or real extents to mapped disks, etc.