Shared-Everything Storage Layout for Zero-Touch RAID Grouping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage architectures with a one-to-one controller arrangement face challenges in scalability, cost, and redundancy, as adding or replacing controllers requires upgrading associated storage devices, leading to increased costs and downtime, and redundancy is limited by a single controller scheme.
Innovation Solution
Implementing a shared-everything architecture with multiple controllers and storage devices organized into redundancy groups, allowing any controller to access any storage device, and automating the process of adding new devices to optimal RAID groups without manual intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a one-to-one controller architecture is used, then each controller can be dedicated to specific storage devices, but adding or replacing controllers requires upgrading associated storage devices which increases cost and downtime
Solution Approach 1:
The patent implements a shared-everything architecture where controllers can access any storage device in the aggregate rather than being dedicated to specific devices. This multi-functionality allows controllers to operate with multiple storage devices and enables flexible reconfiguration when devices are added or replaced, eliminating the requirement to upgrade storage devices when controllers change.
Solution Approach 2:
The storage aggregate is divided into multiple redundancy groups that can be independently managed. When storage devices are added or replaced, the system segments the aggregate into manageable groups and automatically rebalances data across groups, allowing incremental updates without requiring complete system reconfiguration or upgrades.
2Adaptability or versatility
If a one-to-one controller architecture is used, then controller-to-storage mapping is simple, but scalability is limited as compute power cannot be increased without adding storage devices
Solution Approach 1:
The shared-everything architecture enables any controller to access any storage device, providing universal access that decouples compute resources from storage resources. This allows independent scaling of controllers to increase compute power without requiring proportional increases in storage devices, as controllers can dynamically access any available storage capacity in the aggregate.
Solution Approach 2:
The system dynamically reconfigures storage assignments based on current controller capabilities and storage availability. When new controllers are added, the system automatically detects them and redistributes storage tasks across the expanded controller pool, enabling seamless scalability without fixed controller-to-storage mappings.
3Reliability
If redundancy groups with parity disks are used, then data redundancy is improved, but all storage devices in a group fail together when issues occur due to shared metadata and single controller scheme
Solution Approach 1:
The storage aggregate is divided into multiple independent redundancy groups with separate metadata structures. This segmentation ensures that failures in one group do not propagate to other groups, as each group has isolated metadata and can be managed independently by different controllers, preventing cascading failures across the entire storage system.
Solution Approach 2:
The patent introduces a metadata service as an intermediary that manages metadata for multiple redundancy groups independently. This intermediary layer allows different controllers to access and manage different groups' metadata without interference, enabling parallel recovery operations across multiple groups and reducing overall system downtime when failures occur.
4Extent of automation
If manual intervention is required for adding storage devices, then layout management is precise, but the process requires time and processing capacity
Solution Approach 1:
The system implements self-service automation where the storage management infrastructure automatically detects newly added storage devices, evaluates their characteristics, and assigns them to optimal redundancy groups without human intervention. The automated layout management analyzes device properties such as capacity, speed, and form factor to make intelligent placement decisions, matching the precision of manual configuration while eliminating the time and processing overhead.
Data Source
AI summary
The disclosure describes systems, devices, and methods for managing access to storage devices in a shared-everything data storage environment in which any controller can access each storage device of a storage aggregate. In an implementation, a method for managing the layout of the storage aggregate is provided, which may be performed by a controller. The controller receives a request to add a storage device to the data storage environment, processes the request to identify metadata associated with the storage device, including characteristics of the storage device, processes characteristics of the storage device and characteristics of redundancy groups in the storage environment to select a redundancy group for the drive, and adds the storage device to the redundancy group.