Storage Rebuilds Using Non-Redundant Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in efficiently managing data storage and retrieval, particularly in large-scale storage platforms, where data longevity and storage device management become complex issues.
Innovation Solution
The proposed solution involves a storage system architecture that utilizes multiple indirection unit (IU) sizes for efficient data storage and retrieval. This architecture includes a storage system controller that generates commands with IU sizes and a storage device controller that maps data using multiple IU sizes, optimizing storage and retrieval operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional storage systems use redundant data for device rebuilds, then data reliability is maintained, but storage efficiency and rebuild time are reduced
Solution Approach 1:
The system performs preliminary actions by maintaining metadata that tracks the original location and characteristics of non-redundant data before device failures occur. This pre-established metadata information enables rapid identification and retrieval of suitable replacement data during rebuild operations, eliminating the need to scan entire storage systems and significantly accelerating the rebuild process while maintaining data reliability.
Solution Approach 2:
The patent introduces metadata as an intermediary layer between the physical storage devices and the control logic. This metadata acts as a mediator that stores information about non-redundant data locations, device characteristics, and data provenance, enabling the system to make intelligent decisions about data replacement during rebuilds without requiring direct examination of all stored data, thus improving both efficiency and reliability.
2Productivity
If storage systems use uniform indirection unit sizes, then device compatibility is improved, but storage efficiency and retrieval performance are reduced
Solution Approach 1:
The patent segments the storage address space into multiple indirection unit (IU) sizes, allowing different portions of data to be addressed with optimally-sized IUs based on their access patterns and characteristics. This segmentation enables the system to achieve high storage efficiency for various data types while the control plane manages the complexity of coordinating multiple IU sizes across different devices.
Solution Approach 2:
The system adds a new dimension to storage addressing by introducing multiple IU sizes (e.g., 4KB, 8KB, 16KB) rather than relying on a single address dimension. This multi-dimensional addressing approach allows the system to optimize storage efficiency for different data workloads while distributing the complexity management across the control plane rather than requiring complex hardware logic in each device.
3Ease of operation
If data with different lifespans are stored together, then storage space utilization is maximized, but data management and retrieval efficiency are reduced
Solution Approach 1:
The system applies local quality by associating metadata with specific data segments that captures their individual lifespan characteristics, access patterns, and priority levels. This enables the control plane to manage different data regions with different policies appropriate to their specific requirements, improving data management efficiency while the metadata system tracks space utilization across all regions to maximize overall storage capacity usage.
Data Source
AI summary
A failure of a particular storage device is detected where at least a portion of data stored in the particular storage device is readable after the failure. In response to detecting the failure of the particular storage device, one or more copies of the portion of the data that is readable are stored in one or more other storage devices. An addition of a replacement storage device to the storage system that replaces the particular storage device is detected. The one or more copies of the portion of the data that is readable are stored in the replacement storage device.