Heterogeneous Media Storage Organization with Sparse Indexing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network computing and storage systems face challenges in optimizing data performance and integrity, particularly in efficiently storing and retrieving large quantities of data while ensuring durability and availability, especially when dealing with failures and data loss.
Innovation Solution
The implementation of redundancy coding techniques, such as erasure codes, to store original data across multiple volumes, with failure-decorrelated subsets and sparse indexing, allowing for efficient data reconstruction and retrieval, even in the presence of failures, using inventory holders and mobile drive units for physical storage and retrieval of data storage devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored across multiple volumes using redundancy coding, then data durability and availability are improved, but system complexity increases
Solution Approach 1:
The patent segments data into multiple shards distributed across different volumes and storage devices. Each shard is independently stored, and the system uses redundancy coding to ensure that data can be reconstructed even if some shards are lost. This segmentation approach improves durability by spreading risk across multiple independent storage units while managing complexity through systematic organization.
Solution Approach 2:
The patent introduces an intermediary indexing layer that maps logical data identifiers to physical storage locations. This intermediary structure abstracts the complexity of distributed storage from users and applications, allowing them to access data without needing to understand the underlying redundancy and distribution mechanisms. The index acts as a mediator between the simple user interface and the complex distributed storage system.
2Reliability
If data is stored across multiple volumes using redundancy coding, then data availability is improved, but retrieval efficiency deteriorates
Solution Approach 1:
The patent performs preliminary actions by pre-computing and storing index information that maps data logical identifiers to their physical locations across distributed volumes. When data retrieval is needed, the system uses these pre-computed indexes to directly locate the required shards without needing to search through multiple volumes sequentially. This preliminary indexing action maintains high retrieval efficiency while enabling data to be distributed across multiple volumes for improved availability.
3Quantity of substance
If sparse indexing is used for data organization, then storage efficiency is improved, but data retrieval complexity increases
Solution Approach 1:
The patent applies sparse indexing by creating index entries only for specific, strategically selected data locations rather than maintaining continuous indexing across all storage space. This local quality approach - where indexing is concentrated at key points rather than uniformly distributed - improves storage efficiency by reducing index overhead while maintaining the ability to retrieve data through these localized index points. The system manages retrieval complexity by ensuring these sparse index points are strategically positioned to enable efficient data location.
Data Source
AI summary
Techniques and systems for storing and retrieving data storage devices of a data storage system are disclosed. In some embodiments, inventory holders are used to store data storage devices used by a data storage system. When data is to be transacted with the data storage devices, mobile drive units locate appropriate inventory holders and transport them to a device reading station, where an appropriate device retrieval unit transacts the data. In some embodiments, each inventory holder includes a heterogenous mix of data storage device types, the layout of which may be calculated according to the specific mix allocated to a given inventory holder. After the data has been transacted, the data storage devices are returned to the appropriate inventory holders, and the inventory holders are placed by the mobile drive units in locations where they may be accessed in response to further data transactions.