Object Storage Subsystems Metadata Paging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Enterprise data storage systems face challenges in managing large metadata sets efficiently, leading to increased costs due to the need for costly high-performance volatile memory to store all metadata, which can be mitigated by implementing metadata paging using less expensive backing storage devices like SSDs.
Innovation Solution
The system employs a plurality of storage director devices that access object storage subsystems in a global memory, utilizing descriptive data structures and inodes to manage metadata locations and availability across multiple physical storage devices, allowing only a subset of metadata to be present in volatile memory, thereby reducing the need for costly RAM.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If all metadata is stored in volatile memory to enable efficient access, then metadata access speed is improved, but system cost increases due to the need for large amounts of expensive RAM
Solution Approach 1:
The patent segments metadata into two categories: frequently accessed metadata stored in volatile memory and less frequently accessed metadata stored in non-volatile memory. This segmentation allows the system to maintain fast access for critical metadata while reducing overall RAM requirements by storing less frequently accessed metadata in cheaper storage media.
Solution Approach 2:
The patent introduces a metadata cache mechanism that acts as an intermediary between the host and the backing storage device. The cache stores frequently accessed metadata in volatile memory, while less frequently accessed metadata resides in non-volatile storage. This intermediary layer maintains efficient access patterns while reducing the total amount of expensive RAM required.
2Quantity of substance
If metadata is paged to non-volatile storage to reduce RAM requirements, then system cost is reduced, but metadata access time increases due to slower storage retrieval
Solution Approach 1:
The patent implements a metadata cache that pre-loads and stores frequently accessed metadata in volatile memory before it is actually needed. This preliminary action ensures that when metadata access is required, the data is already in fast memory, eliminating access delays and maintaining performance while allowing less frequently accessed metadata to reside in non-volatile storage.
Solution Approach 2:
The patent applies different storage qualities to different portions of metadata based on access patterns. Frequently accessed metadata is maintained in high-speed volatile memory with optimized access characteristics, while less frequently accessed metadata is stored in non-volatile storage. This local differentiation of storage quality optimizes overall system performance while reducing costs.
3Productivity
If a hierarchical metadata structure is implemented to manage large metadata sets, then metadata management efficiency is improved, but device complexity increases
Solution Approach 1:
The patent implements a hierarchical metadata structure organized in multiple levels, where metadata is arranged in parent-child relationships across different tiers. This nested organization allows efficient management of large metadata sets by grouping related metadata together and enabling hierarchical traversal patterns that improve access efficiency while maintaining manageable complexity through structured organization.
Data Source
AI summary
A storage array uses paged metadata. Each storage director has access to a plurality of object storage subsystems in a global memory. The object storage subsystems describe locations of paged metadata in backing storage. Each object storage subsystem includes a unique identifier, a pointer to a descriptive data structure that indicates availability of free space for paged metadata for that object storage subsystem, and different types of inodes which describe objects in backing storage. The object storage subsystems are used to locate metadata for loading into global memory, and creation and deletion of objects.