Metadata Subsystem for Distributed Object Store Navigation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional network storage systems face inefficiencies due to hierarchical organization, which limits data management, navigation, and retrieval, and lacks flexible policy application across data objects, leading to duplication and administrative overhead.
Innovation Solution
A network storage system with a distributed object store and a metadata subsystem that allows location-independent addressing of data objects, enabling efficient search and retrieval through user-specified queries across multiple categories of metadata, and supports flexible schema storage for metadata attributes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If hierarchical organization (path names) is used for data storage, then navigation and retrieval are simplified, but data management flexibility and policy application capability deteriorate
Solution Approach 1:
The patent segments the storage system into two independent components: a hierarchical namespace for navigation and retrieval, and a flat object store with unique identifiers for flexible policy application. This segmentation allows each component to optimize for its specific function without compromising the other.
Solution Approach 2:
The patent introduces an intermediary mapping mechanism that connects the hierarchical namespace to the flat object store. This intermediary layer translates path names to unique identifiers, enabling both hierarchical navigation and flexible policy application to coexist without direct conflict.
2Ease of operation
If path names are tied to storage location, then navigation is straightforward, but data movement across storage tiers becomes complex and inefficient
Solution Approach 1:
The patent extracts the location information from the path name structure and stores it separately in the mapping mechanism. This allows the path name to remain unchanged during data movement, while the mapping is updated to reflect the new storage location, thereby enabling efficient data movement without affecting navigation.
Solution Approach 2:
The mapping mechanism serves as an intermediary that decouples the path name from the physical storage location. When data moves across storage tiers, only the mapping needs to be updated, not the path name itself, thus maintaining navigation simplicity while improving data movement efficiency.
3Stability of the object's composition
If hierarchical organization is used, then data structure is well-defined, but duplication of content and administrative overhead increase
Solution Approach 1:
The patent merges multiple path names that point to the same physical object in the flat object store. The mapping mechanism consolidates references, so that different hierarchical paths can refer to the same underlying data object, thereby reducing content duplication while maintaining a well-defined data structure.
4Ease of manufacture
If conventional storage systems are used, then implementation is straightforward, but metadata management and search capability are limited
Solution Approach 1:
The patent adds a new dimension to the storage system by introducing a separate metadata layer that operates independently from the hierarchical namespace. This metadata layer provides enhanced search and management capabilities without complicating the basic implementation, as it operates in a separate dimension from the traditional file structure.
Data Source
AI summary
A network storage server system includes a distributed object store and a metadata subsystem. The metadata subsystem stores metadata relating to the stored data objects and allows data objects to be located and retrieved easily via user-specified search queries. It manages and allows searches on at least three categories of metadata via the same user interface and technique. These categories include user-specified metadata, inferred metadata and system-defined metadata. Search queries for the metadata can include multi-predicate queries.