Unified Namespace Metadata Coordination for Native Protocol Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current multi-protocol storage solutions struggle to provide a unified storage solution that allows native access to shared data across different data access protocols like NFS, SMB, and S3, due to incompatible native semantics and operations, often compromising on protocol features to ensure compatibility.
Innovation Solution
A unified namespace technique that maintains semantically equivalent metadata separately for each protocol, using a control plane infrastructure and a message bus to coordinate metadata changes across protocol stacks, ensuring eventual consistency and native access while allowing protocol stacks to remain unmodified.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If protocol gateways or on-the-fly filtering/mapping are used to access shared data across protocols, then data accessibility across protocols is improved, but protocol features are compromised due to using lowest common denominator semantics
Solution Approach 1:
The patent segments the storage system into separate protocol-specific namespaces (NFS namespace, SMB namespace, S3 namespace) that can independently maintain their native features and semantics. Each namespace is managed by a dedicated storage service that preserves full protocol functionality while sharing underlying data through a unified data namespace.
Solution Approach 2:
The patent introduces a control plane with a unified data namespace as an intermediary layer between protocol-specific namespaces. This mediator enables cross-protocol data access through event notification systems and metadata synchronization without requiring protocol gateways that would compromise native features.
2Adaptability or versatility
If a single unified storage solution is implemented to allow native access to shared data across protocols, then data sharing capability is improved, but system complexity increases due to maintaining multiple protocol stacks
Solution Approach 1:
The patent merges multiple protocol-specific storage services into a single unified storage system that shares common underlying data. The control plane consolidates metadata management and event notification mechanisms, while each protocol stack operates independently with its own namespace, achieving unity in data management without sacrificing protocol independence.
Solution Approach 2:
The unified data namespace and control plane infrastructure serve multiple protocol stacks simultaneously, providing universal data access and metadata synchronization capabilities. The event notification system and metadata store are designed to work with any data access protocol, making the core infrastructure multi-functional and protocol-agnostic.
3Reliability
If metadata is shared across protocol stacks, then data consistency is improved, but performance degradation occurs due to coordination overhead
Solution Approach 1:
The patent implements preliminary action by pre-processing metadata changes in the source protocol stack before propagation. Event notifications are generated and queued in advance, allowing target protocol stacks to process metadata updates asynchronously without blocking operations. Metadata is cached locally in each namespace to reduce repeated access to the central metadata store.
Solution Approach 2:
The patent uses copying by creating local copies of metadata in each protocol-specific namespace rather than sharing the same metadata structures. Each namespace maintains its own metadata store with protocol-specific semantics, synchronized through event notifications. This eliminates coordination locks and allows parallel metadata operations across different protocols.
Data Source
AI summary
A unified namespace technique provides coherent access to unstructured data across different data access protocols having different logical constructs that are stored and managed on a storage system. A control plane infrastructure operates in connection with storage services to provide support for a vast array of storage platforms including file servers of a file system and object storage servers of an object store. Metadata associated with a data access transaction is processed separately and natively by a protocol stack of a particular storage service according to a particular data access protocol. The processed metadata is stored native to the access protocol in a metadata store associated with the particular storage service and is made available to the protocol stacks of the other storage services. Processed metadata is made available to the protocol stacks via an event notification logging service implemented as a message bus. A single canonical instance of the data is maintained for all of the logical constructs served by the storage system.