Object Storage Volume Service With POSIX Namespace Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object storage resources in cloud environments do not natively support folder-type data structures, leading to performance issues during rename or delete operations due to the lack of folder relationship tracking, resulting in thousands of additional operations.
Innovation Solution
A system providing filesystem access with support for POSIX standards, which virtualizes a hierarchical namespace by mapping paths to folders or files to immutable identifiers stored in a metadata layer, allowing efficient rename and delete operations independently of underlying folders/files.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of stationary object
If object storage is used to store data without native folder support, then storage scalability is improved, but rename and delete operation performance deteriorates due to triggering thousands of additional operations
Solution Approach 1:
The patent introduces a metadata layer as an intermediary between the hierarchical filesystem interface and the flat object storage backend. This metadata layer maintains path-to-identifier mappings and folder relationship information, enabling efficient rename and delete operations by updating metadata rather than traversing thousands of objects, while still providing POSIX-compatible filesystem access and maintaining scalability.
Solution Approach 2:
The patent segments the storage system into two independent parts: a hierarchical namespace layer (metadata layer) that handles folder structures and path mappings, and a flat object storage layer that provides scalable storage. This segmentation allows the metadata layer to manage rename/delete operations efficiently while the object storage layer maintains scalability, resolving the contradiction between operation performance and storage capacity.
2Device complexity
If object storage is used without native folder support, then storage simplicity is improved, but filesystem access compatibility deteriorates due to lack of POSIX support
Solution Approach 1:
The metadata layer serves as an intermediary that translates POSIX filesystem operations into object storage operations. It maintains a hierarchical namespace with folder and file paths, providing POSIX-compatible access patterns while backing them with flat object storage. This enables clients to access data using standard filesystem calls without requiring changes to their applications.
Solution Approach 2:
The patent creates a virtual copy of the hierarchical filesystem structure in the metadata layer that mirrors the actual flat object storage layout. This virtual hierarchy copies the folder and path relationships needed for POSIX compatibility, allowing clients to interact with a familiar filesystem interface while the underlying storage remains simple and scalable.
Data Source
AI summary
Embodiments described herein are generally related to cloud computing environments, such as cloud infrastructure or data analytics environments, and are particularly directed to systems and methods providing an object storage based volume service, with support for filesystem access. In accordance with an embodiment, filesystem access can be provided in accordance with Portable Operating System Interface (POSIX) standards, for storage and use of data within the cloud computing environment. The system can virtualize a hierarchical namespace for use with mounted filesystem paths on client machines, wherein paths to folders or individual files are mapped to immutable identifiers (IDs) are stored in a metadata layer as path-to-IDS mappings. Operations such as folder or file renames or deletes can be performed efficiently and independently of the folders/files below, by respectively updating a new name for an immutable ID, or by removing the immutable ID from the path-to-ID mapping.