Cloud Object Data Layout for Storage Snapshot and Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Object storage technologies face challenges in efficient storage management, particularly in facilitating scalable and flexible data layouts that enable effective storage snapshot management and deduplication operations, lacking the performance of block level storage and the simplicity of file level storage.
Innovation Solution
The implementation of a heterogeneous replication system that converts file level and block level storage data into object storage, using metadata objects and data objects to represent file containers and data units, allowing for efficient snapshot management and deduplication by creating object keys and mapping file identifiers to object keys, enabling seamless access and management across different storage technologies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If object storage is used with flat namespace and flexible container structure, then scalability and flexibility are improved, but storage management efficiency deteriorates
Solution Approach 1:
The patent segments data into distinct metadata objects and data objects. Metadata objects store information about files and folders including their hierarchical relationships, while data objects store actual data content. This segmentation enables efficient storage management operations like snapshots and deduplication while maintaining the flexibility of object storage.
Solution Approach 2:
The patent introduces metadata objects as intermediaries between the flat namespace structure and the actual data objects. These metadata objects maintain hierarchical relationships and enable efficient navigation and management operations without compromising the scalability of the flat namespace architecture.
2Adaptability or versatility
If object storage is used instead of block level storage, then scalability and flexibility are improved, but performance deteriorates
Solution Approach 1:
By segmenting data into data objects that can be independently accessed and metadata objects that manage relationships, the system enables efficient access patterns. Data objects can be retrieved without traversing complex inode structures, improving access speed while maintaining scalability.
3Speed
If object storage is used instead of file level storage, then performance is improved, but simplicity deteriorates
Solution Approach 1:
The patent divides the storage system into two clear segments: metadata objects that handle hierarchical relationships and data objects that store content. This segmentation simplifies access operations by separating metadata resolution from data retrieval, making the system easier to manage despite the object storage architecture.
4Adaptability or versatility
If data is stored as objects in flat namespace, then adaptability is improved, but storage management operations become complex
Solution Approach 1:
Metadata objects serve as intermediaries that encode hierarchical relationships within the flat namespace. These intermediaries maintain folder and file relationships without requiring complex namespace traversal, simplifying storage management operations while preserving flexibility.
Data Source
AI summary
An enterprise may use file level and block level storage for operational access to reap the performance benefits, and use object storage for back-up storage and scale out. Data of objects and objects themselves can be organized to allow for efficient storage management operations, including storage snapshot operations and deduplication operations. A root file container can be represented in object storage with a metadata object (“file container object”) and child file container can be represented in object storage with a metadata object (“file group object”). Each file can be represented in object storage with a metadata object that indicates the data units of the file depending upon the storage technology hosting the file (e.g., a file data block in file level storage). The data units of the file can be represented in object storage as elements of a data object.