Deduplicated Cloud Units for Metadata Space Constrained Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Storage appliances face challenges in managing growing data volumes due to limited metadata storage capacity, leading to high costs and inefficiencies in deduplication systems, especially when integrating with cloud storage solutions.
Innovation Solution
A metadata-data separated architecture is employed, where metadata is stored locally and data is stored on the cloud, allowing for detach-attach workflows that scale cloud capacity without additional metadata storage, using detach-attach workflows to manage metadata space constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If cloud storage is integrated with deduplicated storage appliances, then storage capacity is expanded, but metadata storage capacity becomes constrained
Solution Approach 1:
The patent segments metadata into multiple hierarchical levels (first level, second level, third level) with different storage requirements. This segmentation allows the system to manage metadata storage capacity efficiently while supporting expanded cloud storage capacity by storing different levels of metadata in different locations or formats.
Solution Approach 2:
The patent changes the storage state parameter of metadata by maintaining cloud units in different states (read-write, read-only, offline). The third level of metadata requires less storage than the first and second levels, demonstrating parameter change in storage requirements based on access state.
2Quantity of substance
If more storage disks are added to storage appliances, then storage capacity increases, but hardware costs and maintenance costs increase
Solution Approach 1:
The patent extracts data storage from local storage appliances and places it in cloud storage, while retaining metadata management locally. This separation allows capacity expansion without proportionally increasing local hardware and maintenance costs.
3Area of stationary object
If cloud units are detached from the appliance, then metadata space is freed, but data accessibility is reduced
Solution Approach 1:
The patent implements dynamic state management for cloud units, allowing transitions between read-write, read-only, and offline states. This dynamic approach enables flexible metadata space management while maintaining data accessibility at different levels depending on the cloud unit state.
Data Source
AI summary
Data migrated from a deduplicated storage appliance are stored as cloud units at cloud storage. Each cloud unit includes containers including data containers storing segments of files, segment tree containers storing upper-level segments of segment trees representing the files, and cloud containers storing headers from the data and segment tree containers. A header for a data container includes fingerprints identifying the segments of files. A header for a segment tree container includes fingerprints identifying the upper-level segments. The cloud units are maintained in different states of accessibility including read-write, read-only, and offline. Upon detecting that local storage of the appliance does not have space to support a cloud unit in a read-write or read-only state, another cloud unit that is in a first state is selected, the first state being the read-write or read-only state. The selected cloud unit is placed in a second state, different from the first state.