Cloud Deduplication Metadata Tiering for Capacity Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Storage appliances face challenges in managing growing data volumes due to limited metadata space, leading to high costs for additional storage and maintenance, while cloud storage offers cost-effective scalability but requires efficient integration with deduplicated systems.
Innovation Solution
A metadata-data separated architecture is employed, where metadata is stored locally and data is stored on the cloud, using detach-attach workflows to scale cloud capacity without additional metadata storage, maintaining performance and compliance with detach-attach operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If cloud storage is used to scale storage capacity, then storage capacity and cost-effectiveness are improved, but metadata storage capacity becomes a limiting factor
Solution Approach 1:
The patent segments cloud units into different accessibility states (immediately accessible, recently accessible, archive) and implements separate metadata storage strategies for each state. This segmentation allows the system to scale cloud storage capacity while managing metadata storage requirements differently for each segment, resolving the contradiction between storage capacity and metadata storage constraints.
Solution Approach 2:
The patent introduces a new dimension of cloud unit accessibility states to organize and manage cloud units. By adding this temporal/accessibility dimension, the system can scale storage capacity across multiple states while maintaining manageable metadata requirements for each state, effectively resolving the metadata storage capacity constraint.
2Speed
If cloud units are made immediately accessible for high performance, then access speed is improved, but metadata storage requirements increase
Solution Approach 1:
The patent applies local quality by providing high-speed access only to cloud units in the immediately accessible state that require frequent access, while archiving less frequently accessed units. This allows the system to optimize access speed for critical operations without incurring metadata storage costs for all cloud units, resolving the contradiction between access speed and metadata storage capacity.
Solution Approach 2:
The patent implements partial action by maintaining full metadata locally only for cloud units in the immediately accessible state, while using different metadata management approaches for other states. This partial metadata storage strategy provides high performance where needed while reducing overall metadata storage requirements.
3Adaptability or versatility
If all cloud units are maintained in read-write state for maximum accessibility, then accessibility is improved, but metadata storage space is quickly exhausted
Solution Approach 1:
The patent implements dynamics by allowing cloud units to transition between different accessibility states (immediately accessible, recently accessible, archive) based on access patterns and system conditions. This dynamic state management enables the system to provide high accessibility when needed while conserving metadata storage space by moving less active units to states with reduced metadata requirements.
Solution Approach 2:
The patent changes the accessibility parameter of cloud units by introducing multiple accessibility states. By changing this parameter dynamically, the system can scale cloud storage capacity while managing metadata storage space through parameter-based organization, resolving the contradiction between accessibility and metadata storage space.
4Adaptability or versatility
If detach-attach workflows are implemented for scaling, then scalability is improved, but system complexity increases
Solution Approach 1:
The patent implements discarding and recovering by detaching cloud units from the immediately accessible state when scaling down or archiving. The cloud units are not completely discarded but moved to other states where their metadata requirements are reduced. This allows scalability while managing system complexity through state-based lifecycle management.
Data Source
AI summary
Cloud units in cloud storage include containers including data containers storing file segments, segment tree containers storing upper-level segments of segment trees representing the files, and cloud containers storing headers from the data and segment tree containers. A header for a data container includes fingerprints identifying the file segments. A header for a segment tree container includes fingerprints identifying the upper-level segments. A storage appliance stores a first level of metadata for each cloud unit in a read-write state, a second level of metadata for each cloud unit in a read-only state, and a third level of metadata for each cloud unit in an offline state. The third level requires less storage than the first and second levels. The second level requires less storage than the first level. The limited metadata space of the appliance is managed by maintaining at least a subset of cloud units in the read-only state.