Cloud Object Metadata Attachment for Hybrid Storage Scalability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Primary storage systems' robust features like data replication, deduplication, encryption, and snapshot management are incompatible with cloud storage environments, leading to loss of functionality when used in hybrid setups, resulting in increased costs and reduced scalability.
Innovation Solution
The system provides methods to attach metadata to cloud objects, enabling pseudo read-only snapshots, defragmentation, and garbage collection without altering in-use data, allowing for flexible and scalable management of cloud storage while maintaining data consistency and offloading computationally expensive processes to cloud services.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If primary storage systems are used for data storage and management, then robust functionality including data replication, deduplication, encryption, and snapshot management is provided, but cost increases and scalability decreases compared to cloud storage
Solution Approach 1:
The system segments storage functionality into two parts: primary storage systems handle robust data management operations (replication, deduplication, encryption, snapshot creation), while cloud storage handles scalable storage. This segmentation allows each component to operate in its optimal environment, with the primary storage system providing control plane services and the cloud providing flexible capacity.
Solution Approach 2:
The patent introduces an intermediary layer that translates between primary storage system operations and cloud storage capabilities. This intermediary enables features like snapshot management and data replication to work across the hybrid environment by mediating between the two different storage paradigms.
2Reliability
If primary storage systems are used for data storage, then robust features like data replication, deduplication, encryption, and snapshot management are provided, but these features are incompatible with cloud storage environments, resulting in loss of functionality
Solution Approach 1:
The system changes the operational parameters of cloud storage objects to simulate primary storage system behavior. By attaching metadata to cloud objects and implementing pseudo read-only modes, the system makes cloud storage compatible with traditional primary storage operations like snapshots and replication without requiring full feature parity.
Solution Approach 2:
The patent creates simplified copies of primary storage system functionality that work within cloud storage constraints. Instead of implementing full primary storage features in the cloud, it creates copy-like structures (snapshots as read-only markers, replicated objects with metadata) that provide equivalent functionality through different mechanisms.
3Adaptability or versatility
If cloud storage is used for scalable and cost-effective storage, then scalability and cost savings are achieved, but robust data management features like compression and deduplication are lost
Solution Approach 1:
The system performs data management operations like deduplication and compression in advance, before data is transferred to cloud storage. The primary storage system processes data locally using its robust functionality, then stores the already-optimized data in the cloud, eliminating the need for these features to be implemented in the cloud environment.
Solution Approach 2:
The patent implements feedback mechanisms where the primary storage system monitors cloud storage usage and performance, then adjusts data management strategies accordingly. This feedback loop ensures that data is replicated and managed in the primary system with full functionality, while cloud storage provides complementary scalable capacity.
Data Source
AI summary
Techniques are provided for managing objects within an object store. An object is maintained within an object store. In an embodiment, a rule is enforced for the object that in-use slots of the object are non-modifiable and unused slots of the object are modifiable. Metadata of additional information for a slot within the object is attached to the object header. A first application allowed to access user data within the slot is provided access to the user data without being provided access to the metadata. A second application allowed access to the user data and the additional information is provided with access to the user data and the metadata for identifying a location of additional information within the object.