Cloud Tier Proxy Segment Sampling for Distributed Storage Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current tiering approaches in distributed storage systems are limited in their ability to manage data across multiple cloud storage providers, leading to higher latency and costs due to the need to check every segment for determining the state of a data object, which is inefficient and costly, especially in disaster recovery scenarios.
Innovation Solution
The system employs a cloud tier proxy that samples a subset of segments to approximate the state of a data object, reducing latency and costs by determining the most restrictive state, which is then used to manage data storage and retrieval across multiple cloud storage providers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If every segment is checked to determine the state of a data object, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The patent applies partial action by sampling only a subset of segments (e.g., one segment per cloud provider) rather than checking every segment to determine data object state. This reduces the time and computational resources required while maintaining sufficient accuracy for disaster recovery decisions. The sampled segments provide representative information about the overall data object state across multiple cloud providers.
2Measurement precision
If every segment is checked to determine the state of a data object, then measurement precision is improved, but use of energy increases
Solution Approach 1:
The system performs partial verification by checking only a sampled subset of segments rather than all segments. This significantly reduces the computational energy required for determining data object state while maintaining adequate precision for disaster recovery operations. The energy cost is reduced proportionally to the sampling ratio.
3Loss of energy
If data is tiered to a single cloud storage provider, then cost is reduced, but reliability worsens
Solution Approach 1:
The patent segments data objects into multiple segments distributed across different cloud storage providers. This allows the system to tier data to multiple providers simultaneously, improving disaster recovery reliability while managing costs. Each cloud provider stores a portion of the data, and the system can retrieve data from any available provider.
Solution Approach 2:
The system implements multi-functionality by enabling data tiering to multiple cloud storage providers through a unified interface. The data object can be accessed from any of the tiered locations, providing both cost benefits through selective tiering and reliability benefits through multi-provider redundancy. The system automatically manages the complexity of multi-provider access.
Data Source
AI summary
A system, method, and machine-readable storage medium for analyzing a state of a data object are provided. In some embodiments, the method includes receiving, at a storage device, a metadata request for the data object from a client. The data object is composed of a plurality of segments. The method also includes selecting a subset of the plurality of segments and obtaining a segment state for each segment of the subset. Each segment state indicates whether the respective segment is accessible via a backing store. The method further includes determining a most restrictive state of the one or more segment states and sending state information to the client in response to the metadata request, the state information being derived from the most restrictive state.