Dynamic Storage Strategy for Distributed Object Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current distributed object storage systems face inefficiencies in handling small data objects and data fragments of large objects, as traditional erasure encoding technologies incur high latency and storage overhead, especially when dealing with concurrent failures of storage elements.
Innovation Solution
A distributed object storage system that dynamically adjusts its storage and retrieval methods based on data item size, using replication for small items, erasure coding for medium-sized items, and segmenting large items into sections for efficient storage and retrieval, while maintaining redundancy and flexibility across multiple storage elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If erasure encoding is used for all data objects to ensure fault tolerance, then reliability is improved, but latency increases significantly for small data objects
Solution Approach 1:
The patent changes the parameter of encoding method based on data object size. For small data objects (below threshold), replication is used instead of erasure encoding, thereby avoiding the high latency of encoding/decoding operations while maintaining adequate fault tolerance through multiple copies.
Solution Approach 2:
Different storage strategies are applied to different portions of data based on their size characteristics. Small data objects receive replication treatment while large data objects receive erasure encoding treatment, optimizing the balance between reliability and latency for each local case.
2Reliability
If replication is used to survive concurrent failures, then reliability is improved, but storage overhead increases significantly
Solution Approach 1:
The patent changes the redundancy parameter dynamically: for small data objects, full replication is used to ensure survival of concurrent failures; for large data objects, erasure encoding is used which provides equivalent fault tolerance with reduced storage overhead.
Solution Approach 2:
The redundancy scheme is made dynamic rather than static. The system adapts the redundancy method (replication vs. erasure encoding) based on the size of the data object, allowing optimal trade-off between reliability and storage efficiency for different data types.
3Quantity of substance
If erasure encoding is used for large data objects, then storage overhead is reduced, but retrieval latency increases when only partial data is needed
Solution Approach 1:
The patent segments large data objects into smaller chunks before applying erasure encoding. This allows the system to retrieve only the necessary segments when partial data is needed, avoiding the need to decode and transfer the entire encoded data set, thereby reducing retrieval latency while maintaining storage efficiency.
4Device complexity
If a single storage strategy is used for all data sizes, then system complexity is reduced, but performance optimization is limited
Solution Approach 1:
The patent introduces dynamic selection of storage strategies based on data object size thresholds. The system automatically chooses between replication and erasure encoding methods, providing performance optimization without requiring complex manual configuration or reducing system adaptability.
Data Source
AI summary
A method and computer device for storage and retrieval of a data object on a storage medium. The method includes steps of disassembling the data object into a predetermined number of redundant sub blocks, storing the redundant sub blocks on the storage medium, retrieving at least a predetermined multiple of a minimal spreading requirement of the redundant sub blocks from the storage medium, and assembling the data object from any combination of a particular number of the redundant sub blocks, the particular number corresponding to a predetermined multiple of a minimal spreading requirement. The computer device includes modules for performing the steps.