Distributed Object Storage Spreading Requirement Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current distributed object storage systems face inefficiencies in storing and retrieving small data objects, as erasure encoding methods result in increased storage costs when handling large quantities of small data objects due to suboptimal storage overhead.
Innovation Solution
A distributed object storage system that optimizes storage costs by dynamically adjusting the spreading requirement and using erasure encoding to minimize cumulative sub-fragment file sizes, allowing for efficient storage and retrieval of small data objects while maintaining redundancy, through a controller node with encoding, spreading, clustering, and decoding modules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If erasure encoding is used to distribute data objects across multiple storage elements, then fault tolerance and redundancy are improved, but storage overhead increases for small data objects
Solution Approach 1:
The patent changes the parameter of spreading width from a fixed value to a dynamically adjustable parameter. By calculating the optimal spreading width based on object size and storage element characteristics, the system adapts the encoding parameters to minimize storage overhead for small objects while maintaining fault tolerance for large objects.
Solution Approach 2:
The system dynamically adjusts the spreading width parameter based on the size of the data object being stored. This dynamic adaptation allows the system to optimize storage efficiency for different object sizes, particularly reducing overhead for small objects while maintaining adequate redundancy for larger objects.
2Device complexity
If a fixed spreading width is used in erasure encoding, then system simplicity is maintained, but storage efficiency deteriorates for small data objects
Solution Approach 1:
The patent performs preliminary calculation of the optimal spreading width before the actual encoding operation. By pre-calculating the appropriate spreading width based on object size and system parameters, the system prepares the optimal configuration in advance, avoiding complex real-time adjustments during encoding while achieving storage efficiency optimization.
3Quantity of substance
If minimum spreading width is used to reduce storage overhead, then storage capacity is improved, but fault tolerance capability deteriorates
Solution Approach 1:
The system dynamically changes the spreading width parameter based on the specific requirements of each data object. For small objects where maximum storage efficiency is needed, a minimum spreading width is used. For large objects or when higher fault tolerance is required, the spreading width is increased accordingly, thus balancing storage capacity and fault tolerance capability.
Data Source
AI summary
A distributed object storage system comprises an encoding module configured to calculate for a plurality of predetermined values of the spreading requirement the cumulative size of the sub fragment files when stored on the file system with the predetermined block size; and select as a spreading requirement from said plurality of predetermined values a calculated value that is equal to one of said predetermined values for which the cumulative size is minimal.