Secret Sharing Scheme for Distributed Storage Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face challenges in efficiently recovering data from distributed storage locations due to high decoding bandwidth requirements, which can lead to increased latency and resource inefficiency, especially when dealing with partial erasures or compromised storage nodes.
Innovation Solution
The implementation of a secret sharing scheme that encodes data into multiple sets, allowing recovery from a subset of available storage devices while minimizing decoding bandwidth by using polynomials and Reed-Solomon codes, enabling flexible data retrieval and optimal communication efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored across multiple distributed storage devices using traditional RAID systems, then data protection against drive failure is improved, but decoding bandwidth requirements increase leading to increased latency and resource inefficiency
Solution Approach 1:
The patent segments the encoded data into multiple subsets and distributes them across different storage devices. Each subset contains a portion of the encoded data, allowing the system to retrieve only the necessary portions from available devices during recovery, rather than requiring all devices to be accessed as in traditional RAID systems. This segmentation reduces the amount of data that needs to be decoded and transmitted, thereby reducing latency while maintaining data protection through distributed storage.
2Reliability
If data is stored across multiple distributed storage devices, then data protection is improved, but decoding bandwidth requirements increase leading to resource inefficiency
Solution Approach 1:
The patent applies partial action by retrieving only the necessary portions of encoded data from the available storage devices rather than requiring all encoded data to be retrieved. The system determines the minimum amount of encoded data needed to recover the source data and retrieves only that portion, reducing decoding bandwidth consumption and improving resource efficiency while maintaining data protection through the distributed secret sharing architecture.
3Reliability
If traditional secret sharing schemes are used for distributed storage, then data security is improved, but communication overhead increases reducing system flexibility
Solution Approach 1:
The patent introduces dynamics by making the data retrieval process adaptive to the availability of storage devices. The system dynamically determines which storage devices are available and retrieves encoded data from an appropriate number of them based on real-time conditions. This dynamic approach allows the system to maintain data security through secret sharing while adapting to varying system states, thereby improving flexibility and responsiveness to partial node failures or security threats.
Data Source
AI summary
Methods for distributed storage in accordance with embodiments of the invention enable secret sharing. One embodiment includes encoding source data using an encoding system to produce a plurality of sets of encoded data, where: the source data can be recovered from at least a portion of less than all of the plurality of sets of encoded data; and the source data cannot be recovered using less than a threshold number of the plurality of sets of encoded data; storing each of the plurality of sets of encoded data on a storage device from a set of storage devices on which encoded data is stored; determining a set of storage devices that are available using a decoding system, where the set of storage devices that are available does not include all of the storage devices in the set of storage devices on which encoded data is stored.