MSR Storage Node Repair Using Segmented MDS Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face challenges in achieving high storage efficiency and minimizing repair bandwidth while ensuring data security and availability, particularly due to component failures and malicious attacks, where existing replication and erasure coding methods are inefficient and resource-intensive.
Innovation Solution
A method is introduced that splits data into segments, preprocesses them through deduplication, compression, and encryption, and then encodes them using a maximum distance separable (MDS) code to distribute them across multiple storage nodes, ensuring secure and efficient data reconstruction with optimized bandwidth and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If replication is used to protect data against storage node failures, then data reliability is improved, but storage efficiency deteriorates
Solution Approach 1:
The patent divides data into multiple segments and distributes them across storage nodes using erasure coding, where the original data is split into k segments and distributed to n storage nodes. This allows the system to tolerate up to n-k node failures while using less redundant storage space compared to full replication, thereby improving storage efficiency while maintaining data reliability.
Solution Approach 2:
The patent employs maximum distance separable (MDS) codes with configurable parameters (k, n) to adjust the trade-off between storage efficiency and reliability. By changing the parameters of the erasure coding scheme, the system can optimize storage efficiency while maintaining the required level of data protection against node failures.
2Quantity of substance
If MDS codes are used to achieve high storage efficiency, then storage efficiency is improved, but repair cost deteriorates
Solution Approach 1:
The patent segments the repair process by identifying which specific segments need to be recovered and only retrieving the necessary portions from surviving storage nodes, rather than reconstructing entire data sets. This reduces the I/O overhead and computational complexity associated with repairing failed nodes while maintaining the storage efficiency benefits of MDS codes.
3Use of energy by moving object
If locally decodable codes are used to minimize I/O overhead, then I/O overhead is reduced, but the number of practical constructions deteriorates
Solution Approach 1:
The patent uses well-established MDS code constructions (such as Reed-Solomon codes) that have practical, proven implementations available. These codes provide locally decodable properties that minimize I/O overhead during repair operations, while benefiting from the extensive existing literature and practical constructions that make them easy to implement in real systems.
4Reliability
If data encryption is used to meet security requirements, then data security is improved, but computation complexity deteriorates
Solution Approach 1:
The patent integrates encryption with the segmented erasure coding structure, applying encryption operations only to the necessary data segments rather than encrypting entire data sets multiple times. This segmented approach maintains strong security guarantees while reducing the overall computational complexity compared to traditional full-data encryption methods.
5Loss of energy
If repair bandwidth is reduced to optimize cloud storage, then bandwidth optimization is improved, but latency deteriorates
Solution Approach 1:
The patent pre-positions repair data on surviving storage nodes during the normal operation phase, so that when a node failure occurs, the repair process can immediately retrieve the necessary segments from nearby nodes without requiring long-distance data transmission. This preliminary preparation reduces both the bandwidth required for repair and the latency experienced during node recovery.
Data Source
AI summary
The present disclosure is based on erasure coding, information dispersal, secret sharing and ramp schemes to assure reliability and security. More precisely, the present disclosure combines ramp threshold secret sharing and systematic erasure coding.