Distributed Storage Coding for Secure Redundancy and Fast Recovery

The patent segments data into multiple chunks and distributes them across storage nodes using erasure coding. Instead of replicating entire data blocks, the system divides data into segments, applies encoding to create redundant segments, and distributes them across multiple nodes. This segmentation approach enables efficient redundancy with lower storage overhead compared to full replication.

The patent changes the redundancy parameter from replication factor to erasure coding parameters (k, n, d). By using maximum distance separable (MDS) codes and regenerating codes with configurable parameters, the system can adjust the trade-off between storage efficiency and data protection. The repair bandwidth parameter d allows optimization of repair operations without proportionally increasing storage overhead.

The patent performs erasure coding in advance during data ingestion, converting raw data into encoded segments before storage. This preliminary encoding action eliminates the need for complex real-time computation during read operations. The system pre-computes the redundant segments and stores them, so that during retrieval, the system only needs to perform simpler decoding operations rather than full encoding computations.

The patent introduces an intermediary encoding layer between data and storage nodes. This encoding layer acts as a mediator that transforms data into a form suitable for efficient storage and retrieval. The encoded segments serve as intermediaries that can be independently stored, retrieved, and combined to reconstruct original data, reducing the computational burden on the storage system during operational phases.

The patent employs a universal erasure coding framework that can accommodate multiple code constructions including MDS codes, regenerating codes, and locally decodable codes. This multi-functional approach allows the system to select the most appropriate code construction based on specific operational requirements such as repair bandwidth constraints, storage efficiency targets, or I/O overhead considerations, thereby providing versatility without limiting practical implementations.

The patent implements dynamic code selection and parameter adjustment based on system conditions. The system can adaptively choose different erasure coding schemes and parameters (k, n, d) depending on the operational context, such as the number of available storage nodes, network bandwidth conditions, and repair priorities. This dynamic adaptability overcomes the limitation of fixed code constructions by allowing the system to optimize for different performance metrics as needed.

The patent applies regenerating codes specifically to repair operations where only partial data recovery is needed. Instead of requiring full data reconstruction, the system uses the local repair capability of regenerating codes to recover only the missing segments. This partial action approach optimizes bandwidth usage for repair operations without requiring the system to support all operation types with the same optimized approach, thus maintaining versatility for other operations.

The patent implements different coding strategies for different operational contexts. For repair operations, it uses regenerating codes with optimized local repair properties. For other operations like initial data ingestion or full reconstruction, it may use different erasure coding schemes. This local quality approach tailors the coding strategy to the specific operational requirement, maximizing bandwidth optimization for repairs while maintaining adaptability for other operations through alternative coding choices.

The patent performs encryption in advance during data ingestion, converting plaintext data into ciphertext before erasure coding and storage. This preliminary encryption action ensures that even if storage nodes are compromised, the stored data remains protected. By encrypting data before it enters the storage system, the patent eliminates the need for continuous cryptographic operations during storage and retrieval, thereby reducing computational complexity during operational phases while maintaining strong security.

The patent segments encrypted data into chunks and applies erasure coding to the ciphertext segments. This segmentation approach allows the system to work with smaller, manageable encrypted blocks, reducing the computational burden of cryptographic operations. By processing data in segments rather than large monolithic blocks, the system maintains security through encryption while minimizing the computational complexity of cryptographic operations through efficient segment-level processing.