Dispersed Storage Encoding with Integrity Verification and Secure Slices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current dispersed storage networks face challenges in securely and reliably storing data across multiple geographically distributed storage units, particularly in maintaining data integrity and security against unauthorized access and failures without the need for redundant copies.
Innovation Solution
A dispersed storage network (DSN) architecture that utilizes error encoding techniques, such as Cauchy Reed-Solomon encoding, to distribute data across multiple storage units, ensuring data can be recovered from a subset of encoded slices, and includes a managing unit for secure data management, integrity processing, and secure encoding/decoding protocols to maintain data integrity and security.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored using traditional RAID or simple replication, then data security and reliability are improved, but storage efficiency deteriorates due to redundant copies
Solution Approach 1:
The patent segments data into multiple slices and disperses them across different storage units. Instead of storing complete redundant copies, the data is divided into N slices where only K slices are needed for reconstruction. This segmentation allows the system to achieve the same reliability as replication while using less storage space, as each slice is smaller than a complete data copy.
Solution Approach 2:
The patent changes the storage parameter from storing complete data copies to storing encoded data slices. By applying error correction encoding, the system transforms the data into N slices where K slices suffice for recovery. This parameter change enables the system to maintain data security while improving storage efficiency by eliminating the need for full redundant copies.
2Reliability
If data is dispersed across multiple geographically distributed storage units, then system availability and fault tolerance are improved, but data integrity verification becomes more complex
Solution Approach 1:
The patent incorporates integrity verification mechanisms that provide feedback about the state of stored data slices. The system can verify data integrity by checking checksums or other verification data associated with each slice, and can request retransmission or repair of corrupted slices. This feedback mechanism simplifies integrity verification across distributed units by providing automated detection and correction capabilities.
3Reliability
If error correction encoding is applied to distribute data, then data recovery capability is improved, but processing time increases due to encoding and decoding operations
Solution Approach 1:
The patent applies error correction encoding in advance during the data writing phase. By pre-encoding the data into N slices with built-in error correction capability, the system prepares the data for potential future recovery operations. This preliminary action ensures that when data needs to be recovered, the decoding process is faster because the encoding structure is already in place, reducing the time penalty associated with error correction operations.
Data Source
AI summary
A method for execution by an integrity processing unit includes performing a deterministic function on data for storage to produce an integrity value. The data and the integrity value are combined in accordance with a combining function to produce a data package. The processing system determines an encryption approach in response to determining to encrypt the data package. The data package is encrypted in accordance with the encryption approach to produce a secure package. The secure package is encoded to produce a set of slices. The set of slices is decoded to reproduce the secure package. The secure package is decrypted to reproduce the data package. The data package is de-combined in to generate reproduced data and a received integrity value. The deterministic function is performed on the data to produce a calculated integrity value, and the received integrity value is compared to the calculated integrity value.