Distributed Block Storage With AONT Verification Across Sites
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage and processing systems face challenges in ensuring secure, reliable, and efficient storage and retrieval of large volumes of data across distributed networks, particularly in the face of device and network failures, while maintaining data integrity and security.
Innovation Solution
A distributed computing system that employs dispersed storage and task processing units, utilizing error encoding and decoding techniques to store and process data across geographically diverse locations, ensuring redundancy and security, and incorporating management units for vault creation, error correction, and task distribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored in a distributed network across multiple devices, then reliability and fault tolerance are improved, but system complexity and coordination overhead increase
Solution Approach 1:
The patent segments data into multiple slices that are distributed across different storage devices in the network. Each slice is independently stored, and the system can reconstruct the original data from any sufficient subset of slices. This segmentation approach improves reliability by eliminating single points of failure while managing complexity through standardized slice management protocols.
Solution Approach 2:
The patent introduces intermediary components including a distributed file system layer and metadata management systems that mediate between users and the underlying distributed storage infrastructure. These intermediaries abstract the complexity of data distribution, replication, and reconstruction, allowing users to access data as if from a single storage device while the intermediary handles the complex coordination across the network.
2Reliability
If data is encrypted and secured in distributed storage, then security is improved, but processing speed and retrieval efficiency decrease
Solution Approach 1:
The patent applies encryption and security protocols during the data ingestion and storage phase rather than during retrieval. Data is encrypted before being sliced and distributed, and security credentials are pre-configured in the metadata. This preliminary action ensures security without adding computational overhead during retrieval operations, as the decryption and reconstruction processes can proceed in parallel using pre-established security contexts.
3Reliability
If error correction codes are applied to distributed data, then data integrity is improved, but computational overhead and storage requirements increase
Solution Approach 1:
The patent merges error correction capabilities with the data slicing mechanism itself. Rather than adding separate error correction layers, the system uses erasure coding where the slice structure inherently provides fault tolerance. Any sufficient subset of slices can reconstruct the original data, combining the functions of data distribution and error correction in a single integrated approach that minimizes additional storage overhead.
Data Source
AI summary
A distributed computer system operates by storing data as encoded blocks in storage units of the distributed computer system at physically different sites and for an indefinite period of time, wherein the encoded blocks are encrypted as a group via a hash-based authentication function. The distributed computer system further operates by receiving, at the storage units of the distributed computer system, an all or nothing transaction (AONT) request associated with the data; and verifying the AONT request, via the hash-based authentication function and based on a decryption of the encoded data blocks stored in the storage units of the distributed computer system at the physically different sites.