Threshold Computing for Resilient Dispersed Storage Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems lack efficient means to leverage resources effectively for data processing operations, leading to suboptimal execution and utilization of computing and storage capabilities.
Innovation Solution
A dispersed storage network (DSN) utilizing computing devices with dispersed storage error encoding and decoding, managed by a managing unit and integrity processing unit, employs error correction schemes like Reed-Solomon encoding to distribute data across multiple storage units, ensuring data integrity and resilience against failures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored in a dispersed storage network with error encoding, then data integrity and resilience against failures are improved, but device complexity increases
Solution Approach 1:
The patent divides data into multiple data slices and distributes them across different storage units in the dispersed storage network. Each storage unit stores only a portion of the data, and error correction codes are applied to each slice independently. This segmentation allows the system to tolerate failures of individual storage units while maintaining data integrity, as the original data can be reconstructed from any sufficient number of surviving slices.
Solution Approach 2:
The patent introduces a threshold computing unit that acts as an intermediary between the dispersed storage units and the data retrieval process. This unit coordinates the retrieval of data slices from multiple storage units, manages the reconstruction process, and ensures that the minimum threshold of slices is collected before data can be accessed. This intermediary layer simplifies the complexity of coordinating distributed storage operations.
2Reliability
If error correction schemes like Reed-Solomon encoding are used to distribute data, then resilience against storage unit failures is improved, but manufacturing precision requirements increase
Solution Approach 1:
The patent employs Reed-Solomon error correction codes, which transform the data encoding parameters to provide robust error correction capabilities. By changing the encoding scheme to include redundancy symbols, the system can detect and correct errors that occur during data transmission or storage. This parameter change in the encoding process enables the system to maintain data integrity even when storage units fail or data becomes corrupted.
3Duration of action of stationary object
If data is distributed across multiple storage units, then data retention capability is improved, but loss of information increases due to potential failures
Solution Approach 1:
The patent applies error correction codes to data slices before storing them in the dispersed storage network. This beforehand cushioning ensures that even if some storage units fail or data becomes corrupted during retention, the original data can still be reconstructed. The error correction codes act as a protective buffer that prevents information loss during the data retention period.
Solution Approach 2:
The patent creates multiple copies of data in the form of data slices distributed across different storage units. Instead of storing a single copy of data, the system creates multiple redundant slices that can be used to reconstruct the original data. This copying strategy ensures that data retention is maintained even when some storage units become unavailable, as the data can be recovered from the remaining slices.
4Ease of operation
If a managing unit and integrity processing unit are implemented, then ease of operation is improved, but device complexity increases
Solution Approach 1:
The patent implements a managing unit that performs multiple functions including data allocation, storage unit coordination, and integrity verification. This multi-functional approach consolidates management operations into a single unit, making the system easier to operate. The managing unit handles various tasks such as distributing data slices, monitoring storage unit status, and coordinating error correction operations, thereby simplifying user interaction with the dispersed storage network.
Solution Approach 2:
The integrity processing unit automatically verifies data integrity and performs error correction without requiring manual intervention. The system self-monitors the health of storage units and autonomously manages error correction processes. This self-service capability simplifies operation by eliminating the need for users to manually manage data integrity, while the complexity is contained within the automated processing units.
Data Source
AI summary
A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. The computing device selects a subset of the other computing devices to perform a computing task on a data object. The computing device determines processing parameters of the data and determines task partitioning. The computing device also processes the data based on processing parameters to generate data slice groupings and partitions the task based on the task partitioning to generate partial tasks. The computing device obtains and processes at least the decode threshold number of the plurality of partial results generated by the subset of the other computing devices to generate a result.