Distributed Storage Data Slicing for Fault Tolerance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current distributed storage systems face challenges in ensuring data integrity and availability across multiple storage units, particularly in handling large datasets and complex tasks, as they lack efficient error correction and task processing mechanisms.
Innovation Solution
A distributed computing system that employs dispersed error encoding and decoding, allowing data to be segmented, encoded, and distributed across multiple geographically disparate storage units, enabling robust storage and task processing with error correction and secure retrieval, while also facilitating secure and scalable data storage and task execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored across multiple dispersed storage units, then data availability and fault tolerance are improved, but system complexity and error correction requirements increase
Solution Approach 1:
The patent segments data into multiple data slices and distributes them across different storage units. The data is divided into a first set of data slices stored at first storage units and a second set of data slices stored at second storage units, enabling fault tolerance while maintaining manageable complexity through systematic segmentation
Solution Approach 2:
The patent implements preliminary error correction by encoding data before storage. Error correction codes are applied to data slices before they are distributed to storage units, allowing the system to tolerate failures without requiring complex real-time correction mechanisms, thus improving reliability while controlling complexity
2Reliability
If error correction codes are applied to dispersed storage, then data integrity is improved, but processing time and computational overhead increase
Solution Approach 1:
The patent segments the error correction process by applying codes to individual data slices rather than entire datasets. This segmented approach allows parallel processing and reduces computational overhead while maintaining data integrity across all slices
Solution Approach 2:
The patent optimizes error correction parameters including the number of data slices, code rates, and distribution patterns to balance integrity requirements with processing efficiency. By adjusting these parameters, the system achieves reliable data integrity without excessive computational overhead
3Productivity
If large datasets are processed in distributed manner, then task processing capability is improved, but coordination complexity and communication overhead increase
Solution Approach 1:
The patent segments large datasets into manageable data slices that can be independently processed across distributed storage units. This segmentation enables parallel task execution and improves processing capability while reducing coordination complexity through localized operations
Solution Approach 2:
The patent creates multiple copies of data slices at different storage units, enabling redundant processing and improving task processing capability. The distributed copies allow independent task execution while reducing the need for complex coordination through replication
Data Source
AI summary
A method for execution by one or more computing devices includes scheduling processing of one or more data access requests of a plurality of data access requests to produce an execution schedule, where the scheduling processing is in accordance with an access performance level for previous data access requests and in accordance with a desired access performance level. The method further includes executing the one or more data access requests in accordance with the execution schedule. The method further includes determining an updated access performance level compares unfavorably to the desired access performance level. The method further includes implementing an alternate throughput scheme for subsequent data access requests of the plurality of data access requests.