Memory Selection via Scoring in Dispersed Storage Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current distributed computing systems face challenges in efficiently managing and processing data across multiple storage units and task execution units, particularly in ensuring data integrity and availability while maintaining security and scalability.
Innovation Solution
A distributed computing system that employs a dispersed storage and task (DST) execution unit, which uses a decentralized agreement module to rank and select suitable memory resources for data storage and retrieval based on a scoring function, ensuring reliable and secure data storage and task processing across geographically disparate locations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is dispersed across multiple storage units in a distributed computing system, then data availability and fault tolerance are improved, but system complexity and difficulty in managing data integrity increase
Solution Approach 1:
The patent segments data into multiple encoded data slices and disperses them across different storage units. Each slice is independently stored, allowing the system to maintain data availability even when individual storage units fail. The segmentation is achieved through encoding schemes that divide original data into multiple fragments that can be reconstructed from a subset of slices.
Solution Approach 2:
The patent introduces intermediary components including a processing module that coordinates data access requests and a decentralized agreement module that manages consensus among distributed storage units. These intermediaries handle the complexity of data integrity verification and coordination, shielding users from the underlying system complexity while maintaining reliability.
2Reliability
If data is encoded and dispersed across multiple locations, then data security and fault tolerance are improved, but data access time and processing efficiency may worsen
Solution Approach 1:
The patent performs preliminary encoding of data into multiple slices before distribution, and pre-establishes mapping relationships between data slices and storage units. When data access is requested, the system can quickly identify which slices to retrieve from which storage units based on pre-computed mappings, reducing access time compared to ad-hoc retrieval strategies.
Solution Approach 2:
The patent employs encoding parameters and threshold values that can be adjusted to balance between fault tolerance and access speed. By changing parameters such as the number of required slices for reconstruction or the redundancy level, the system can optimize performance characteristics based on specific operational requirements.
3Reliability
If a decentralized agreement module is used to rank and select memory resources, then data integrity and security are improved, but computational overhead and processing time increase
Solution Approach 1:
The decentralized agreement module enables storage units to autonomously participate in consensus-making and data integrity verification without requiring centralized coordination for every operation. Each storage unit can independently verify data integrity using pre-shared encoding information, reducing the computational burden on any single node and minimizing overall processing overhead.
Data Source
AI summary
A method begins by a processing module of a dispersed storage and task (DST) execution unit receiving a data request for execution by the DST execution unit, the data request including a slice name associated with an encoded data slice of the data request. The method continues with the processing module generating a scoring resultant corresponding to each of a plurality of memories of the DST execution unit, in accordance with a ranking function and the slice name. The method continues with the processing module selecting one of the plurality of memories of the DST execution unit in accordance with a mapping function and executing the data request utilizing the one of the plurality of memories of the DST execution unit.