Distributed Storage Pillar Width Adjustment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current distributed computing systems face challenges in reliably and securely storing and retrieving large amounts of data across multiple geographically dispersed locations, while ensuring data integrity and availability, especially in the presence of failures and network issues.
Innovation Solution
A distributed computing system that employs dispersed error encoding and decoding schemes, where data is segmented, encoded, and distributed across multiple storage units, allowing for reliable storage and retrieval even with failures, using a network of geographically dispersed storage and task processing units that can execute tasks on data partitions and reassemble results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored across multiple geographically dispersed storage units, then data availability and reliability are improved, but system complexity and difficulty of managing data integrity increase
Solution Approach 1:
The patent segments data into multiple data partitions that are distributed across different storage units. Each partition can be independently stored and retrieved, enabling fault tolerance while maintaining manageable complexity through modular data organization.
Solution Approach 2:
The system employs feedback mechanisms where the controller receives status information from storage units and adjusts data distribution accordingly. This feedback loop enables automatic rebalancing and integrity maintenance without manual intervention, managing complexity through automated control.
2Reliability
If dispersed error encoding is used to ensure data integrity, then data security and reliability are improved, but processing time and computational resources increase
Solution Approach 1:
The system performs error encoding and decoding operations in advance during data storage and retrieval phases. By pre-processing data partitions with error correction codes, the system ensures integrity without requiring complex real-time verification, thus reducing processing time during critical operations.
Solution Approach 2:
The patent dynamically adjusts encoding parameters based on data characteristics and storage conditions. By optimizing error correction strength and data partitioning parameters, the system achieves reliable data integrity while minimizing computational overhead and processing time.
3Quantity of substance
If data is divided into partitions for distributed storage, then storage capacity and availability are improved, but difficulty of retrieving and assembling data increases
Solution Approach 1:
The controller is designed with multi-functional capabilities to handle various data retrieval scenarios. It can assemble data partitions from multiple storage units, manage redundancy, and provide unified data access interfaces, thereby simplifying the retrieval process despite the distributed nature of storage.
Solution Approach 2:
The system merges data partitions from multiple storage units into a complete data set through coordinated retrieval operations. The controller combines results from different storage units, ensuring data consistency and availability while presenting a simplified access interface to users.
Data Source
AI summary
A method includes receiving, from a user device, a data object for storage. The method further includes determining dispersed storage error encoding parameters for the data object based on the virtual storage vault. The method further includes determining to adjust the pillar width number based on activation status of storage units in the set of storage units and others. When it is determined to adjust the pillar width number, adjusting the pillar width. The method further includes dispersed storage error encoding the data object in accordance with the decode threshold number, the encoding function, and the adjusted pillar width number. The method further includes sending a subset of encoded data slices from each of the sets of encoded data slices to active storage units of the set of storage units and sending an encoded data slice from each of the sets of encoded data slices to an active storage unit.