Dispersed Storage Network Synchronous Data Writing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current dispersed storage networks face challenges in synchronously storing data across multiple locations while ensuring data integrity and security, particularly in handling large datasets and complex tasks, and they lack efficient mechanisms for data retrieval and access.
Innovation Solution
A distributed computing system that utilizes dispersed error encoding and decoding techniques to store and retrieve data across multiple geographically dispersed storage vaults, employing a network of DST execution units that perform error encoding, task distribution, and data synchronization, ensuring data integrity and security through error correction and secure storage protocols.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored in multiple dispersed storage locations, then data reliability and availability are improved, but data synchronization complexity increases
Solution Approach 1:
The patent segments data into multiple data slices that are independently stored across different dispersed storage access units (DSAUs). Each slice is treated as an independent unit that can be stored, retrieved, and synchronized separately, reducing the complexity of managing entire data sets across multiple locations.
Solution Approach 2:
The patent introduces a dispersal storage network manager as an intermediary component that coordinates synchronization operations between multiple DSAUs. This manager handles the complex tasks of tracking data slice locations, managing synchronization state, and coordinating updates, thereby isolating the complexity from the storage operations themselves.
2Reliability
If error correction encoding is applied to data, then data integrity is improved, but processing time increases
Solution Approach 1:
The patent applies error correction encoding to data slices during the initial storage operation, before the data is actually written to dispersed storage locations. By performing the encoding upfront and storing the encoded slices directly, the system avoids the need for repeated encoding operations during synchronization or retrieval, thereby reducing overall processing time.
Solution Approach 2:
The error correction is applied independently to each data slice rather than to the entire data set. This segmentation allows parallel processing of encoding operations for multiple slices simultaneously, reducing the total processing time while maintaining data integrity through comprehensive error correction coverage.
3Reliability
If data is encrypted for security, then data security is improved, but access and retrieval performance deteriorates
Solution Approach 1:
The patent implements selective encryption where different portions of the data structure are encrypted with different keys or encryption methods. Specifically, data slices are encrypted with unique keys, but the metadata and indexing structures use efficient encryption schemes that allow faster access. This local differentiation of encryption quality balances security requirements with performance needs.
Data Source
AI summary
A method begins with a processing module obtaining a data object from a user and identifying a plurality of virtual storage vaults (VSVs) for storage of the data object. The method continues with the processing module determining dispersal error encoding parameters associated with a first virtual storage vault and error encoding the data object in accordance with those dispersal error encoding parameters to produce a corresponding plurality of sets of encoded data slices (EDSs), issuing write slice requests for the EDSs and receiving EDS information from at least some of the plurality of VSVs. When the EDS information indicates that a threshold number of VSVs have successfully stored the plurality of sets of encoded data slices within a synchronization timeframe, the method continues with the processing module issuing a favorable response to the user, or when a threshold number of VSVs have not successfully stored the plurality of sets of encoded data slices initiating a retry process.