Distributed Storage Network Data Slicing and Error Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage systems face challenges with data integrity and security due to the high probability of disc failures in redundant arrays, which increases maintenance demands and introduces security risks from multiple data copies, while also incurring efficiency losses as data volume grows.
Innovation Solution
A distributed storage network (DSN) system that uses error coding dispersal storage to partition data into slices, which are then encoded and stored across multiple physically diverse locations, allowing for reliable and secure data retrieval even in the event of individual storage device failures, with a storage integrity processing unit for periodic verification and rebuilding of corrupted slices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If redundant arrays of independent discs (RAID) are used to protect against disc failures, then data reliability is improved, but device complexity and maintenance demands increase
Solution Approach 1:
The patent divides data into multiple slices and disperses them across many independent storage locations rather than using traditional RAID arrays. Each slice is stored separately in different physical locations, transforming the system from a concentrated redundant array into a distributed segmented storage network, thereby reducing device complexity while maintaining reliability
Solution Approach 2:
The patent introduces error coding as an intermediary layer between the data and storage locations. Instead of directly replicating data across multiple discs as in RAID, error coding transforms the data into encoded slices that can be recovered even if some storage locations fail, simplifying the overall system architecture while improving reliability
2Reliability
If multiple copies of data are stored for redundancy, then data reliability is improved, but security risks increase due to multiple data copies
Solution Approach 1:
The patent segments data into slices and disperses them across multiple locations without creating complete copies. Each location holds only a portion of the encoded data, making it impossible for any single location or small group of locations to compromise the entire dataset, thus reducing security risks while maintaining reliability through error coding
Solution Approach 2:
The patent applies different properties to different parts of the stored data. Instead of uniform replication, each slice is uniquely encoded and stored in specific locations with particular access controls. This local differentiation means that compromising one location does not expose the entire dataset, reducing security risks while maintaining overall data reliability
3Quantity of substance
If data volume grows in conventional storage systems, then storage capacity is improved, but efficiency losses occur due to increased maintenance demands
Solution Approach 1:
The patent segments data into slices that can be independently managed and accessed. As storage capacity grows by adding more locations, each new location receives only a portion of the encoded slices, allowing the system to scale efficiently without proportionally increasing maintenance overhead, since error coding automatically handles recovery across the distributed network
Solution Approach 2:
The patent implements automatic error detection and correction through error coding. The system self-manages data integrity verification and recovery without requiring manual intervention, even as data volume and storage locations grow. This automation maintains efficiency by eliminating the need for proportional increases in maintenance personnel and procedures
Data Source
AI summary
A method begins with receiving a data search criteria. The method continues by accessing a master database that includes a plurality of entries, wherein an entry includes a data name field, a metadata field, a data DSN address, and a metadata DSN address. The method continues by indexing the database based on a comparison of the data search criteria with metadata contained in the metadata field of the entries. The method continues by, when one or more entries of the database have the metadata that substantially matches the data search criteria, utilizing the data DSN address of the one or more entries to retrieve one or more sets of encode data slices. The method continues by decoding the one or more sets of encoded data slices to retrieve one or more data segments corresponding to the data search criteria.