Distributed Media Retrieval Using Error-Coded Data Slices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage systems face challenges with data integrity and security due to the failure of physical movement-based memory devices, such as disc drives, which can lead to data loss and increased maintenance needs, and RAID systems suffer from efficiency and security issues as more discs are added, particularly with the growth of data volume.
Innovation Solution
A distributed storage network (DSN) system that uses error-coded data slices stored across multiple geographically diverse locations, allowing for reliable and secure data storage and retrieval through error correction and redundancy, managed by a DS managing unit and processed by a DS processing unit, ensuring data integrity and security.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple levels of redundant disc drives are used to replicate data, then data reliability is improved, but storage efficiency deteriorates due to overhead
Solution Approach 1:
The patent segments data into discrete data units that can be independently encoded and distributed across multiple storage devices. Each data unit is divided into portions that are stored separately, allowing for efficient redundancy without requiring complete data duplication. This segmentation enables the system to achieve both reliability and storage efficiency by storing only the necessary redundant portions rather than full copies.
Solution Approach 2:
The patent employs error correction coding that transforms data parameters by adding redundant information in an optimized manner. Instead of simple replication, the system uses coding schemes that change the data representation to include error correction capabilities while minimizing the overhead ratio. This parameter transformation allows the system to maintain data reliability while improving storage efficiency compared to traditional RAID approaches.
2Reliability
If more disc drives are added to RAID array, then data security is improved, but system complexity increases
Solution Approach 1:
The patent implements a self-managing storage system where the control unit automatically performs data encoding, distribution, and recovery operations without requiring complex manual configuration or intervention. The system self-adjusts to failures by automatically detecting missing data units and regenerating them from redundant portions, eliminating the need for complex administrative overhead typically associated with RAID systems. This self-service capability reduces system complexity while maintaining enhanced data security through distributed redundancy.
3Speed
If physical movement-based memory devices are used, then data access speed is improved, but data integrity deteriorates due to device failure
Solution Approach 1:
The patent replaces reliance on mechanical disc drive reliability with a mathematical error correction system. Instead of depending on the physical integrity of moving mechanical parts, the system uses error correction codes that can reconstruct data even when physical devices fail. This substitution of mechanical reliability with mathematical redundancy allows the system to maintain fast data access speeds while dramatically improving data integrity by eliminating the single point of failure inherent in physical movement-based devices.
Data Source
AI summary
A method begins by a dispersed storage (DS) processing module receiving a first sub-set of encoded data slices, wherein a data segment of multi-media content was encoded using a dispersed storage error coding function to produce a set of encoded data slices, wherein the set of encoded data slices is partitioned into the first sub-set of encoded data slices and a second sub-set of encoded data slices. The method continues with the DS processing module sending accessing information when the second sub-set of encoded data slices is to be requested. The method continues with the DS processing module receiving, in response to the accessing information, at least one of the encoded data slices of the second sub-set of encoded data slices such that a decode threshold number of encoded data slices have been received.