Encoded Data Slice Storage Using Local and Dispersed Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computer storage systems face challenges with data integrity and security due to the failure of physical movement-based memory devices, such as disc drives, and the inefficiencies and security risks associated with redundant array of independent discs (RAID) solutions, particularly as data volume grows and maintenance demands increase.
Innovation Solution
A dispersed storage network (DSN) system that uses error coding dispersal storage to distribute data across multiple physically diverse locations, employing a DS processing unit to encode data into slices, which are stored on multiple DS units, allowing for reliable and secure data retrieval even in the event of device failures, and periodic integrity verification to rebuild corrupted slices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If redundant array of independent discs (RAID) is used to protect against disc drive failure, then data reliability is improved, but device complexity and maintenance overhead increase
Solution Approach 1:
The patent segments data into multiple slices and distributes them across different storage locations (local flash memory and remote DS units). This segmentation approach provides redundancy without requiring complex RAID configurations, as each slice can be independently stored and retrieved, simplifying the overall system architecture while maintaining data reliability.
Solution Approach 2:
The patent introduces a dispersed storage network (DSN) as an intermediary layer between the computing device and storage media. The DSN handles the complexity of data distribution, redundancy management, and retrieval coordination, thereby reducing the complexity burden on individual devices while improving overall system reliability.
2Reliability
If multiple levels of redundant disc drives are used to replicate data, then data security is improved, but storage capacity is reduced due to overhead
Solution Approach 1:
The patent stores a decode threshold number of encoded data slices, which is sufficient for data reconstruction but not necessarily all slices. This partial storage approach provides the necessary redundancy for data security while minimizing the storage overhead, as only the minimum required slices are maintained across local and remote storage.
Solution Approach 2:
The patent uses error coding to transform data into encoded slices with specific parameters (decode threshold, pillar width). By adjusting these parameters, the system can optimize the balance between data security (reliability) and storage capacity utilization, storing only the necessary number of slices rather than full replicas.
3Reliability
If data is distributed across physically diverse locations, then fault tolerance is improved, but data retrieval complexity increases
Solution Approach 1:
The patent implements a feedback mechanism where the computing device receives acknowledgments from DS units about slice storage status and retrieval availability. This feedback enables automated tracking of distributed data locations, simplifying the retrieval process by providing real-time information about where data slices are stored without requiring complex manual tracking.
Solution Approach 2:
The dispersed storage network automatically manages the distribution, tracking, and retrieval of data slices across physically diverse locations. The system self-services by autonomously handling the complexity of coordinated access to distributed slices, eliminating the need for manual intervention in managing fault tolerance across multiple locations.
Data Source
AI summary
A method begins by a processing module encoding data utilizing a dispersed storage error coding function to produce a set of encoded data slices, wherein the dispersed storage error coding function includes a decode threshold parameter and a pillar width parameter. The method continues with the processing module storing a number of encoded data slices of the set of encoded data slices in a local memory, wherein the number is based on the decode threshold parameter and is less than the pillar width parameter, and outputting remaining encoded data slices of the set of encoded data slices to dispersed storage network (DSN) memory.