Distributed Data Storage With Recursive Parity for Multi-Failure Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage and transmission methods are prone to data loss, corruption, and insecurity, especially in cases of hardware failure, theft, or catastrophic events, and lack flexibility and resilience in handling multiple disk failures or expanding storage capacity.
Innovation Solution
A method of recursively separating data into subsets and generating parity data to recreate lost subsets, allowing dynamic distribution and storage across multiple locations, with optional encryption and authentication for enhanced security and reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If RAID arrays use disk mirroring and parity disks to protect against individual disk failures, then data reliability is improved, but the system cannot easily deal with more than two disk failures and cannot be expanded without rebuilding the array
Solution Approach 1:
The patent segments data into multiple subsets and distributes them across numerous storage locations beyond traditional RAID configurations. This segmentation allows the system to handle multiple disk failures by reconstructing data from remaining subsets and enables dynamic expansion without rebuilding the entire array, as new storage locations can be added incrementally
Solution Approach 2:
The patent implements dynamic data distribution where data subsets can be reassigned across storage locations as needed. This dynamic approach allows the system to adapt to disk failures by redistributing data subsets and to expand capacity by adding new storage locations without requiring a complete array rebuild, unlike static RAID configurations
2Device complexity
If disks in a RAID array are located near to each other for ease of configuration, then device complexity is reduced, but resilience to catastrophic events such as fire or flood is compromised
Solution Approach 1:
The patent segments data into multiple subsets and distributes them across geographically dispersed storage locations. This segmentation strategy maintains configuration simplicity through automated management while achieving catastrophic event resilience by ensuring that not all data subsets are stored in the same physical location, preventing total data loss from localized disasters
3Ease of operation
If data is stored on a single drive for simplicity, then ease of operation is improved, but data security and reliability are compromised in cases of theft, loss, or failure
Solution Approach 1:
The patent segments data into multiple subsets that are automatically distributed across separate storage locations. This segmentation provides enhanced security and reliability by ensuring that no single storage location contains the complete data set, while maintaining operational simplicity through automated management systems that handle the distribution and reconstruction processes transparently
Solution Approach 2:
The patent introduces an intermediary system that manages data segmentation, distribution, and reconstruction. This intermediary layer shields users from the complexity of distributed storage while providing enhanced security, as the system automatically handles data subset management across multiple locations without requiring user intervention
4Reliability
If nested level RAID arrays are configured to improve resilience to further failed disks, then data reliability is improved, but the systems become complicated, expensive and cannot be expanded without rebuilding the array
Solution Approach 1:
The patent segments data into subsets distributed across storage locations without requiring nested RAID structures. This segmentation approach achieves resilience to multiple disk failures through mathematical reconstruction algorithms while avoiding the complexity of nested RAID configurations and enabling flexible expansion by adding new storage locations to the distributed network
Data Source
AI summary
Storing, retrieving, transmitting and receiving data (20) by a) separating the data into a plurality of data subsets (A, B); b) generating parity data (P) from the plurality of data subsets (A, B) such that any one or more of the plurality of data subsets may be recreated from the remaining data subsets and the parity data (P). Steps a and b may be repeated on any one or more each of the plurality of data subsets and parity data providing further data subsets and further parity data; and d) storing each of the further data subsets and further parity data in separate storage locations (380) or transmitting the further data subsets and further parity data.