Microshard Data Fragmentation for Automatic Storage Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data protection methods, such as encryption and tokenization, are inadequate against increasing computational power, and systems for managing microshard data storage face challenges with error recovery in geographically expansive setups, leading to potential data loss and corruption.
Innovation Solution
A storage resource manager configures and controls the storage and recovery of microshard data across three resource sets: a primary, secondary, and recovery resource, using redundancy and error correction techniques to ensure resilience against storage failures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If microshard data is stored in a vast array of storage devices spanning wide geographical and network areas, then data security is improved through distribution, but data access speed and availability deteriorate due to communication failures
Solution Approach 1:
The patent divides data into microshard fragments and distributes them across multiple storage devices in different locations. This segmentation allows data security to be improved through distribution while the system manages access by reconstructing data from available fragments, partially mitigating the speed deterioration caused by geographical distribution.
Solution Approach 2:
The patent implements preliminary error correction coding and redundancy mechanisms when data is initially fragmented and stored. This preliminary action prepares the system to handle future communication failures and access issues, allowing faster recovery by pre-computing error correction codes and maintaining redundancy information across the distributed storage network.
2Adaptability or versatility
If ad hoc storage management is used for distributed microshard data, then storage flexibility is improved, but error recovery capability deteriorates leading to data loss and corruption
Solution Approach 1:
The patent implements feedback mechanisms where the system continuously monitors the status of distributed storage devices and automatically triggers error recovery processes when failures are detected. This feedback loop maintains reliability by systematically responding to storage issues while preserving the ad hoc flexibility of distributed storage management through automated rather than manual intervention.
Solution Approach 2:
The patent dynamically adjusts error correction parameters and redundancy levels based on the specific storage configuration and failure patterns observed. By changing parameters such as error correction code strength and fragment distribution strategies, the system maintains flexibility in storage management while improving error recovery capability for different operational scenarios.
3Reliability
If encryption complexity is increased to counter computational power increases, then data security is improved, but decryption time and computational resources worsen
Solution Approach 1:
The patent segments encrypted data into microshard fragments distributed across multiple locations. This segmentation improves security by requiring possession of multiple fragments for decryption, while reducing the computational burden on any single system. The decryption time is distributed across multiple nodes rather than concentrated on one system, partially mitigating the time loss.
Solution Approach 2:
The patent introduces intermediary error correction codes and fragment metadata that facilitate the decryption process. These intermediaries contain pre-computed information that reduces the computational complexity required for decryption, allowing strong encryption to be maintained while reducing the time and resources needed for decryption operations.
Data Source
AI summary
A system and method for automatic recovery of a portion of source data, broken into a plurality of microshard data fragments, is described. The system and method include storing one data fragment in a first storage element of a first storage resource having a plurality of storage elements, creating and storing a duplicate of the one data fragment in a first storage element of a second storage resource having at least the same number of storage elements as the first storage resource, determining whether an error has occurred in the first storage element of the first storage resource, and creating and storing a duplicate of the data fragment from the first storage element of the second storage resource in a first recovery storage resource based on the determination. The duplicate of the data fragment from the first recovery storage resource is used to recover the portion of the source data.