Hierarchical Metadata for Distributed Data Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Remote data storage solutions, such as cloud storage, lack control and security for data controllers as data is stored with a single provider, potentially vulnerable to seizures by foreign governments, and the location of storage is not under their control.
Innovation Solution
Data is striped across multiple disparate repositories with metadata stored hierarchically and encrypted, allowing retrieval without compromising file security, and the metadata is also striped, ensuring no single repository can reconstruct the file without decryption keys.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If data is stored with a single cloud storage provider, then storage simplicity and ease of operation are improved, but security and control over data are worsened due to potential access by the provider and vulnerability to government seizure
Solution Approach 1:
The patent divides data into multiple fragments and distributes them across different storage repositories. Each fragment alone is insufficient to reconstruct the original data, requiring a threshold number of fragments. This segmentation ensures that no single provider or repository can access the complete data, thereby maintaining security while distributing storage across multiple locations.
Solution Approach 2:
The patent introduces a coordinator component that acts as an intermediary between the user and the distributed storage system. The coordinator manages metadata, tracks fragment locations, and coordinates retrieval operations without having access to the actual data fragments. This intermediary enables simplified operation for users while maintaining the security benefits of distributed storage.
2Reliability
If data is distributed across multiple repositories, then security and control are improved by preventing single-point access, but device complexity and operational difficulty are worsened
Solution Approach 1:
The coordinator serves as a central intermediary that manages the complexity of distributed storage. It maintains metadata about fragment locations, handles retrieval coordination, and abstracts the distributed nature of storage from users. This allows the system to achieve high security through distribution while presenting a simplified interface to users.
Solution Approach 2:
The system performs preliminary organization of data fragments and metadata before distribution. The coordinator pre-establishes the mapping between data, fragments, and repository locations, and pre-configures retrieval paths. This preliminary action reduces the operational complexity during actual data retrieval and management operations.
3Reliability
If metadata is stored in encrypted form, then file security is improved, but retrieval speed and productivity are worsened due to decryption requirements
Solution Approach 1:
The patent applies different encryption characteristics to different parts of the metadata structure. File-identifying information in the metadata is encrypted to maintain security, while other metadata that needs frequent access for traversal and location purposes is kept in accessible form. This selective encryption approach maintains security for sensitive information while preserving retrieval speed for operational metadata.
Data Source
AI summary
Files are striped across multiple disparate data repositories. Metadata is stored hierarchically in a tree data structure to allow retrieval and reassembly of the files from the multiple repositories. The metadata is at least partially encrypted. The hierarchical tree data structure may be traversed without decrypting file identifying information. In this way, a file storage coordinator may quickly traverse the metadata without compromising the security of the striped files. Identification and retrieval of files associated with the metadata requires access to a decryption key, generally unavailable to the file storage coordinator. The metadata may be striped across the repositories in much the same way as the files are striped.