Distributed File Storage with Separated Control for Fast Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems with error correction code redundancy schemes have single points of failure and cannot reconstruct failed drives quickly, as responsibility for data storage and maintenance resides in a single component, leading to inefficiencies in fault tolerance and recovery.
Innovation Solution
A dynamically scalable redundant distributed storage system that separates logical responsibility for data from actual control, using distributed hash tables and components like file and storage managers, allowing for error correction codes or replication methods, and enabling dynamic addition or removal of components without interrupting service.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If error correction code redundancy schemes are used in existing storage systems, then fault tolerance is achieved, but single points of failure exist and reconstruction of failed drives is slow because responsibility for data storage and maintenance resides in a single component
Solution Approach 1:
The patent segments the storage system into multiple independent components (storage elements) distributed across a network, with each component holding portions of redundant data. This segmentation eliminates single points of failure and enables parallel reconstruction operations, as multiple storage elements can simultaneously contribute to rebuilding failed drives, thereby improving both reliability and reconstruction speed
2Reliability
If replication methods are used for fault tolerance, then data redundancy is achieved, but system complexity increases and scalability is limited
Solution Approach 1:
The patent introduces a distributed hash table as an intermediary layer that automatically manages data location, redundancy distribution, and retrieval across storage elements. This intermediary abstracts the complexity of distributed replication from users and applications, enabling scalable fault tolerance without requiring complex manual management of redundant data copies
3Ease of operation
If centralized control is used for data management, then coordination is simplified, but bandwidth bottlenecks occur and multiple failures cannot be tolerated
Solution Approach 1:
The patent segments centralized control functions into distributed control across multiple storage elements that autonomously participate in data management operations. Each storage element can independently handle data operations and coordinate with peers through the distributed hash table, eliminating single points of failure and enabling the system to tolerate multiple simultaneous failures while maintaining operational simplicity
Data Source
AI summary
A distributed heterogeneous or homogeneous file storage system divides logical responsibility for data from actual control of the data by using separate file and storage managers. Each file manager provides access to stored files and stores metadata associated with each of the files for which it is responsible. A file manager control component allocates responsibilities among file managers, assigns management responsibilities for individual files, and maintains a record of which files are assigned to each manager. Each storage manager is responsible for managing storage of files on data storage devices and maintains a record of the location of each stored file. A storage manager control component allocates responsibilities among storage managers, assigns responsibilities for individual files, and maintains a record of which files and storage devices are assigned to each manager. A client provides user access, initiates actions within the system, and implements a redundant data recovery and reconstruction scheme.