Distributed Storage File Operations via Virtual File Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage systems face challenges in efficiently managing file operations across distributed storage systems, particularly in scaling to large clusters and handling high loads without compromising performance.
Innovation Solution
The proposed system employs a distributed storage architecture with virtual file systems (VFS) that utilize non-volatile memory (NVM) and erasure coding to manage metadata across multiple nodes, enabling load balancing and high performance even in large-scale configurations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional data storage systems are used, then simplicity of implementation is maintained, but scalability to large clusters and performance under high load deteriorates
Solution Approach 1:
The patent segments the storage system into distributed nodes, each independently managing file operations and metadata. This segmentation enables the system to scale to large clusters while maintaining performance, as each node can operate autonomously and handle load locally without creating a single point of failure or bottleneck
Solution Approach 2:
The patent introduces virtual file systems as an additional layer between the physical storage and the data, creating a new dimension of abstraction. This allows the system to manage complexity by separating the interface from the implementation, enabling scalable architecture where the virtual layer handles coordination while physical nodes handle execution
2Adaptability or versatility
If distributed storage architecture with virtual file systems is implemented, then scalability and performance are improved, but system complexity increases
Solution Approach 1:
The patent implements universal interfaces and protocols that allow different storage nodes to communicate through standardized mechanisms. The virtual file system provides a unified interface that works across diverse hardware platforms, enabling scalability without requiring complex custom integrations for each node type
Solution Approach 2:
The virtual file system acts as an intermediary layer between the data and the physical storage infrastructure. This mediator handles the complexity of distributed coordination, metadata management, and load balancing, allowing the underlying nodes to remain simple while achieving system-wide scalability and adaptability
Data Source
AI summary
A plurality of computing devices are communicatively coupled to each other via a network, and each of the plurality of computing devices is operably coupled to one or more of a plurality of storage devices. A plurality of failure resilient address spaces are distributed across the plurality of storage devices such that each of the plurality of failure resilient address spaces spans a plurality of the storage devices. The plurality of computing devices maintains metadata that maps each failure resilient address space to one of the plurality of computing devices. Each of the plurality of computing devices is operable to read from and write to a plurality of memory blocks, while maintaining an extent in metadata that maps the plurality of memory blocks to the failure resilient address space.