Hub-Spoke File System Namespace Sharing via Inode Masks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern distributed file systems face challenges in efficiently sharing namespaces across clusters, particularly in large data sets and geographically distant locations, leading to issues like poor responsiveness and redundant data copying due to reliance on long-distance connectivity.
Innovation Solution
The implementation of a hub-spoke file system architecture, where a hub file system shares portions of its namespace and data with spoke file systems, using inode masks and leases to manage data blocks, allowing for on-demand data copying and efficient access while minimizing redundant data storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a central file system is used to enable shared access across distant locations, then data accessibility is improved, but responsiveness deteriorates due to long-distance connectivity dependencies
Solution Approach 1:
The patent divides the centralized file system into distributed clusters (spoke clusters and hub clusters), where each cluster maintains local copies of namespace metadata. This segmentation allows local access operations to occur without long-distance communication, improving responsiveness while maintaining data accessibility through the distributed cluster architecture.
Solution Approach 2:
The patent introduces a new dimension of namespace sharing by implementing inode masks that can be shared across clusters without copying entire data blocks. This dimensional change from full data replication to selective metadata sharing enables fast access to namespace information while minimizing data transmission overhead.
2Speed
If data is copied across distributed file systems to enable local access, then responsiveness is improved, but redundant data storage increases
Solution Approach 1:
The patent implements selective copying of only necessary namespace metadata (inode masks) rather than entire data blocks. The inode masks contain references to actual data block locations, allowing the system to copy minimal metadata while maintaining references to original data, thus improving responsiveness without proportionally increasing storage redundancy.
Solution Approach 2:
The patent changes the parameter of what is being copied from full data blocks to compressed namespace metadata (inode masks). This parameter change reduces the volume of data that needs to be replicated across clusters, improving responsiveness while minimizing redundant storage requirements.
3Adaptability or versatility
If entire data blocks are copied to spoke file systems, then data accessibility is improved, but data distribution efficiency deteriorates due to redundant copying
Solution Approach 1:
The patent extracts only the essential namespace metadata (inode masks) from the complete data blocks and copies only these extracted portions to spoke file systems. The inode masks contain the necessary information to access and identify data blocks without requiring the actual data to be present locally, thus improving data accessibility while maintaining distribution efficiency.
Solution Approach 2:
The patent applies partial action by copying only the necessary portion of data (inode masks) rather than complete data blocks. This partial copying strategy provides sufficient information for local namespace operations while avoiding the excessive data transmission and storage that would result from copying entire data blocks.
Data Source
AI summary
Embodiments are directed to sharing namespaces across file system clusters. A file in a spoke file system may be determined based on a command provided to the spoke file system such that the file may be associated with an inode mask. Data blocks associated with the file may be determined based on the command and the inode mask such that the data blocks may be absent from the spoke file system. Requests for leases may be generated based on the absent data blocks such that each lease request corresponds to a portion of the absent data blocks Employing the leases provided by a hub file system to: copy the absent data blocks from the hub file system where each portion of absent data blocks may be associated with a lease; updating the inode mask to include the leases and each copied portion of absent blocks.