Filesystem Hierarchy Table for Metadata Query Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large file systems with deep hierarchies and billions of files and directories require efficient extraction of metadata, which is time-consuming and resource-intensive, and existing protocols like NFS and SMB lack specific commands for external applications, leading to inefficiencies in querying and data retrieval.
Innovation Solution
A filesystem hierarchy table representing the hierarchy in a tabular format allows for efficient querying using SQL commands, with each file's pathname split into components stored across columns, enabling flexible and customized queries, and an alias table manages path changes to reduce latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If recursive traversal of filesystem hierarchies is used to extract files, then complete file extraction is achieved, but time and compute resources are consumed excessively
Solution Approach 1:
The patent segments the filesystem hierarchy into a tabular structure where path components are divided into separate columns. This segmentation allows the query system to access specific path components directly without traversing the entire hierarchy, thereby maintaining complete file extraction capability while dramatically reducing query execution time.
Solution Approach 2:
The patent performs preliminary action by pre-processing the filesystem hierarchy into a flattened table structure with path components organized in columns before queries are executed. This pre-organization enables efficient filtering and searching operations without requiring recursive traversal during actual query execution.
2Adaptability or versatility
If standard filesystem protocols (NFS, SMB) are used for querying, then compatibility is maintained, but specific commands for external applications are not supported
Solution Approach 1:
The patent implements a universal query interface that works across different filesystem protocols while providing enhanced functionality for external applications. The tabular metadata structure serves as a common foundation that supports both standard protocol operations and specialized query requirements, enabling one system to fulfill multiple functions.
3Adaptability or versatility
If proprietary management commands with filters are used, then specific client needs are met, but the complicated filesystem hierarchy structure makes efficient scanning difficult
Solution Approach 1:
The patent segments the complicated filesystem hierarchy into discrete path component columns in a table structure. This segmentation transforms the complex hierarchical navigation problem into simple column-based filtering operations, maintaining versatile filter capabilities while eliminating the scanning complexity associated with deep hierarchies.
4Reliability
If the entire filesystem tree is scanned to extract metadata, then all files are captured, but the process is time-consuming and resource-intensive
Solution Approach 1:
The patent extracts only the necessary path component metadata from the filesystem hierarchy into a separate tabular structure. This extraction allows the system to retrieve metadata for specific files or directories without scanning the entire filesystem tree, maintaining complete metadata capture capability while dramatically reducing compute resource consumption.
Data Source
AI summary
A method for obtaining information related to a filesystem path in a file system that is stored in a storage system, the method may include (i) receiving a request to obtain hierarchy information related to file system entities stored under the filesystem path; and (ii) retrieving the hierarchy information; wherein the retrieving comprises accessing a filesystem hierarchy table that stores information regarding pathnames of file system entities of the file system; wherein the retrieving includes searching for path components of the filesystem path in subsets of cells of the filesystem hierarchy table, wherein each subset of cells is part of a set of cells that include path components of a certain pathname of a filesystem entity that belong to the file system.