Filesystem Object List for NAS Search Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing filesystem search commands, such as the UNIX 'find' command, are inefficient due to massive disk access and network latency, especially when used with Network-Attached Storage (NAS), leading to slow performance and erroneous results from periodic index updates.
Innovation Solution
A method is implemented where a storage system maintains a filesystem data structure with metadata, including pathnames and attributes, which is updated in real-time and sent to clients upon request, allowing for efficient retrieval of filesystem object lists and attributes, thereby reducing the need for clients to perform extensive searches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the UNIX 'find' command is used to search through the filesystem, then files can be located according to specified criteria, but the command becomes slow due to massive disk access
Solution Approach 1:
The storage system pre-generates and maintains a filesystem object list containing metadata about all files in the filesystem before any search operation occurs. This preliminary action allows the client to receive ready-made search results without performing time-consuming disk access operations during the actual search
Solution Approach 2:
The patent introduces an intermediary filesystem object list that acts as a mediator between the raw filesystem data and the search operation. Instead of directly querying the filesystem for each search, the client receives this intermediate structure that has already organized the data, reducing the computational burden
2Reliability
If periodic queries are performed to update an index database, then the index can be maintained, but the updates are time consuming and executed only every few days resulting in erroneous content
Solution Approach 1:
The filesystem object list is updated continuously whenever filesystem changes occur (file creation, deletion, modification), rather than through periodic batch updates. This continuous maintenance ensures the index remains accurate and current without requiring time-consuming periodic full scans
Solution Approach 2:
The system implements feedback mechanisms where the storage system monitors filesystem changes and automatically triggers updates to the filesystem object list. This feedback loop ensures the index database remains synchronized with the actual filesystem state without manual intervention or fixed scheduling
3Adaptability or versatility
If network-attached storage is used to access the filesystem, then centralized storage is achieved, but network latency makes search operations unreasonably slow
Solution Approach 1:
The patent extracts the time-consuming search and metadata retrieval operations from the network path. By pre-generating the filesystem object list on the storage system and transferring it to the client, the system removes the need for repeated network round-trips during search operations, effectively taking out the latency bottleneck
Data Source
AI summary
A method for managing a filesystem by a storage system, the method may include maintaining, by a storage system, a filesystem data structure that comprises filesystem pathnames metadata related to a filesystem that is stored in the storage system; receiving, from a client that is coupled to the storage system via a network, a request to obtain a client filesystem object list related to at least a certain portion of the filesystem; generating, in response to the request and in response to the filesystem data structure, the client filesystem object list, wherein the client filesystem object list comprises at least one pathname of at least one filesystem object that belongs to the at least certain portion of the filesystem; and sending the client filesystem object list to the client.