Storage Index Management via Hash Files
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing methods for managing indexes in storage systems result in significant computing and storage overheads, particularly when dealing with large amounts of data, as they require extensive resource allocation for index maintenance and updates, leading to inefficiencies.
Innovation Solution
The proposed method involves generating a file to store hashes of data objects within the storage system, using these hashes to create an index, and leveraging file system functions for management, thereby reducing overheads and simplifying index maintenance by recording hierarchical relationships through file names and contents.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional index management methods are used in storage systems, then data access speed is improved, but computing resource overhead and storage resource overhead increase significantly
Solution Approach 1:
The patent extracts the index management function from the storage system core and implements it using external file system functions. By generating index files that store hash values and using file system operations for index maintenance, the computing overhead is shifted from the storage system to the file system, thereby reducing the computing resource overhead while maintaining fast data access capability
Solution Approach 2:
The patent makes the file system serve multiple purposes: it stores both actual data objects and index information (hash values). By using the file system's native capabilities for both data storage and index management, the system avoids dedicating separate computing resources for index maintenance, thus reducing overall computing resource overhead
2Speed
If traditional index management methods are used in storage systems, then data access speed is improved, but storage resource overhead increases
Solution Approach 1:
The patent extracts index storage from dedicated storage structures and places it in external files. By storing index hash values in file system files rather than in separate storage pools, the system reduces storage resource overhead while maintaining the ability to provide fast data access through efficient index searching
Solution Approach 2:
The patent merges the storage of data objects and index information into the same file system infrastructure. By combining these functions and using the file system's efficient storage mechanisms for both, the system reduces total storage resource overhead compared to maintaining separate storage structures for data and indexes
3Productivity
If indexes are maintained and updated in traditional storage systems, then data access efficiency is improved, but complexity of index maintenance increases
Solution Approach 1:
The patent introduces index files as intermediaries between the storage system and the data objects. These files, containing hash values, serve as a buffer that simplifies index maintenance operations. By using file system functions to manage these intermediary files, the complexity of index maintenance is reduced while data access efficiency is maintained
Solution Approach 2:
The patent enables the file system to self-manage index operations using its native functions. By leveraging the file system's built-in capabilities for file creation, deletion, and management, the system reduces the complexity of index maintenance without sacrificing data access efficiency, as the file system automatically handles index operations
Data Source
AI summary
The present disclosure relates to a method, a device, and a computer program product for managing indexes in a storage system. The storage system includes storage data. In the method, a first set of data objects associated with the storage data is acquired. A first set of hashes of the first set of data objects is determined respectively. Hashes in the first set of hashes are hashes of data objects in the first set of data objects. A first file is generated in the storage system to store the first set of hashes. A first name of the first file is determined based on the hashes in the first set of hashes. An index of the storage data is created based on the first file.