Storage Management via Merged File Indexing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current storage systems face inefficiencies when accessing large numbers of small files, requiring multiple I/O accesses due to the need to read directory entries and inodes, leading to low access performance.
Innovation Solution
The solution involves merging multiple small files into a single 'merged file' with sections for each file, using a mapping from characteristic values to index information to index and access these files, reducing the number of I/O accesses and improving access efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional file access methods are used to read directory entries and inodes for each small file, then file access can be performed individually, but the number of I/O accesses increases excessively leading to low access performance
Solution Approach 1:
Multiple small files are merged into a single merged file structure where files are organized in sections. Instead of accessing each small file individually through separate directory entries and inodes, the system accesses the merged file once and retrieves multiple files from their respective sections, dramatically reducing I/O accesses and improving access performance
2Ease of operation
If multiple small files are stored separately in the storage system, then each file can be managed independently, but the storage system requires reading multiple directory entries and inodes for each file access
Solution Approach 1:
The system merges multiple small files into a unified merged file structure with organized sections, reducing the complexity of multiple separate I/O operations while maintaining the ability to manage and access individual files through the merged file's section structure
Solution Approach 2:
The merged file acts as an intermediary structure between the storage system and individual small files. Instead of directly accessing multiple separate files through their各自的 directory entries and inodes, the system accesses the merged file which contains all small files organized in sections, simplifying the access process
Data Source
AI summary
Embodiments of the present disclosure provide a method, a device, and a computer program product for storage management. The method for storage management includes: acquiring a reading request for a first target file, the reading request including a first target file name of the first target file. The method further includes: determining a first target characteristic value for the first target file based on the first target file name. The method further includes: determining first target index information for the first target file from a mapping from characteristic values to index information based on the first target characteristic value, wherein the first target index information is used to index the first target file from a merged file. The method further includes: reading the first target file included in the merged file from a storage system based on the first target index information. As a result, access to a large number of small files can be converted into access to large files, thereby improving the overall file access efficiency.