Local file system-based small file reading and writing method and system

Abstract

The invention discloses local file system-based small file reading and writing method and system. The method comprises the following steps: aggregating small files belonging to a same directory in a local file system into N large files, wherein the N is a positive integer; generating a corresponding index file for each directory one to one, wherein the index files comprise names of all the small files in the directories, names of the large files where the small files are arranged and positions of the small files in the large files; receiving small file operating instructions comprising path names of the small files and determining the index files corresponding to the directories which the small files belong to according to the small file operating instructions, wherein the operations comprise reading and / or writing; modifying the names and the positions of the small files in the path names of the small file operating instructions to the names of the large files where the small files are arranged and the positions of the small files in the large files according to the determined index files; and executing the corresponding operations on the large files where the small file are arranged according to the modified instructions. By means of the method and the system which are disclosed by the invention, the small file operating efficiency is increased.

Description

technical field [0001] The invention relates to the field of file systems, in particular to a method for reading and writing small files based on a local file system. The invention also relates to a small file reading and writing system based on the local file system. Background technique [0002] With the development of technology and the improvement of storage performance, the stored data is becoming more and more diverse, and the application scenarios of storage are becoming more and more extensive. One of the application scenarios is mainly based on a large number of small files. For this application scenario, the widely used storage method is multi-node distributed storage. However, distributed storage is not available in all cases, and the cost is relatively high. Therefore, in the traditional unified storage, there are also application scenarios based on massive small files. [0003] Traditional unified storage uses single-node local file systems such as XFS (Exten...

AI Technical Summary

Problems solved by technology

[0003] Traditional unified storage uses single-node local file systems such as XFS (Extensions for Financial Services, financial service extensions), Ext4 (Fourth Extended Filesystem, fourth-generation extended file system), etc., in applications that focus on massive small files In the scenario, when operating on each small file, the corresponding data needs to be read from the disk. Obviously, since each small file is stored in a different location on the disk, frequent operations on a large number of different small files are required. Changing lanes to read data consumes a lot of seek time on the disk, which greatly reduces the efficiency of the local file system for small files.

Images