Low-latency file system address space management method and system and medium

Abstract

The invention discloses a low-latency file system address space management method and system and a medium. The method comprises the steps that a super block and a block group distribution table are generated from an address space of storage equipment, file system information and the distribution situation of a block group on a linear address space of the storage equipment are stored in the super block, and the block group distribution table is used for marking the distribution situation of data blocks in the corresponding block group; when the file is created, dynamically creating or selectinga corresponding block group according to the size of a specified data block, allocating the data block, writing file data into the allocated data block, and updating the information of a block groupallocation table and a super block. It can be guaranteed that which data blocks are occupied by one file on the storage device can be known only through one IO operation, the read-write delay of the file can be remarkably reduced, the addressing expenditure of file read-write can be remarkably reduced, the file read-write sequence can be remarkably improved, and therefore the read-write performance is improved.

Description

technical field [0001] The present invention belongs to the field of file systems for large-scale storage, and in particular relates to a low-latency file system address space management method, system and medium. A novel data structure is used to manage the physical address space of the file system to significantly reduce the File read and write delays. Background technique [0002] The local file system for storage devices is the basis of all data storage and management systems. For example, mainstream databases are all built on the file system, and the distributed file system must rely on the local file system to realize the data organization on the device. The big data storage systems represented by Hbase and Dynamo are either built on the distributed file system, Either directly call the local file system to read and write data on the storage device. In conclusion, the performance of the local file system has a crucial impact on all data storage and management systems...

AI Technical Summary

Problems solved by technology

However, current operating systems generally support multi-core and multi-threading. A file system often faces concurrent writes to a large number of files at the same time. In this way, the continuous write operation for a specific file is often disrupted, making it difficult to create a long file. Extent, cross-order writing for multiple files will significantly reduce the optimization effect of Extent technology

[0010] In short, the current file system is difficult to meet the low latency requirements of data access when faced with large files, and data access latency is a key factor affecting application performance in many scenarios

Images