Input and output pre-reading method and device of distributed file system

Abstract

The invention discloses an input and output pre-reading method and device for a distributed file system, and the method comprises the steps: obtaining a pre-reading request, and determining a pre-reading interval based on the pre-reading request; determining at least one strip to be pre-read based on the pre-read interval and the strip division position of the distributed file system; generating at least one corresponding sub-request for each of at least one strip to be pre-read, wherein at least one sub-request is used to perform input / output pre-reading in place of the pre-read request in amutually independent manner. According to the invention, the IO friendliness of the storage pool service can be improved, repeated pre-reading is avoided, and the pre-reading efficiency of the distributed file system is improved.

Description

technical field [0001] The present invention relates to the field of data pre-reading, more specifically, a method and device for input and output pre-reading of a distributed file system. Background technique [0002] For a distributed file system (object storage), the minimum read-ahead length of a large file is the same as the read length. Since the actual read length is smaller than the object stripe size, the low-level small IO shielding performance is poor. Low-level small IO mainly refers to the fact that multiple IO (input and output) requests with small data volumes read the same data object at the same time. Because of reading the same object, read competition occurs with each other, and the delay is high. At the same time, all stripes (objects) of the pre-read data are sent at once, and all the stripes are forcibly associated. For example, when three stripe objects are read at the same time, a pre-read request will be issued, and only three stripe objects are load...

AI Technical Summary

Problems solved by technology

During the process of waiting for all stripes to be loaded, the loaded stripes may be read first, and the following situations will occur: 1) The read data is discarded, and the read-ahead will read this stripe again until all the stripe data is loaded Complete; 2) After the pre-reading is completed, the pre-read data will be advanced in advance, and the heat of the data that has been read will be higher than that of the un-pre-read data, resulting in the subsequent unread data being discarded first

[0003] There is currently no effective solution to the problem that a large number of low-level small IOs and repeated reading of associated stripes affect the efficiency of the storage pool in the existing technology

Images