Data reading method, system and device based on distributed storage system

Abstract

The invention provides a data reading method based on a distributed storage system, which comprises the following steps: receiving a read operation request sent by a client; counting the heat of the data to be read in the read operation request; judging whether the current operation is a read operation or not, and if not, continuing to receive a read operation request sent by the client; if so, determining whether the object data requested by the read operation exists in the pgcache, and if so, assembling the contents of the data to be read; if not, processing the read operation request by theOSD, and after the processing is completed, assembling the contents of the data to be read. Therefore, the method can effectively reduce the read delay, improve the data read speed of the distributedstorage system, and further improve the read and write performance of the whole system by caching some hot data on the main pg and carrying out the data aging and hot data updating in the pgcache intime with the data reading

Description

Technical field [0001] The present invention relates to the field of data storage technology, and more specifically to a data reading method, system and device based on a distributed storage system. Background technique [0002] With the development of computer technology, servers store more and more data, and a single server cannot meet the requirements of storing data. Therefore, a distributed storage system has been developed. [0003] In the current commonly used distributed storage systems, considering the reliability of data storage, erasure coding technology is commonly used to make data redundant to increase the reliability of the system, and the stored data is divided into K data blocks. The coding algorithm calculates M parity blocks and stores them in the erasure data pool. The original data can be recovered by using any K data blocks among the K+M data blocks. Through the erasure correction rule setting of the erasure code, the storage space size can be controlled, whi...

AI Technical Summary

Problems solved by technology

[0004] However, since the existing processing logic is to use a fixed stripe size to create an erasure data pool, it is impossible to set different stripe sizes for different data, and it is also unable to adapt to different types of file storage, resulting in a decline in space utilization

The decline in space utilization will lead to read and write operations of redundant data, resulting in a decline in the overall read and write performance of the system

Therefore, the read operation delay of the erasure correction data pool on the distributed storage system is relatively large, and the response when users access the erasure correction data pool is too slow, which seriously affects the user experience.

Images