File writing method and system for distributed storage system

Abstract

The invention discloses a file writing method and system for a distributed storage system. The method comprises the following steps: a client sends a metadata request to a metadata server; the client judges whether the length of data to be written is larger than a preset data length threshold or not according to information of the data to be written in a message returned by the metadata server; if the length of data to be written is larger than the preset data length threshold, after the data to be written is written into a cache of the client, a file handle of the data to be written is immediately released, and dirty data is written into an object-based storage device (OSD); and if the length of data to be written is not larger than the preset data length threshold, after the data to be written is written into the cache of the client, the file handle of the data to be written is added into a delayed release link list, and when the number of file handles in the delayed release link list is larger than a preset handle number threshold, the file handles in the delayed release link list are released, and the dirty data is written into the OSD. The method and system adopting the technical scheme has the advantages that interaction between the client and the writing OSD is reduced during writing of a large number of small files, so that the small file writing performance is improved.

Description

technical field [0001] The invention relates to the technical field of distributed storage, in particular to a method and system for writing files in a distributed storage system. Background technique [0002] With the rapid development of network and information technology, the production speed of data is getting faster and faster. In the era of big data, the requirements for data storage and access speed are getting higher and higher. Traditional data storage methods are obviously unable to meet the growing demand for data storage, and distributed storage systems emerge as the times require. [0003] A distributed storage system stores data dispersedly on multiple independent devices, while a network storage system uses a centralized storage server to store all data. [0004] Most of the information on the Internet is a large number of small files. At present, when storing a large number of small files in a distributed storage system, there are still problems such as low...

AI Technical Summary

Problems solved by technology

[0006] When a large number of files are written, the client needs to frequently interact with MDS and OSD, which is likely to cause request blocking, and the back-end storage has a high delay

Images