Small file access method accelerated based on solid state disk for distributed file system

Abstract

The invention discloses a small file access method accelerated based on a solid state disk for a distributed file system. The small file access method comprises the following steps of setting the threshold value of the size of an embedded file; when a client establishes a file, if the size of the established file is less than the threshold value of the size of the embedded file, embedding and storing the established file in a file metadata area, which is located in the solid state disk, of a metadata server as the embedded file, otherwise, storing the established file in an object storage server; when a process of the client opens the file, according to a storage position in file metadata information, responding to the access request of a user and returning an execution result by the metadata server or the object storage server. According to the small file access method, the data of a small file are directly stored in a metadata area of the solid state disk; the computing time of the storage position of the data of the small file is saved; the access communication time delay of the small file is saved; the access performance of the small file is promoted; the small file is conveniently optimized; the load of the metadata server cannot be aggravated; the overall performance of the distributed file system cannot be decreased.

Description

technical field [0001] The invention relates to the technical field of distributed file system data processing, in particular to a method for accessing small files in a distributed file system based on solid-state disk acceleration. Background technique [0002] With the rapid development of cloud computing and big data technology, the amount of global data is growing explosively, which brings new development opportunities and challenges for distributed file systems, from high-performance computing to data centers, from data sharing to Internet applications , the distributed file system has penetrated into all aspects of data applications. Most of the current distributed file systems are for the processing of large-scale big data files, and the storage optimization of small files is relatively lacking. With the development of social networks, mobile communications and personal terminals, the proportion of small files in the storage system is getting higher and higher. For ...

AI Technical Summary

Problems solved by technology

In this way, even when users access small files, they need to perform multiple IO requests, and the high network delay and multiple disk IO delays take up a lot of time for reading small files, which affects the access performance and efficiency of small files

There are two problems in the access of small files in the current distributed file system: 1) The data volume of small files is very small, and it needs to communicate with the data server network after obtaining metadata, which takes up network delay; 2) In order to obtain the actual data of small files File data requires one or more disk IOs with the object server, which not only increases disk IO overhead, but also occupies data server IO resources

Images