File read and write method, system and equipment and storage medium

Abstract

The invention discloses a file read and write method, system and equipment and a storage medium. The method comprises the steps that a reading request of a target file sequence transmitted by an upperlayer interface is received, and a target file is selected in the target file sequence; the size of the target file is determined, and whether or not the size of the file is smaller than a preset value is judged; if yes, the target file is read in a data memorizer and saved in an inode node of a client, and a file apart from the target file is selected again in the target file sequence as a target file until files in the target file sequence are read; data in the inode node is transmitted to the upper layer interface. Compared with a method in which reading is directly conducted and files arereturned one by one in the prior art, the transmission efficiency of small files is improved, and the read and write performance of a storage system is improved.

Description

technical field [0001] The present invention relates to the field of storage technology, and more specifically, to a file reading and writing method, system and equipment, and a computer-readable storage medium. Background technique [0002] With the rapid development of the Internet era, the amount of data in many industries has reached the P level. For example, the available storage data of Taobao, the largest online shopping website in China, is 35PB, and the actual used capacity is 24PB, storing nearly 100 billion pictures. information, and the number of these information increases by billions every day, which makes the storage of massive data a great challenge. [0003] Through big data analysis, it is concluded that 60% of the stored data are small files with an average size of only 17.45KB on the entire data application side and data storage side. These file formats are various, and may be image JPG format, executable file EXE or a large number of custom formats. Th...

AI Technical Summary

Problems solved by technology

The existing technology mainly adopts the method of directly reading and writing a large number of small files, so that when the client reads and writes cluster files, the IOPS (full name in English: Input / Output Operations Per Second, full name in Chinese: the number of read and write operations per second) is very high. High, which seriously affects the read and write performance of the storage system and reduces the transmission rate of cluster files

Images