Data storage method and system, electronic equipment and storage medium

Abstract

The invention discloses a data storage method. The data storage method comprises: setting the byte length of an erasure unit in a storage end; when a write creation request including a small file identifier is received, judging whether the length to be written is greater than the byte length of the erasure unit or not according to the write creation request; if so, calculating an erasure code of the to-be-written data and storing the erasure code in a corresponding disk; if not, packaging M+1 copy data writing sub-requests according to the to-be-written data, and packaging K-1 empty object creating sub-requests; and sending the copy data writing sub-request and the empty object sub-request to corresponding disks so as to write data. According to the invention, the data storage mode can beoptimized, and the data reading efficiency is improved. The invention further discloses a data writing system, electronic equipment and a storage medium, which have the above beneficial effects.

Description

technical field [0001] The present application relates to the technical field of data storage, and in particular to a data storage method, a system, an electronic device, and a storage medium. Background technique [0002] In a distributed storage system, data redundancy can be achieved using erasure codes. Erasure codes can improve storage space utilization (for K+M redundancy, the actual available space is K / (K+M)), but the read delay is high, and it is generally used in large file scenarios, using pre-reading on the client side function to offset the read latency of the backend. In the scenario of mixed storage of large files and small files, if the number of storage nodes remains unchanged, the use of erasure codes will seriously reduce the reading speed of small files, and the blocking of a single request will drag down the entire distributed storage cluster performance, causing normal access to large files to become stuck. [0003] Therefore, how to optimize the dat...

AI Technical Summary

Problems solved by technology

Erasure codes can improve storage space utilization (for K+M redundancy, the actual available space is K / (K+M)), but the read delay is high, and it is generally used in large file scenarios, using pre-reading on the client side function to offset the read latency of the backend

In the scenario of mixed storage of large files and small files, if the number of storage nodes remains unchanged, the use of erasure codes will seriously reduce the reading speed of small files, and the blocking of a single request will drag down the entire distributed storage cluster performance, causing normal access to large files to become stuck

Images