File storage method and distributed storage system

Abstract

The invention discloses a file storage method applied to a distributed storage system. The method includes acquiring a first level of a first file, to be stored, from a first client; determining a first-level storage server set corresponding to the first level, from M level storage server sets set in the distributed storage system; determining a first storage server set from the first-level storage server set according to a predetermined strategy; determining a first server from the first storage server set according to a predetermined rule; acquiring the first-level first file to be stored, and storing the first file to be stored into the first server. The first server is capable of synchronizing the first file to be stored to each one of other servers in the first storage server set.

Description

technical field [0001] The invention relates to the field of file storage, in particular to a method for storing files and a distributed file storage system. Background technique [0002] The design of existing distributed file storage systems is generally based on the following principles: CAP theory: it is impossible for a distributed storage system to meet the three requirements of consistency, availability and partition tolerance at the same time, and at most two can be satisfied at the same time. Among them, C: Consistency In a distributed environment, data at multiple points is consistent; A: Availability Each operation can always return at a certain time, that is, the system is available at any time; P: Tolerance of network Partition (partition tolerance) In the event of a network partition (such as a network outage), the separated systems can also function normally. [0003] For existing distributed storage systems, partition tolerance is a basic requirement, so the...

AI Technical Summary

Problems solved by technology

[0005] One: the existing distributed storage system does not distinguish the files to be stored and the storage server group in the process of storing files, but some files in the files to be stored are important to users, while some files is less important, causing the existing distributed storage system to be unable to store the files to be stored in a differentiated manner

[0006] Second: when the existing distributed storage system stores files, it generally adopts a consistent hash algorithm to allocate a first storage server group to store the stored files. Once the first storage server group breaks down, the All data that will be stored in the first storage server group is stored in the first second storage server group that is clockwise connected to the first storage server group, so that the second storage server group The increase of the data stored in the storage system and the increase of the storage pressure of the second storage server group not only make the data stored in each storage server group in the distributed storage system unbalanced, but may also lead to data loss

[0007] Third: the existing distributed storage system records the location and path of all stored files through the Tracker (distributor), which causes a large amount of storage space in the Tracker to be occupied. Once the data stored in the Tracker reaches a certain amount, it will Contributing to the reduction of Tracker's work efficiency

[0008] Its four: the existing distributed storage system generally stores the stored file after fragmentation processing, if the user wants to view the file, the distributed storage system will assemble each fragment of the file After completion, the user can view the file, but when storing small files in the distributed storage system, it will reduce the performance of the distributed storage system

Images