Method for storing files and distributed storage system

Abstract

The invention discloses a method for storing files, which is applied in a distributed storage system to obtain the first level of the first file to be stored from a first client, and M levels set from the distributed storage system Determining a first-level storage server group corresponding to the first level in the storage server group, and then based on a preset policy, determining a first-level storage server group from the first-level storage server group, and then based on a preset policy A rule is set to determine a first server from the first storage server group, and then obtain the first file to be stored at the first level, and store the first file to be stored in the first server, Wherein, the first server can synchronize the first file to be stored to each server in the other servers of the first storage server group.

Description

technical field [0001] The invention relates to the field of file storage, in particular to a method for storing files and a distributed file storage system. Background technique [0002] The design of existing distributed file storage systems is generally based on the following principles: CAP theory: it is impossible for a distributed storage system to meet the three requirements of consistency, availability and partition tolerance at the same time, and at most two can be satisfied at the same time. Among them, C: Consistency In a distributed environment, data at multiple points is consistent; A: Availability Each operation can always return at a certain time, that is, the system is available at any time; P: Tolerance of network Partition (partition tolerance) In the event of a network partition (such as a network outage), the separated systems can also function normally. [0003] For existing distributed storage systems, partition tolerance is a basic requirement, so the...

AI Technical Summary

Problems solved by technology

[0005] One: the existing distributed storage system does not distinguish the files to be stored and the storage server group in the process of storing files, but some files in the files to be stored are important to users, while some files is less important, causing the existing distributed storage system to be unable to store the files to be stored in a differentiated manner

[0006] Second: when the existing distributed storage system stores files, it generally adopts a consistent hash algorithm to allocate a first storage server group to store the stored files. Once the first storage server group breaks down, the All data that will be stored in the first storage server group is stored in the first second storage server group that is clockwise connected to the first storage server group, so that the second storage server group The increase of the data stored in the storage system and the increase of the storage pressure of the second storage server group not only make the data stored in each storage server group in the distributed storage system unbalanced, but may also lead to data loss

[0007] Its three: the existing distributed storage system records the location and path of all storage files by Tracker (distributor), which causes a large amount of storage space in the Tracker to be taken. Once the data stored in the Tracker reaches a certain amount, it will Contributing to the reduction of Tracker's work efficiency

[0008] Its four: the existing distributed storage system generally stores the stored file after fragmentation processing, if the user wants to view the file, the distributed storage system will assemble each fragment of the file After completion, the user can view the file, but when storing small files in the distributed storage system, it will reduce the performance of the distributed storage system

Images