A method, device and system for storing data based on majority

Abstract

The present invention discloses a method, device and system for storing data based on majority. In the method, the front-end server can select a file group from the file group with the first preset number according to the received file upload request, and store the The data in the received files are respectively uploaded to the storage unit of the second preset value number corresponding to the selected file group, and the uploading of the data in the file is determined to be completed, and the data of the third preset value number or more is received. When the physical information of the file is returned by the storage unit, the data is successfully written, and then the metadata information in the file is updated to the database. At this time, the file upload request is returned as successful. According to the technical solution of the present invention, by adopting the successful method of the majority to write files, there will be no single-point problem, and any failure of the copy device of the minority will not cause data loss. The content is transferred to each copy, which can reduce disk IO usage.

Description

technical field [0001] The present invention relates to the technical field of data storage, in particular to a method, device and system for data storage based on majority. Background technique [0002] In a large-scale distributed storage system, equipment downtime, network abnormalities, and disk failures can be seen everywhere. In order to solve these abnormal situations, distributed storage systems often store multiple redundant copies of data. Each copy of data is called a copy. In this way, when a device fails, data can be accessed from other devices. At the same time, when the data of a copy is lost, it can be recovered from other copies, ensuring the reliability of the data. [0003] Since multiple copies need to be written when writing data, there needs to be a way how to write multiple copies to count as a successful request. The existing methods for writing multiple copies are: [0004] 1. Temporary storage and asynchronous distribution methods [0005] On t...

AI Technical Summary

Problems solved by technology

[0009] 1. Files are temporarily stored on the device disk, and there will be a single point problem. Before the file is distributed to the storage device, if the device goes down or the disk storing the file fails, the file will be permanently lost;

[0010] 2. Temporarily storing received files and writing them to the disk and then asynchronously distributing them will lead to an increase in disk IO of the temporary storage machine. When the load is high, it will seriously affect the performance of the device;

[0011] 3. Write the master and multi-slave method, you need to maintain the master-slave relationship between the shards. When the master is abnormal, all write operations mapped to the shard will fail before a new master is elected.

Images