A data processing method and device

Abstract

The invention provides a data processing method and apparatus. The method comprises the steps of obtaining to-be-stored original data; dividing the to-be-stored original data into at least two data blocks; coding the at least two data blocks by utilizing a first algorithm and a second algorithm to obtain at least one first coded block and at least one second coded block corresponding to the at least two data blocks; and storing the obtained data blocks, first coded blocks and second coded blocks in rack servers of different machine rooms. Data redundancy is realized by coding, so that the storage cost can be reduced while the data reliability is ensured. When a machine room has a machine room level fault, the machine room can be subjected to data restoration, thereby realizing storage disaster recovery of the machine room level fault. The data restoration rate of one algorithm in the first algorithm and the second algorithm is higher than that of the other algorithm, and when a rack level fault occurs, the data restoration can be performed through the algorithm with the relatively high data restoration rate, so that the data restoration rate is increased.

Description

technical field [0001] The invention belongs to the technical field of data processing, and more specifically relates to a data processing method and device. Background technique [0002] For a distributed storage system, when a fault occurs at any level such as machine level, rack level, and computer room level in the distributed storage system, it is important to ensure the reliability of the data stored in the distributed storage system when these faults occur. The top priority of distributed storage systems. [0003] At present, the distributed storage system adopts the data redundancy method to ensure the reliability of the stored data, that is, for any piece of data, different copies of the data are stored in different rack servers in different computer rooms, so that the distributed storage system If a piece of data is lost, the lost data can still be recovered through the copy of the data stored in the rack server in other computer rooms, but this data redundancy me...

AI Technical Summary

Problems solved by technology

[0003] At present, the distributed storage system adopts the data redundancy method to ensure the reliability of the stored data, that is, for any piece of data, different copies of the data are stored in different rack servers in different computer rooms, so that the distributed storage system If a piece of data is lost, the lost data can still be recovered through the copy of the data stored in the rack server in other computer rooms, but this data redundancy method will increase the storage cost, so in ensuring the reliability of the data Reducing storage cost while maintaining is an important issue in distributed storage systems

Images