Data redundancy processing method and device and storage medium

Abstract

The embodiment of the invention provides a data redundancy processing method and device and a storage medium. The method comprises the steps of: obtaining and mapping IO data into a set, wherein the set comprises a main node and erasure code nodes corresponding to the main node, and the IO data are respectively mapped and stored into the main node and the erasure code nodes corresponding to the main node; the set receives an IO data processing request and executes IO data processing in the set, wherein the IO data processing comprises write-in IO processing and read-out IO processing; detecting whether a main node in the set and an erasure code node corresponding to the main node have the IO data storage fault or not; if the IO data storage fault exists, the IO data storage fault is eliminated, and then IO data redundancy processing is completed; according to the IO data redundancy processing method and device, whether the IO data can be read and written or not in redundancy processingcan be detected, and then the IO data is recovered when the IO data is lost and fails, so that the integrity of the whole IO data redundancy processing is better.

Description

technical field [0001] The invention relates to the technical field of data processing in artificial intelligence, in particular to a data redundancy processing method, device and storage medium. Background technique [0002] Cloud storage is a mode of online online storage, that is, data is stored on multiple virtual servers usually hosted by a third party, rather than on a dedicated server. Hosting companies operate large-scale data centers, and those who need data storage and hosting can meet their data storage needs by purchasing or leasing storage space from them. According to the needs of customers, data center operators prepare and store virtualized resources at the back end, and provide them in the form of storage resource pools, and customers can use this storage resource pool to store files or objects. [0003] Three indicators of cloud storage system: high reliability, low storage overhead, high read and write performance; multi-copy and erasure coding are two st...

AI Technical Summary

Problems solved by technology

[0003] Three indicators of cloud storage system: high reliability, low storage overhead, high read and write performance; multi-copy and erasure coding are two strategies widely used in storage systems. They are selected under the premise of ensuring high reliability. Different extreme trade-offs are made. Multi-copy storage has high overhead, but better performance; erasure code storage has low overhead, but poor performance.

[0004] At present, the storage strategy identifier of the data to be stored is obtained; the storage method corresponding to the storage strategy identifier is used to store the data to be stored, and the storage strategy identifier is used to indicate the use of at least one of the following storage methods: copy redundancy processing method, erasure correction codec Processing method; that is, data redundancy for multiple copies and erasure codes; data redundancy in the form of multiple copies and erasure codes is a relatively common data processing method, but currently for multiple copies and erasure codes. When data redundancy is processed, the problems of possible data loss and failure; and after data loss or failure, data recovery is difficult and difficult to operate

Images