Method and device for detecting and repairing data consistency and medium

Abstract

The invention discloses a data consistency detection and repair method and device and a medium, and relates to the technical field of storage. According to the method, consistency detection is carried out on the files, and when it is detected that the files are inconsistent, the files are repaired. Obtaining a data object of the to-be-repaired file in the repairing process and determining the to-be-repaired data object; and repairing the to-be-repaired data object according to the copy data corresponding to the data object. Therefore, according to the method, verification information does not need to be attached to the whole IO path in the process of realizing the data consistency, so that the data consistency is relatively easily realized; secondly, whether the files are consistent or not is judged firstly, then the to-be-repaired files are divided into the multiple data objects, the files with consistent data do not need to be divided into the multiple data objects, and therefore the time for determining the to-be-repaired data objects is shortened, the data consistency detection and repair process is relatively simple, and the data consistency detection efficiency is improved. And the data consistency detection and repair efficiency is improved.

Description

technical field [0001] The present application relates to the field of storage technology, in particular to a method, device and medium for data consistency detection and repair. Background technique [0002] With cloud computing and digital transformation, the amount of data generated is increasing exponentially. Therefore, there are new technical requirements for massive data storage. In order to meet the storage requirements of massive data, distributed storage technology has emerged under the current technical background. However, the architecture of the distributed storage system is complex. When there are more cluster nodes, the probability of failure of different cluster nodes is higher. The factors causing data corruption mainly include hardware errors, such as memory, central processing unit (Central Processing Unit, CPU) ), network card, etc.; signal-to-noise interference during data transmission, such as SATA hard disk (Serial Advanced Technology Attachment hard ...

AI Technical Summary

Problems solved by technology

However, the architecture of the distributed storage system is complex. When there are more cluster nodes, the probability of failure of different cluster nodes is higher. The factors causing data corruption mainly include hardware errors, such as memory, central processing unit (Central Processing Unit, CPU) ), network card, etc.; signal-to-noise interference during data transmission, such as SATA hard disk (Serial Advanced Technology Attachment hard disk), Fiber Channel (Fiber Channel, FC) and other protocols; firmware problems, such as disk array (Redundant Arrays of Independent Disks, RAID ) controller, disk controller, etc.; software issues, such as operating system kernel issues, local file system issues, network system issues, general block layer issues, IO scheduling layer issues, etc.; operation and maintenance factors, when extreme conditions occur, the distribution System maintenance operation error

[0003] All of the above factors will cause data damage, resulting in inconsistent data in the distributed cluster system, business IO errors, and data unavailability, which in turn reduces the reliability of the data.

At present, end-to-end data verification is generally used to achieve data consistency, but it is necessary to attach verification information to the entire IO path. Since it is difficult to attach verification information to the entire IO path, data consistency is achieved. The process is more complicated

Images