A data verification method, device, apparatus and medium

Abstract

The application discloses a data checking method and device, equipment and medium, and relates to the technical field of distributed storage. The data checking method is applied to a distributed storage system and includes the following steps: obtaining original data written into a disk through a write operation, and encoding the original data based on LDPC to obtain encoded data; generating at least one checking copy and a plurality of normal copies according to the encoded data; when it is determined that the encoded data in the plurality of normal copies is damaged through a read operation, performing LDPC decoding on the checking copy by calling the read operation to obtain the original data, and repairing the encoded data in the plurality of normal copies according to the original data. Since the LDPC decoding is introduced, when there is only one checking copy, the damaged original data can be restored through the checking copy, and thus, the error data can be restored in the case that the number of normal copies is small and the normal copies are damaged.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed storage, in particular to a data verification method and device, equipment and medium. BACKGROUND

[0002] In recent years, with the rapid development of the Internet, the demand for data storage capacity and quality is increasing day by day. In a distributed storage system, ensuring the reliability of data writing is the top priority, so in order to ensure the storage quality of data, data verification is essential. The existing data verification method is to verify by ordinary copy, and after verifying the data error, according to another ordinary copy and the ordinary copy, the error data is determined and corrected. In order to ensure the accuracy of the data, multiple copies are generally generated, but when multiple copies are abnormal, the error data cannot be corrected, and multiple copies occupy system space, which also has a great impact on the front-end business.

[0003] In view of the above problems, how to recover error data when using fewer ordinary copies and ordinary copies are error is a problem that technicians in the field strive to solve. SUMMARY

[0004] The purpose of the present application is to provide a data verification method, device, equipment and medium, which is used to correct error data when using fewer copies and copies are error.

[0005] To solve the above technical problems, the present application provides a data verification method applied to a distributed storage system, comprising:

[0006] Obtaining original data written to the disk through a write operation, and encoding the original data based on LDPC to obtain encoded data;

[0007] Generating at least one verification copy and multiple ordinary copies according to the encoded data;

[0008] When the encoded data in the multiple ordinary copies is determined to be damaged through a read operation, performing LDPC decoding on the verification copy through the read operation to obtain the original data, and repairing the encoded data in the multiple ordinary copies according to the original data. Preferably, the determination that the encoded data in the multiple ordinary copies is damaged through the read operation comprises:

[0009] When the write check code obtained through the write operation and the read check code obtained through the read operation are inconsistent, it is determined that the encoded data in the multiple ordinary copies is damaged.

[0010] Preferably, the write check code obtained through the write operation and the read check code obtained through the read operation comprise:

[0011] When the encoded data is written and stored to the disk through the write operation, the encoded data is divided, and a plurality of write check codes are obtained;

[0012] When the read operation is called to perform LDPC decoding on the check copy, the encoded data stored in the check copy is divided, and a plurality of read check codes are obtained;

[0013] The write check code and the read check code correspond to each other.

[0014] Preferably, dividing the encoded data comprises:

[0015] The encoded data is divided according to a first preset standard, wherein the first preset standard is that the encoded data is divided in units of 4k.

[0016] Preferably, dividing the encoded data stored in the check copy comprises:

[0017] The encoded data in the check copy is divided according to a second preset standard, wherein the second preset standard is that the amount of data of the encoded data in the check copy corresponds to the amount of data of the encoded data divided in units of 4k.

[0018] Preferably, after determining that the encoded data in the plurality of normal copies is damaged, the method further comprises:

[0019] Outputting first prompt information representing that the encoded data in the plurality of normal copies is damaged.

[0020] Preferably, after repairing the encoded data in the plurality of normal copies according to the original data, the method further comprises:

[0021] Outputting second prompt information representing that the encoded data in the plurality of normal copies damaged is repaired.

[0022] To solve the above technical problems, the application further provides a data check device, comprising:

[0023] An obtaining and encoding module is configured to obtain original data written to a disk through a write operation, and encode the original data based on LDPC to obtain encoded data;

[0024] A generating module is configured to generate at least one check copy and a plurality of normal copies according to the encoded data;

[0025] A calling and repairing module is configured to, when it is determined that the encoded data in the plurality of normal copies is damaged through a read operation, call the read operation to perform LDPC decoding on the check copy to obtain the original data, and repair the encoded data in the plurality of normal copies according to the original data.

[0026] In addition, the device further comprises the following modules:

[0027] Preferably, determining the encoded data damage in the plurality of normal copies through the read operation comprises:

[0028] The determining module is configured to determine the encoded data damage in the plurality of normal copies when the write check code obtained through the write operation and the read check code obtained through the read operation are inconsistent.

[0029] Preferably, the write check code obtained through the write operation and the read check code obtained through the read operation comprise:

[0030] The first dividing module is configured to divide the encoded data when the encoded data is written and stored to the disk through the write operation, and obtain a plurality of write check codes.

[0031] The second dividing module is configured to divide the encoded data stored in the check copy when the read operation is called to perform LDPC decoding on the check copy, and obtain a plurality of read check codes; wherein the write check code and the read check code correspond to each other.

[0032] Preferably, dividing the encoded data comprises:

[0033] The third dividing module is configured to divide the encoded data according to a first preset standard, wherein the first preset standard is to divide the encoded data in units of 4k.

[0034] Preferably, dividing the encoded data stored in the check copy comprises:

[0035] The fourth dividing module is configured to divide the encoded data in the check copy according to a second preset standard, wherein the second preset standard is a data amount of the encoded data in the check copy corresponding to a data amount of the encoded data divided in units of 4k.

[0036] Preferably, after determining the encoded data damage in the plurality of normal copies, the method further comprises:

[0037] The first output module is configured to output first prompt information representing the encoded data damage in the plurality of normal copies.

[0038] Preferably, after repairing the encoded data in the plurality of normal copies according to the original data, the method further comprises:

[0039] The second output module is configured to output second prompt information representing the repaired encoded data in the plurality of normal copies.

[0040] To solve the above technical problems, the present application also provides a data check device, comprising:

[0041] The memory is configured to store the computer program.

[0042] The processor is configured to point to the computer program to realize the steps of the data check method.

[0043] To solve the above technical problems, the application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program.

[0044] The data verification method provided by the application is applied to a distributed storage system, and includes the following steps: obtaining original data written into a disk through a write operation, and encoding the original data based on LDPC to obtain encoded data; generating at least one check copy and a plurality of normal copies according to the encoded data; when it is determined through a read operation that the encoded data in the plurality of normal copies is damaged, performing LDPC decoding on the check copy by calling the read operation to obtain the original data, and repairing the encoded data in the plurality of normal copies according to the original data. Since the LDPC decoding is introduced, when there is only one check copy, the damaged original data can also be recovered through the check copy, and thus the error data is recovered in the case that the number of normal copies is small and the normal copies are damaged.

[0045] The application further provides a data verification device, equipment and medium, and the effects are the same as above. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly explain the embodiments of the application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0047] Figure 1 A data verification method flow chart provided by an embodiment of the application;

[0048] Figure 2 A data verification device structure diagram provided by an embodiment of the application;

[0049] Figure 3 A data verification device structure diagram provided by an embodiment of the application. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, but not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0051] The core of the present application is to provide a data verification method, device, equipment and medium, which can correct error data in the case of using fewer copies and copy errors.

[0052] In order to enable the person skilled in the art to better understand the present application scheme, the present application will be further described in detail below in combination with the drawings and specific embodiments.

[0053] The data verification method in the distributed storage system mainly includes two kinds. One is to verify through the normal copy (it can be understood that, in order to prevent the data from being modified due to misoperation and other reasons in the verification process, the original data is written into a new file to obtain a file called normal copy). When writing data into the normal copy, the calculation is performed on every 4k data to obtain a verification block. When reading the normal copy, the calculation is performed on the read data again to obtain a verification block. The verification blocks of the read data and the write data correspond to the verification values respectively. Whether the data is damaged is determined by comparing the verification values corresponding to the verification blocks of the read data and the write data. If the verification values of the read data and the write data are inconsistent, it is confirmed that the data is damaged, and the correct data is read from other normal copies for verification correction.

[0054] The second is a consistency verification algorithm. Specifically, the normal copy in the distributed storage system is periodically scanned for data, and the consistency of the data stored in multiple normal copies is detected. The consistency includes consistency verification and deep consistency verification. The consistency verification only scans the metadata, and the scanning operation is executed faster. The deep consistency verification not only scans the metadata, but also scans the additional data stored, and the scanning operation is executed slower. At this time, in order to ensure the reliability of the written data in the distributed storage system, as many normal copies as possible are generated, and the correctness of the data is verified through the read operation after the write operation is completed. When multiple normal copies are generated, a large amount of system space is occupied, and a large amount of system memory is occupied in the read operation process, which not only prolongs the operation time but also greatly affects the front-end business. It can be seen that the existing data verification method is verified through the normal copy. When the normal copy is damaged, the data determined to be incorrect after verification cannot be recovered.

[0055] Based on the above problems, the present application provides a data checking method based on LDPC check code, which can generate a check copy by performing LDPC encoding operation on one of the copy data in the multi-copy storage mode, and perform data recovery by performing LDPC decoding operation on the check copy when data damage is found in the data checking. Moreover, when the check copy is damaged, the original data can still be obtained by decoding, which effectively improves the data security. When the check copy is protected, the number of data copies can be appropriately reduced, which can also ensure data security and reliability, and effectively save storage space.

[0056] Figure 1 A data checking method flow chart provided by the embodiment of the present application is shown in FIG. 1. As shown in the figure, the data checking method is applied to a distributed storage system, and includes the following steps. Figure 1

[0057] S10: obtaining original data written to a disk by a write operation, and encoding the original data based on LDPC to obtain encoded data.

[0058] S11: generating at least one check copy and a plurality of normal copies according to the encoded data.

[0059] S12: when it is determined that the encoded data in the plurality of normal copies is damaged by a read operation, performing LDPC decoding on the check copy by calling the read operation.

[0060] to obtain the original data, and repairing the encoded data in the plurality of normal copies according to the original data.

[0061] In the embodiment, when the write check code obtained by the write operation is inconsistent with the read check code obtained by the read operation, it is determined that the encoded data in the plurality of normal copies is damaged.

[0062] Correspondingly, the write check code obtained by the write operation and the read check code obtained by the read operation include: when the encoded data is written and stored to the disk by the write operation, the encoded data is divided, and a plurality of write check codes are obtained; when the LDPC decoding is performed on the check copy by calling the read operation, the encoded data stored in the check copy is divided, and a plurality of read check codes are obtained; wherein the write check code and the read check code correspond one by one.

[0063] ​It should be noted that the encoding data is divided according to the first preset standard, wherein the first preset standard is that the encoding data is divided in units of 4k. Similarly, the encoding data in the check copy is divided according to the second preset standard, wherein the second preset standard is that the data amount of the encoding data in the check copy corresponds to the data amount of the encoding data divided in units of 4k. For example: in the normal copy obtained through the write operation, 4k data is taken as the data division unit, and one write check code is obtained for each 4k unit of encoding data, and at this time, the encoding data amount is 4k; in the check copy obtained through the read operation, the encoding data in the check copy is divided, and the division standard is that the data amount of the decoded data in the check copy is 2k, and the 2k data amount corresponds to the 4k data in the write operation, and at this time, the second preset standard is to divide the encoding data in units of 2k. The above description is only one of many embodiments, and the specific implementation can be flexibly changed according to the implementation scene.

[0064] In addition, it should also be noted that, due to the nature of LDPC encoding and decoding, the length of the encoding and decoding data block needs to be the same, in the multi-copy distributed storage system, the security of the data is improved by encoding one of the normal copies, which is called the check copy. The write operation will have different block sizes, such as 4k, 8k, 64k, and 1M data blocks for write operation, and the corresponding check matrix is used for encoding when writing different size data blocks. Different size check matrices can be saved in the distributed storage system, and the corresponding check matrix is determined by the data block size before the write operation, and the check matrix is read and operated into the memory (which can be understood as a disk) for the subsequent write operation. This application mainly takes the LU encoding algorithm in LDPC as an example for description. The encoding process is as follows:

[0065] First, the check matrix H required in the encoding and / or decoding process is obtained through the LU encoding algorithm, and after the check matrix H is saved, the data is encoded and decoded through H. The generation method of the check matrix H is as follows:

[0066] The check matrix is written in the following form:

[0067] H = [H1, H2]

[0068] Wherein, the matrix of H1 is an m x k matrix, and the matrix of H2 is an m x m matrix. Let x = [s, p], according to the check formula Hx T = 0, it is expanded to:

[0069]

[0070] The matrix equation is expanded to:

[0071] H2p T = H1s T

[0072] When the check matrix H is non-singular, H2 is full rank, and the following equation can be obtained:

[0073]

[0074] At this time, H2p T = H1s T When the check matrix satisfies the lower triangular structure, the encoding can be performed by iteration. Assuming that the H matrix is a full rank matrix, the iteration formula of the check code can be derived according to

[0075]

[0076] where j is any value of k of the H1 matrix of m x k, and the value range is 1-k; i is any value of m of the H2 matrix of m x m, and the value range is 1-m.

[0077] After the encoding is completed to generate the check copy, the check copy is binary data after encoding, and when data damage is found in data checking, the check copy data is decoded again through the check matrix H to recover the original data. At this time, the decoding algorithm is the probability BP decoding algorithm, which can also be called the sum-product algorithm. The algorithm is based on the probability of information on the Tanner graph, and finally realizes decoding. The BP decoding algorithm tries to decode once for each iteration, and if the decoding is successful, the iteration is immediately ended instead of being iterated for a fixed number of times. If the decoding is not successful after reaching the pre-set number of iterations, the decoding fails, which effectively reduces the computational complexity.

[0078] Suppose that the read code word after disk damage is w q = u q + n q , n is the influence of the disk damage rate on the data, and the sequence is obtained by decoding w. q = u q + n q , n q is a Gaussian variable, therefore, the mean of w q is 1, and the variance of the Gaussian variable is σ 2 .

[0079] It should be noted that at this time, the lemma 1 needs to be introduced, where the lemma 1 is: first, assume that a binary sequence a = (a1, a1, …, a s ), where s represents any element in the binary sequence a, and the total number of elements is s; where the probability of occurrence of any element a r is 1 P(a r ​=1) is denoted as P r Then, the probability that the binary sequence a contains an even number of 1s is: Where z is the first z elements out of s elements; similarly, in a binary sequence a, the probability of containing an odd number of 1s is...

[0080] The BP decoding algorithm can be roughly divided into the following four steps:

[0081] First, initialization:

[0082] The initial probability P of the variable node is obtained based on the received sequence and channel characteristics. f (1) and P f (0); then set the message that each variable node f passes to the check node g∈D(f) connected to it as:

[0083]

[0084] Where D(f) is the set of check nodes connected to the variable node, v f For receiving symbols. and These represent the messages transmitted from the variable node to the verification node with probabilities of 0 and 1, respectively, during the 0th iteration.

[0085] Secondly, message processing of the verification node:

[0086] All verification nodes receive messages transmitted from the variable nodes f∈R(g) connected to them. and Afterwards, the first processing is performed, and the processed message is then passed to the variable node. According to Lemma 1, the message passed from the verification node to the variable node is:

[0087]

[0088] Where R(g) represents the set of all variable nodes connected to the check node. and Let R represent the messages transmitted by the verification node to the variable node with probabilities of 0 and 1, respectively, during the l-th iteration. g / f represents the set of variable nodes connected to the check node, excluding the f node.

[0089] Third, variable node message processing:

[0090] All variable nodes receive messages from the verification node g∈D(f) and Then, the variable node message is processed, and then passed to the verification node, completing one iteration. Therefore, the message from the variable node to the verification node is:

[0091]

[0092] where, and denote the messages transmitted from variable nodes to check nodes with probability 0 and 1 at the lth iteration, respectively. f / g denotes the set of check nodes connected to the variable node except g. K fg denotes the correction factor such that and are normalized.

[0093] Fourth, the decoding decision:

[0094] Hard decision is made for all variable nodes at each iteration, if the decoding is successful, otherwise the iteration continues until the maximum iteration number is reached. The hard decision message is

[0095]

[0096] where, and are the hard decision messages. K f denotes the correction factor, and the hard decision message is normalized.

[0097] if the decision is otherwise

[0098] On the basis of the above-mentioned embodiments, as a more preferred embodiment, after determining that the encoded data in the plurality of normal copies is damaged, further comprising: outputting first prompt information representing that the encoded data in the plurality of normal copies is damaged. Similarly, after repairing the encoded data in the plurality of normal copies according to the original data, further comprising: outputting second prompt information representing that the damaged encoded data in the plurality of normal copies is repaired. It should be noted that whether the first prompt information or the second prompt information can be expressed in the form of text or data string. When the first prompt information is expressed in the form of text, the first prompt information can be expressed as "encoded data is damaged" and the like; when the first prompt information is expressed in the form of data string, the data string can be 1 bit, 2 bits, 4 bits, 8 bits, etc., which can be expressed in the above-mentioned order as "1", "10", "1100" "00100111", it should be noted that the above-mentioned representation method of the first prompt information is only a few of the many embodiments, and does not limit the representation method of the first prompt information, in addition, it can also be converted to a decimal value through the data string, and it is judged whether the value exceeds the preset value, when it exceeds the preset value, the first prompt information is outputted; it can also be judged whether the number of 0s and 1s in the data string exceeds the preset number, if it exceeds the preset number, the first prompt information is outputted. The above-mentioned implementation manner does not make any limitation on the prompt information in the present application, and the implementation manner can be determined according to the implementation scene. Similarly, when the second prompt information is expressed in the form of text, the second prompt information can be expressed as "repaired" or "yes" and the like; when the second prompt information is expressed in the form of data string, the data string can be 1 bit, 2 bits, 4 bits, 8 bits, etc., which can be expressed in the above-mentioned order as "0", "01", "1010" "01100011", it should be noted that the above-mentioned representation method of the second prompt information is only a few of the many embodiments, and does not limit the representation method of the second prompt information, in addition, it can also be converted to a decimal value through the data string, and it is judged whether the value exceeds the preset value, when it exceeds the preset value, the second prompt information is outputted; it can also be judged whether the number of 0s and 1s in the data string exceeds the preset number, if it exceeds the preset number, the second prompt information is outputted. The above-mentioned implementation manner does not make any limitation on the second prompt information in the present application, and the implementation manner can be determined according to the implementation scene.

[0099] In the above embodiments, the data verification method has been described in detail. This application also provides embodiments corresponding to the data verification device. It should be noted that this application describes the embodiments of the device from two perspectives: one is based on the functional modules, and the other is based on the hardware.

[0100] Figure 2 This is a structural diagram of a data verification device provided in an embodiment of this application. Figure 2 As shown, this application also provides a data verification device, including:

[0101] The acquisition and encoding module 20 is used to acquire the raw data written to the disk through a write operation, and encode the raw data based on LDPC to obtain encoded data;

[0102] Generation module 21 is used to generate at least one verification copy and multiple ordinary copies based on the encoded data;

[0103] The module 22 is invoked and repaired to perform LDPC decoding on the verification copy when the coded data in multiple ordinary copies is determined to be corrupted through a read operation, so as to obtain the original data and repair the coded data in multiple ordinary copies based on the original data.

[0104] In addition, the device also includes the following modules:

[0105] Preferably, determining that the encoded data in multiple ordinary copies is corrupted through read operations includes:

[0106] The determination module is used to determine that the encoded data in multiple ordinary copies is corrupted when the write check code obtained through a write operation and the read check code obtained through a read operation are inconsistent.

[0107] Preferably, the write checksum obtained through the write operation and the read checksum obtained through the read operation include:

[0108] The first partitioning module is used to partition the encoded data and obtain multiple write check codes when the encoded data is written and stored to the disk through a write operation.

[0109] The second partitioning module is used to partition the encoded data stored in the verification copy and obtain multiple read verification codes when the read operation is called to perform LDPC decoding on the verification copy; among them, the write verification code and the read verification code correspond one-to-one.

[0110] Preferably, the division of coded data includes:

[0111] The third partitioning module is used to partition the encoded data according to the first preset standard, wherein the first preset standard is to partition the encoded data in units of 4k.

[0112] Preferably, the step of dividing the encoded data stored in the check copy comprises:

[0113] a fourth dividing module configured to divide the encoded data in the check copy according to a second preset standard, wherein the second preset standard is a data amount of the encoded data in the check copy corresponding to a data amount of the encoded data divided by 4k units.

[0114] Preferably, after determining the encoded data in the plurality of normal copies is damaged, the method further comprises:

[0115] a first outputting module configured to output first prompt information representing the encoded data in the plurality of normal copies is damaged.

[0116] Preferably, after repairing the encoded data in the plurality of normal copies according to the original data, the method further comprises:

[0117] a second outputting module configured to output second prompt information representing the encoded data in the plurality of normal copies is repaired.

[0118] Since the embodiments of the device part correspond to the embodiments of the method part, the embodiments of the device part are described in the description of the embodiments of the method part, and are not described here.

[0119] Figure 3 A data check device provided by the embodiments of the present application has a structure as shown in Figure 3 The data check device comprises:

[0120] a memory 30 configured to store a computer program;

[0121] a processor 31 configured to implement the steps of the data check method mentioned in the above embodiments when executing the computer program.

[0122] The data check device provided by the embodiments of the present application can include but is not limited to a smart phone, a tablet computer, a notebook computer or a desktop computer, etc.

[0123] The processor 31 can include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 31 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), a programmable logic array (PLA), and the like. The processor 31 can also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also referred to as a central processing unit (CPU). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 31 can be integrated with a graphics processor (GPU) for rendering and drawing content required to be displayed by the display screen. In some embodiments, the processor 31 can further include an artificial intelligence (AI) processor for processing machine learning related computing operations.

[0124] The memory 30 can include one or more computer-readable storage media that can be non-transitory. The memory 30 can also include a high-speed random access memory, and a nonvolatile memory such as one or more disk storage devices, flash storage devices. In this embodiment, the memory 30 is at least used to store the following computer programs, wherein the computer programs are loaded and executed by the processor 31, and can implement the related steps of the data verification method disclosed in any one of the preceding embodiments. In addition, the resources stored by the memory 30 can also include an operating system and data, and the storage mode can be temporary storage or permanent storage. The operating system can include Windows, Unix, Linux, and the like. The data can include but is not limited to the data verification method and the like.

[0125] In some embodiments, the data verification device can further include a display screen, an input / output interface, a communication interface, a power supply, and a communication bus.

[0126] Those skilled in the art can understand that the structure shown in the above embodiments does not constitute a limitation on the data verification device, and can include more or fewer components than those shown in the drawings. Figure 3

[0127] The data verification device provided by the embodiments of the present application includes a memory 30 and a processor 31. The processor 31 can implement the data verification method when executing the program stored in the memory 30. ​

[0128] Finally, the present application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps described in the above method embodiments.

[0129] It can be understood that if the method in the above embodiments is implemented in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and performs all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0130] The above provides a data verification method, device, equipment and medium. The description of the embodiments of the present application is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts of each embodiment can be referred to. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts are described in the method part. It should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, the present application can be improved and modified, and these improvements and modifications also fall within the protection scope of the claims of the present application.

[0131] It should be further noted that in the present specification, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or equipment including the element.

Claims

1. A data checking method, characterized by, The application is applied to a distributed storage system, and comprises the following steps: Obtaining original data written into a disk through a write operation, and encoding the original data based on LDPC to obtain encoded data; Generating at least one check copy and a plurality of normal copies according to the encoded data; When it is determined through a read operation that the encoded data in the plurality of normal copies is damaged, performing LDPC decoding on the check copy through the read operation to obtain the original data, and repairing the encoded data in the plurality of normal copies according to the original data.

2. The data check method of claim 1, wherein, The determination that the encoded data in the plurality of normal copies is damaged through the read operation comprises: When a write check code obtained through the write operation is inconsistent with a read check code obtained through the read operation, it is determined that the encoded data in the plurality of normal copies is damaged.

3. The data check method of claim 2, wherein, The write check code obtained through the write operation and the read check code obtained through the read operation comprise: When the encoded data is written and stored into the disk through the write operation, the encoded data is divided, and a plurality of write check codes are obtained; When the read operation is called to perform LDPC decoding on the check copy, the encoded data stored in the check copy is divided, and a plurality of read check codes are obtained; The write check code and the read check code correspond to each other.

4. The data check method of claim 3, wherein, The division of the encoded data comprises: The encoded data is divided according to a first preset standard, wherein the first preset standard is that the encoded data is divided in units of 4k.

5. The data check method of claim 4, wherein, The division of the encoded data stored in the check copy comprises: The encoded data in the check copy is divided according to a second preset standard, wherein the second preset standard is that the data amount of the encoded data in the check copy corresponds to the data amount of the encoded data divided in units of 4k.

6. The data check method of claim 2, wherein, After the determination that the encoded data in the plurality of normal copies is damaged, the method further comprises: Outputting first prompt information representing that the encoded data in the plurality of normal copies is damaged.

7. The data checking method of claim 1, wherein, After the repair of the encoded data in the plurality of normal copies according to the original data, the method further comprises: Outputting second prompt information representing that the encoded data in the plurality of normal copies damaged is repaired.

8. A data checking apparatus, characterized by comprising: The application comprises the following steps: An obtaining and encoding module is configured to obtain original data written into a disk through a write operation, and encode the original data based on LDPC to obtain encoded data; A generating module is configured to generate at least one check copy and a plurality of normal copies according to the encoded data; A calling and repairing module is configured to, when it is determined through a read operation that the encoded data in the plurality of normal copies is damaged, perform LDPC decoding on the check copy through the read operation to obtain the original data, and repair the encoded data in the plurality of normal copies according to the original data.

9. A data checking device, characterized by The application comprises the following steps: A memory is configured to store a computer program; A processor is configured to execute the computer program to implement the steps of the data check method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the data checking method in any one of claims 1 to 7.

Images