Data check method, and device, computer program product and storage medium

By employing cross-validation with different replication methods in a distributed storage system, different groups of nodes are selected to replicate data blocks and checksums, thus solving the problem of missed error detection during data replication and improving the reliability of data replication and the accuracy of verification.

WO2025248327A1PCT designated stage Publication Date: 2025-12-04CLOUD INTELLIGENCE ASSETS HOLDING (SINGAPORE) PTE LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
PCT/IB2025/053057
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2024-05-28
Filing Date
2025-03-24
Publication Date
2025-12-04

AI Technical Summary

Technical Problem

Existing distributed storage systems cannot detect errors in the data copying and reconstruction process in a timely manner, resulting in insufficient data replication reliability, and there are also issues with missed detections in read operation verification.

Method used

By storing data blocks and checksums on multiple data nodes and using cross-validation with different replication methods, different node groups are selected to replicate the data blocks and checksums, ensuring the reliability of the verification results.

Benefits of technology

It effectively covers the verification of the entire data replication chain, reduces the probability of false negatives in read operations, and improves the reliability of data replication.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure IB2025053057_04122025_PF_FP_ABST
    Figure IB2025053057_04122025_PF_FP_ABST
Patent Text Reader

Abstract

Provided in the embodiments of the present disclosure are a data check method, and a device, a computer program product and a storage medium. Target data is stored on a plurality of data nodes, and data blocks generated for the target data and check codes corresponding to the data blocks are stored on the data nodes. When it is necessary to perform data block replication for the target data, it is proposed to preferentially select different replication modes for a target data block to be obtained by means of replication and a target check code corresponding to the target data block, so as to perform cross-validation by means of the different replication modes, so that error detection of a data copy / reconstruction stage in a data replication link can be realized; and it is further proposed to select a first node group and a second node group, which are different from each other, so as to perform cross-validation by means of different data sources, so that not only can the error detection of the data copy / reconstruction stage in the data replication link be realized, but the false negative probability of errors in read operations can also be effectively reduced. The two cross-validation concepts, either individually or in combination, can effectively improve the reliability of data replication in a storage system.
Need to check novelty before this filing date? Find Prior Art

Description

[0001]This disclosure claims priority to Chinese Patent Application No. 202410676328.9, filed on May 28, 2024, entitled "A Data Verification Method, Device, Computer Program Product, and Storage Medium," the entire contents of which are incorporated herein by reference. Technical Field This disclosure relates to the field of storage technology, and more particularly to a data verification method, device, computer program product, and storage medium. Background Art In distributed storage systems, data is typically stored in multiple copies or erasure coding schemes. During operation, distributed storage systems may experience node failures or node up / down states, necessitating data replication. Data replication refers to the operation of creating a new copy of data to replace the old data in a storage system through copying or reconstructing. To ensure data reliability, distributed storage systems need to verify the data generated by data replication. Currently, data verification in distributed storage systems mainly focuses on verifying read / write operations, that is, verifying whether the read data conforms to the checksum, and verifying whether the written data conforms to the checksum. Thus, for the data replication process, only errors occurring during read or write operations can be detected, but errors occurring during data copying / reconstruction cannot be detected. Furthermore, read operation verification may miss errors because incorrect read data may still conform to the checksum (e.g., CRC32 encoding collisions). Such missed errors lead to incorrect data replication results, but cannot be detected. Therefore, current data verification schemes cannot detect data replication errors in a timely manner, resulting in insufficient reliability of data replication. The present invention discloses a data verification method, device, computer program product, and storage medium to improve the reliability of data replication in storage systems.This disclosure provides a data verification method. Target data is stored on multiple data nodes. Each data node stores data blocks generated for the target data and corresponding checksums. The method includes: responding to a data block copying instruction initiated for the target data; selecting a first node group and a second node group from the multiple data nodes, wherein the data nodes in the first node group and the second node group differ; copying a required target data block based on the data blocks stored for the target data in the first node group; and copying a target checksum corresponding to the target data block based on the checksums stored for the target data in the second node group; if the copied target checksum matches the copied target data block, then the copied target data block is determined to be reliable. Further, selecting a first node group and a second node group from the plurality of data nodes includes: after selecting the first node group or the second node group from the plurality of data node groups, preferentially selecting data nodes for the other node group from the remaining nodes in the plurality of data nodes; if the remaining nodes are insufficient, supplementing the other node group with data nodes from the already selected node groups. Further, the method also includes: determining a copying method for the target data block and the target checksum respectively; copying the required target data block includes: copying the target data block according to the first copying method determined for the target data block; copying the target checksum corresponding to the target data block includes: copying the target checksum according to the second copying method determined for the target checksum. Further, selecting different first node groups and second node groups from the plurality of data nodes includes: selecting the first node group from the plurality of data nodes according to the data requirements corresponding to the storage method of the target data and the first replication method; and selecting the second node group from the plurality of data nodes according to the data requirements corresponding to the storage method of the target data and the second replication method.Further, according to the storage method of the target data and the data requirements corresponding to the first replication method, the first node group is selected from the plurality of data nodes, including: if the storage method adopts erasure coding and the first replication method is copy, then the data node where the target data block is located is selected from the plurality of data nodes to form the first node group; if the storage method adopts erasure coding and the first replication method is reconstruction, then k data nodes are selected from the remaining nodes other than the data node where the target data block is located from the plurality of data nodes to form the first node group, where k is the number of data blocks required for reconstruction; if the storage method adopts a multi-replica mechanism and the first replication method is copy, then the data node where any replica is located is selected from the plurality of data nodes to form the first node group. Further, the replication method is determined for the target data block and the target checksum respectively, including: when the target data supports multiple replication methods, different replication methods are selected for the target data block and the target checksum. Further, copying the target data block according to the first copying method includes: if the target data is stored using an erasure coding mechanism and the first copying method is copy, then performing a copy operation on the target data block at the data node where the target data block is located to copy the target data block; if the target data is stored using an erasure coding mechanism and the first copying method is reconstruction, then reading the data block where the target data is stored from the first node group; and reconstructing the target data block based on the read data block. Further, copying the target checksum according to the second copying method includes: if the target data is stored using an erasure coding mechanism and the second copying method is reconstruction, then reading the checksum where the target data is stored from the second node group; and reconstructing the target checksum based on the read checksum. If the target data is stored using an erasure coding mechanism and the second copying method is copy, then performing a copy operation on the target checksum at the data node where the target checksum is located to copy the target checksum. Further, based on the read checksum, reconstructing the target checksum includes: if an erasure coding relationship exists between the checksums stored in the target data, then encoding the target checksum based on the erasure coding relationship between the read checksum and the target checksum. Further, the method also includes: if the target data supports a single copying method, selecting the copying method supported by the target data for the target data block and the target checksum.Furthermore, the target data supports copying and / or replication. This disclosure also provides a data verification method, wherein the target data is stored on multiple data nodes, and each data node stores a data block generated for the target data and a corresponding checksum for the data block. The method includes: when the target data supports multiple replication methods, in response to a data block replication instruction initiated for the target data, selecting different replication methods for the target data block to be replicated and the target checksum corresponding to the target data block; replicating the required target data block according to the first replication method selected for the target data block based on the data block stored for the target data; and replicating the target checksum according to the second replication method selected for the target checksum based on the checksum stored for the target data; if the replicated target checksum matches the replicated target data block, then the replicated target data block is determined to be reliable. This disclosure also provides a computing device, including a memory, a processor, and a communication component; the memory is used to store one or more computer instructions; the processor is coupled to the memory and the communication component, and is used to execute the one or more computer instructions to perform the aforementioned data verification method. This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by one or more processors, causes the one or more processors to perform the aforementioned data verification method. This disclosure also provides a computer program product, including a computer program, which, when executed by one or more processors, causes the one or more processors to perform the aforementioned data verification method. In this embodiment, the target data is stored on multiple data nodes, and it is proposed to store data blocks generated for the target data and corresponding check codes on the data nodes. Based on this, when data blocks need to be copied for the target data, it is proposed to prioritize the selection of different copying methods for the target data blocks to be copied and the target check codes corresponding to the target data blocks. This can prevent the copied target data blocks and copied target check codes from malfunctioning synchronously if a certain copying method has persistent problems in the execution entity. This concept of cross-validation through different copying methods can realize error detection in the data copy / reconstruction link of the data copying chain, and improve the problem of missed error detection in this link.The paper also proposes selecting distinct first and second node groups from these multiple data nodes. This ensures that the data sources used to replicate the target data block and the target checksum are different. This concept of cross-validation through different data sources not only enables error detection in the data copy / reconstruction stage of the data replication chain, improving the problem of missed error detection in this stage, but also effectively reduces the probability of missed error detection for read operations. The above two cross-validation concepts, alone or in combination, can effectively improve the reliability of data replication in the storage system. The accompanying drawings, which are provided to further illustrate this disclosure and constitute a part of this disclosure, illustrate exemplary embodiments and their descriptions, and are used to explain this disclosure, but do not constitute an undue limitation of this disclosure. In the accompanying drawings: Figure 1 is a flowchart illustrating a data verification method provided in an exemplary embodiment of this disclosure; Figure 2 is a logical diagram illustrating a data verification method provided in an exemplary embodiment of this disclosure; Figure 3 is a logical diagram illustrating an exemplary copying scheme provided in an exemplary embodiment of this disclosure; Figure 4 is a logical diagram illustrating another exemplary copying scheme provided in an exemplary embodiment of this disclosure; Figure 5 is a logical diagram illustrating yet another exemplary copying scheme provided in an exemplary embodiment of this disclosure; Figure 6 is a flowchart illustrating a data verification method provided in yet another exemplary embodiment of this disclosure; Figure 7 is a structural diagram illustrating a computing device provided in yet another exemplary embodiment of this disclosure. Detailed Description: To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions of this disclosure will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure. Before providing a detailed description of the technical solutions provided in the embodiments of this disclosure, the following is a brief explanation of several technical concepts involved in this disclosure: A distributed storage system is a storage system that distributes data across multiple storage nodes. A distributed storage system typically contains several data nodes. A data node is a node in a distributed storage system used to store data and can perform tasks such as data reading, writing, and deletion. Erasure coding (EC) is a distributed storage mechanism or a data protection mechanism. It generates a set of erasure code fragments corresponding to the data by dividing the data into data blocks and creating redundant check blocks, which are usually called erasure code fragment groups (EC groups); these erasure code fragments are then stored on different storage nodes.In this way, even if some erasure coding fragments are missing, the original data can be recovered by combining other erasure coding fragments. Furthermore, some erasure coding fragments can be used to reconstruct other erasure coding fragments. For example, if the erasure coding fragment group contains k+m erasure coding fragments, any one of the remaining m erasure coding fragments can be reconstructed using any k of them. Multiple replicas are also a distributed storage mechanism or data protection mechanism. It generates multiple copies of the data and stores these copies on different data nodes. Thus, even if a copy is missing on some data nodes, the other copies can still guarantee the recovery of the original data. Data replication refers to the operation of creating a new copy of data to replace the old data in a storage system through data copying or data reconstruction. As introduced in the background technology, distributed storage systems require data replication due to various situations, and the data generated by data replication also needs to be verified to ensure the reliability of data in the distributed storage system. Current data verification schemes typically only verify read / write operations in the data transmission link of the data replication chain, and they also suffer from the problem of missing errors in read operations. Therefore, current data verification schemes cannot detect data replication errors in a timely manner, resulting in insufficient reliability of data replication. To address this, this disclosure proposes a data verification method that covers the entire data replication chain and can effectively improve the problem of missing errors in read operations, thereby enabling timely detection of data replication errors and effectively improving the reliability of data replication. The technical solutions provided by various embodiments of this disclosure are described in detail below with reference to the accompanying drawings. Figure 1 is a schematic flowchart of a data verification method provided by an exemplary embodiment of this disclosure. This method can be executed by a data verification device, which can be implemented as software, hardware, or a combination of software and hardware, and can be integrated into a computing device. It should be understood that the data verification method provided in this embodiment performs reliability verification on data replication. Therefore, the computing device in this embodiment can be the replication end originally responsible for data replication in a distributed storage system. For example, the replication end can be any data node in the distributed storage system. That is, the data verification device in this embodiment can be integrated into the replication end of the distributed storage system. Of course, this embodiment is not limited to this. The computing device in this embodiment can also be an independent device set up outside the distributed storage system, etc., and there is no limitation on the deployment method. Figure 2 is a logical schematic diagram of a data verification method provided by an exemplary embodiment of this disclosure.Referring to Figure 2, in this embodiment, the target data is stored on multiple data nodes. The data verification method in this embodiment supports storage methods such as erasure coding or multiple replicas for the target data. For ease of description, in this embodiment, erasure coding fragments or replicas stored on the data nodes for the target data are collectively referred to as data blocks. That is, under the erasure coding mechanism, the data blocks described in this embodiment are erasure coding fragments; under the multiple replica mechanism, the data blocks described in this embodiment are replicas. Furthermore, this embodiment proposes generating a checksum for each data block and storing it in association with the data block. Thus, in this embodiment, in addition to storing the target data blocks, the data nodes also store the checksums corresponding to the data blocks. In this embodiment, a checksum algorithm can be used to perform verification calculations on the data blocks themselves to generate checksums for the data blocks. The principle of the checksum algorithm will not be detailed here. Moreover, the checksums in this embodiment can be generated not only by performing verification calculations on the data blocks themselves, but also by reconstructing calculations based on the mathematical relationships between the checksums. The latter generation method is similar to the reconstruction calculation of erasure code slices in the erasure coding mechanism. Based on this, referring to Figure 1, the data verification method provided in this embodiment may include: Step 100: In response to a data block copying instruction initiated for the target data, a first node group and a second node group are selected from multiple data nodes, wherein the data nodes contained in the first node group and the data nodes contained in the second node group are different; Step 101: Based on the data blocks stored for the target data in the first node group, the required target data block is copied, and based on the checksum stored for the target data in the second node group, the target checksum corresponding to the target data block is copied; Step 102: If the copied target checksum matches the copied target data block, the copied target data block is determined to be reliable. In Step 100, the data block copying instruction indicates the identification information of the target data block to be copied. Thus, based on the data block copying instruction, it can be determined which of the multiple data blocks corresponding to the target data needs to be copied. In step 100, in response to a data block copy instruction, a first node group and a second node group can be selected from multiple data nodes used to store the target data. Here, the data nodes contained in the first node group and the data nodes contained in the second node group are different, referring to Figure 2; that is, the data nodes in the two node groups selected in step 100 will not completely overlap.Based on this, in this embodiment, the first node group can be used as the data source for copying the target data block, while the second node group can be used as the data source for copying the target checksum corresponding to the target data block. Thus, in this embodiment, the data source for copying the target data block and the data source for copying the target checksum corresponding to the target data block will be different. In this embodiment, the number of nodes in the first and second node groups is not limited; any number of nodes capable of supporting the copy can be used. That is, in this embodiment, the number of nodes in the first and second node groups can be equal to or greater than the number of nodes required to complete the copy. Referring to Figure 1, in step 101, the required target data block can be copied based on the data block stored for the target data in the first node group; the target checksum corresponding to the target data block can also be copied based on the checksum stored for the target data in the second node group. It should be understood that the copy operation in step 101 is the aforementioned data copying. The copying method can include copying or reconstruction. In step 101, the replication method for the target data and the replication method for the target checksum can be determined as needed. This embodiment does not limit the replication method. Thus, in step 101, the target data block and the target checksum corresponding to the target data block can be replicated. Referring again to Figure 1, in step 102, the replicated target data block can be reliably verified based on the replicated target checksum. If the replicated target checksum matches the replicated target data block, the replicated target data block is determined to be reliable. As mentioned earlier, in this embodiment, in addition to storing the target data block, the data node also stores the checksum corresponding to the data block. In the storage stage, a verification calculation rule can be designed. By performing verification calculations on the data block, the checksum corresponding to the data block can be generated, and the checksum is associated with the data block for storage. Thus, in this embodiment, the data block is associated with a checksum when it is stored. Based on this, the matching mentioned in step 102 can be understood as the target checksum being equal to the checksum that the target data block should possess when it is stored. Optionally, in step 102, a checksum calculation can be performed based on the copied target data block itself to generate a new checksum for the copied target data block. Then, the new checksum and the copied target checksum can be compared. If they match, it is determined that the copied target checksum matches the copied target data block; otherwise, it is determined that they do not match.In this embodiment, when performing verification calculations on the target data block itself, the verification calculation rules designed in the storage stage can be adopted. Based on this, if it is determined that the copied target checksum does not match the copied target data block, steps 100-102 can be re-executed. During re-execution, the first node group and the second node group can be adjusted in step 100, and the copying method can be adjusted in step 101, etc. If, after multiple re-executions, the copied target checksum still does not match the copied target data block, manual intervention can be performed to manually check for errors throughout the data copying process. Referring to Figure 2, firstly, in this embodiment, both the target data block and the target checksum are copied. If there is an error in the process of copying the target data block or the target checksum, or if there is an error in the storage stage of the data block stored in the first node group or the checksum stored in the second node group, the verification result in step 102 will be a mismatch. Therefore, the data verification method provided in this embodiment can cover the data copying / reconstruction and data storage stages in the data replication chain. Combined with the coverage of the data transmission stage in the data replication chain in traditional solutions, the data verification method provided in this embodiment can cover the entire data replication chain. Secondly, as mentioned earlier, the first node group and the second node group in this embodiment are different. Therefore, in this embodiment, the data source used to replicate the target data block and the data source used to replicate the target checksum are different. This is equivalent to using different data sources to perform cross-verification on the entire data replication chain, which can effectively reduce the probability of false negatives in read operations. This is because, if there is a special node with an inherently erroneous data block among multiple data nodes, based on the cross-validation mechanism proposed in this embodiment, the probability that this special node is simultaneously located in the first node group and the second node group is very low. Even if the special node is located in both the first and second node groups, since the first and second node groups are different, the data relationship used when copying the target data block from the first node group is different from the data relationship used when copying the target checksum from the second node group. The probability that the erroneous data block and its associated checksum can simultaneously conform to these two different data relationships is even lower. Thus, even if the special node is missed during the read operation verification process, it will still cause verification failure in step 102 of this embodiment. For example, if the target data is stored on 6 data nodes: data node 0-data node 6, where data node 0 is the aforementioned special node.Based on step 100 in this embodiment, data node 0 is assigned to the second node group, but the first node group does not contain data node 0. Since the data blocks in the first node group are all correct, the target data block copied from the first node group is correct. However, because the data block in data node 0 itself is incorrect, even though the checksum stored on it is associated with and matches the data block, this checksum is actually incorrect. Therefore, the target data copied according to the data relationship between the checksums in the second node group will be incorrect. Accordingly, the copied target checksum will not match the copied target data block, thus avoiding the problem of missed checks in the read operation verification process. For example, the target data is stored on six data nodes: data node 0 to data node 6, where data node 0 is the aforementioned special node that stores the incorrect data block and checksum. If data node 0 is simultaneously assigned to both the first and second node groups, then during the replication of the target data block based on the first node group, taking the replication through reconstruction under the erasure coding mechanism as an example, the erroneous data block needs to be encoded and decoded together with other data blocks in the first node group (first type of data relationship) to generate the target data block. During the replication of the target checksum based on the second node group, the checksum corresponding to the coarse even data block needs to be encoded and decoded together with other checksums in the second node group (second type of data relationship) to generate the target checksum. Since the other data nodes cooperating with data node 0 in the first data relationship are different from those cooperating with data node 0 in the second data relationship, the replicated target data block and the target checksum will inevitably have different deviations. That is, it is virtually impossible for them to have completely identical errors. Therefore, the replicated target data block and the target checksum will be mismatched. In this case, the problem of missed detection in the read operation verification stage can also be avoided. In summary, this embodiment stores the target data on multiple data nodes, and proposes storing data blocks generated for the target data and corresponding checksums on these data nodes. Based on this, when data block copying of the target data is required, a first node group and a second node group, which are different from the multiple data nodes, can be selected. Specifically, the required target data block can be copied based on the data blocks stored for the target data within the first node group; and a target checksum corresponding to the target data block can be copied based on the checksums stored for the target data within the second node group. If the copied target checksum matches the copied target data block, the copied target data block can be determined to be reliable.Accordingly, on the one hand, since both the target data block and the target checksum used for verification are replicated, the verification result can be guaranteed to cover the entire data block replication chain. On the other hand, by selecting different first and second node groups, the data sources used to replicate the target data block and the target checksum can be different. This is equivalent to cross-validating the entire data block replication chain using different data sources. This cross-validation mechanism can effectively reduce the probability of false negatives in read operations. Through the combination of these two aspects, the reliability of data replication in the storage system can be effectively improved. In the above or below embodiments, various implementation methods can be used to select the first and second node groups. The following provides a preferred implementation method. In this preferred implementation method, after selecting the first or second node group from multiple data node groups, data nodes are preferentially selected from the remaining nodes in the multiple data nodes for another node group; if the remaining nodes are insufficient, data nodes are then supplemented from the already selected node groups for another node group. In this preferred implementation, there is no limitation on which node group is selected first; that is, either the first node group or the second node group can be selected first. When selecting data nodes for another node group after selecting one node group, priority can be given to selecting from the remaining nodes among the multiple data nodes. Here, remaining nodes refer to data nodes that were not selected in the previous node group. For example, if the first node group is selected first, data nodes for the second node group can be selected from the data nodes that were not selected in the first node group. During the research process, the inventors discovered that when the number of multiple data nodes used to store the target data is relatively large, the remaining nodes are sufficient to support the selection of the subsequent node group. In this case, the selected first node group and the second node group will not store the same nodes; that is, the first node group and the second node group do not overlap. However, there may also be a situation where there are insufficient remaining nodes. In this case, in this preferred implementation, data nodes can be supplemented for the other node group from the already selected node groups. Taking the initial selection of the first node group as an example, if the number of nodes in the second node group is still insufficient after the remaining nodes are selected, some data nodes from the first node group can be selected to supplement the second node group. In this case, the first and second node groups will store some identical data nodes. It can be seen that in this preferred implementation, the intersection between the first and second node groups can be minimized. Based on the principle of cross-validation mechanism's improvement on read operation error detection, as explained above, the smaller the intersection between the first and second node groups, the lower the probability of read operation error detection.Therefore, the intersection minimization scheme proposed in this preferred implementation can effectively reduce the probability of false negatives in read operations. Furthermore, this embodiment also proposes a preferred implementation to more rationally allocate data nodes and determine the number of nodes for the first and second node groups. This preferred implementation proposes: determining the replication method for the target data block and the target checksum respectively; selecting the first node group from multiple data nodes according to the storage method of the target data and the data requirements corresponding to the replication method determined for the target data block; and selecting the second node group from multiple data nodes according to the storage method of the target data and the data requirements corresponding to the replication method determined for the target checksum; wherein the replication method includes copying or reconstructing. In this preferred implementation, the storage method of the target data and the replication method determined for the target data block and the target checksum are comprehensively considered when allocating data nodes and determining the number of nodes for the first and second node groups. This ensures that the selected data nodes in the first and second node groups not only support the completion of replication, but also that the number of nodes precisely meets the replication requirements. In this way, the number of nodes in the first and second node groups can be reduced, which can effectively reduce the overlap between the two groups while ensuring the need for replication. As mentioned earlier, the storage method of the target data may include erasure coding mechanism or multiple copy mechanism, and the replication method supported in this embodiment may include copying or reconstruction. Hereinafter, in this preferred implementation, a scheme for allocating data nodes to the first and second node groups will be provided for various combinations of storage and replication methods. Since the logic for allocating data nodes to the first and second node groups is consistent, the following explanation uses the first node group as an example: In this preferred implementation, if the storage method uses erasure coding and the replication method determined for the target data block is copy, then the data node containing the target data block is selected from multiple data nodes to form the first node group. If the storage method uses erasure coding and the replication method determined for the target data block is reconstruction, then k data nodes are selected from the remaining nodes excluding the data node containing the target data block to form the first node group, where k is the number of data blocks required for reconstruction. If the storage method uses a multi-replica mechanism and the replication method determined for the target data block is copy, then the data node containing any replica is selected from multiple data nodes to form the first node group. It should be understood that the same allocation scheme can be used to allocate data nodes to the second node group. Furthermore, the two preferred implementations provided in this embodiment can be combined with each other.It is worth noting that, in this embodiment, besides the preferred implementation described above, other implementation methods can also be used to select the first node group and the second node group, as long as the first node group and the second node group can support the completion of replication. Further examples of implementation methods are not provided here. Accordingly, in this embodiment, data can be reasonably allocated to the first node group and the second node group and the number of nodes can be determined based on the storage method of the target data and the replication method determined for the target data block and the target checksum. Furthermore, the intersection of the first node group and the second node group is minimized, thereby better reducing the probability of false negatives in read operations and further improving the reliability of data replication. In the above or following embodiments, replication methods can be determined for the target data block and the target checksum respectively. Based on this, the target data block can be replicated according to the first replication method determined for the target data block; and the target checksum can be replicated according to the second replication method determined for the target checksum. In this embodiment, multiple implementation methods can be used to determine the replication method for the target data block and the target checksum. In one preferred implementation, when the target data supports multiple replication methods, different replication methods can be preferentially selected for the target data block and the target checksum. Since the replication operations for the target data block and the target checksum in this embodiment are both performed by the same execution entity (i.e., the data verification device or computing device in this embodiment), if a certain replication method in the execution entity continues to have problems, by preferentially selecting different replication methods in this preferred implementation, it is possible to avoid the synchronized errors of the replicated target data block and the replicated target checksum. This further improves the error detection rate of the data copy / reconstruction stage in the data replication chain, avoiding the problem of missed errors in this stage. In this preferred implementation, an exemplary replication method selection scheme can be: if the target data storage method adopts an erasure coding mechanism, then the first replication method can be determined as reconstruction, and the second replication method can be determined as copy. Another exemplary replication method selection scheme can be: if the target data storage method adopts an erasure coding mechanism, then the first replication method can be determined as copy, and the second replication method can be determined as reconstruction. In this preferred implementation, a scheme for copying the target data block and the target checksum will be provided for various combinations of storage and copying methods. Figure 3 is a logical schematic diagram of an exemplary copying scheme provided by an exemplary embodiment of this disclosure. Referring to Figure 3, in this exemplary copying scheme: if the target data is stored using an erasure coding mechanism and the first copying method is copy, then a copying operation is performed on the target data block from the data node where the target data block is located to copy the target data block.Referring to Figure 3, there are six data nodes used to store the target data: data node 1 to data node 6. Additionally, data node 7 in the distributed storage system serves as the computing device (or replication endpoint) in this embodiment. The replication method determined for the target data block is copy. Based on the data requirements of this copy method, the first node group only needs to contain one data node, namely data node 6 where the target data block is located. On this basis, data node 7 can perform a copy operation (which can be understood as a read operation) on the target data block on data node 6 to replicate the target data block on data node 7. Continuing to refer to Figure 3, in this exemplary replication scheme: a different replication method is selected for the target checksum than for the target data block; that is, the second replication method is reconstruction. Based on this, if the target data storage method uses erasure coding and the second replication method is reconstruction, then the checksum stored in the target data is read from the second node group; based on the read checksum, the checksum corresponding to the target checksum is reconstructed. Referring to Figure 3, the second node group includes data nodes 1-4. Based on this, data node 7 can read the checksums for storing the target data from these four data nodes, and reconstruct the target checksum corresponding to the target data block using these checksums. It can be seen that in this exemplary replication scheme, different replication methods are used to replicate the target data block and the target checksum, and the first and second node groups have no overlap. This can promptly detect errors in the entire data replication chain, thereby ensuring the reliability of data replication. Figure 4 is a logical schematic diagram of another exemplary replication scheme provided by an exemplary embodiment of this disclosure. Referring to Figure 4, in this exemplary replication scheme: if the target data is stored using erasure coding and the first replication method is reconstruction, then the data block for storing the target data is read from the first node group; based on the read data block, the target data block is reconstructed. Referring to Figure 4, there are six data nodes used to store the target data: data nodes 1-6. Additionally, data node 7 in the distributed storage system serves as the computing device (or replication end) in this embodiment. The first replication method is reconstruction. Based on the data requirements of this replication method, the first node group contains four data nodes: data node 1 to data node 4. On this basis, data node 7 can read the data blocks stored for the target data from data nodes 1 to 4 respectively, and reconstruct the target data block on data node 7 according to the erasure coding rules. Referring again to Figure 4, in this exemplary replication scheme: a different replication method is selected for the target checksum than for the target data block; that is, the second replication method is copying.Therefore, if the target data is stored using erasure coding and the second replication method is copying, then a copy operation is performed on the target checksum from the data node where the target checksum is located to replicate the target checksum. Referring to Figure 4, based on the data requirements of this copying method, the second node group only needs to include one data node 6. Based on this, data node 7 can perform a copy operation (which can be understood as a read operation) on the target checksum stored on data node 6 to replicate the target checksum corresponding to the target data block on data node 7. It can be seen that in this exemplary replication scheme, different replication methods are used to replicate the target data block and the target checksum, and the first and second node groups have no overlap. This can promptly detect errors throughout the entire data replication chain, thereby ensuring the reliability of data replication. During their research, the inventors discovered that multiple replication methods for target data are not always guaranteed. For example, under erasure coding, if the data node containing the target data block is disconnected, neither the target data block nor the target checksum can be copied; both must be replicated using the same replication method – reconstruction. Similarly, under multiple replica mechanisms, reconstruction is not supported, so both must be replicated using the same replication method – copy. Therefore, this preferred implementation proposes that when the target data supports only one replication method, the replication method supported by the target data is selected for both the target data block and the target checksum; that is, the first and second replication methods are the same. Figure 5 is a logical schematic diagram of another exemplary replication scheme provided by an exemplary embodiment of this disclosure. Referring to Figure 5, in this exemplary replication scheme: If the storage method adopts erasure coding and the replication method determined for both the target data block and the target checksum is reconstruction, then the data block stored for the target data can be read from the first node group; based on the read data block, the target data block is reconstructed; and, the checksum stored for the target data can be read from the second node group; based on the read checksum, the target checksum is reconstructed. Referring to Figure 5, there are 12 data nodes used to store the target data: data nodes 1-6 and data nodes 8-13. In addition, data node 7 in the distributed storage system serves as the computing device (or replication end) in this embodiment. The first replication method is reconstruction. Based on the data requirements of this replication method, the first node group includes 6 data nodes: data nodes 1-3 and data nodes 8-19. On this basis, data node 7 can read the data block stored for the target data from these 6 data nodes respectively, so as to reconstruct the target data block on data node 7 according to the erasure coding rules.The second node group contains six data nodes: data nodes 3-5 and data nodes 11-13. Based on this, data node 7 can read the checksums stored as target data from these six data nodes and reconstruct the target checksum corresponding to the target data block using these checksums. It can be seen that in this exemplary replication scheme, the same replication method is used to replicate the target data block and the target checksum. Furthermore, the first and second node groups store the minimum intersection [data node 3]. In this exemplary replication scheme, errors throughout the entire data replication chain can still be detected in a timely manner, thereby ensuring the reliability of data replication. The above exemplary replication schemes involve the process of replicating the target checksum using a reconstruction method. In this process, the reconstruction calculation can be performed based on the mathematical relationships between the checksums stored as target data. Here, the mathematical relationship can preferably be an erasure coding relationship, meaning that the checksums stored for the target data also conform to the erasure coding mechanism: if the target data has k+m checksums, then any one of the remaining m checksums can be reconstructed using any k checksums according to the erasure coding rules. In other words, if there is an erasure coding relationship between the checksums stored for the target data, then the target verification code can be encoded based on the erasure coding relationship between the read checksum and the target checksum. The above section described several exemplary replication schemes for the case where the storage method uses an erasure coding mechanism. Below, we will describe exemplary replication schemes for the case where the storage method uses a multi-replica mechanism. As mentioned earlier, under a multi-replica mechanism, the replication method determined for the target data block and the target checksum will be copying. Based on this, in an exemplary replication scheme: if the storage method employs a multi-replica mechanism and the replication method for both the target data block and the target checksum is copy, then a copy operation is performed on the target data block from the data nodes included in the first node group to replicate the target data block; and a copy operation is performed on the target checksum from the data nodes included in the second node group to replicate the target checksum. The first node group and the second node group may be independent. It can be seen that in this exemplary replication scheme, the same replication method is used to replicate the target data block and the target checksum, and the first node group and the second node group are independent. In this exemplary replication scheme, errors throughout the entire data replication chain can still be detected in a timely manner, thereby ensuring the reliability of data replication.In summary, this embodiment proposes prioritizing the determination of different replication methods for the target data block and the target checksum. This prevents the replicated target data block and the replicated target checksum from simultaneously malfunctioning, even if a particular replication method consistently presents problems in the execution entity. This further improves the error detection rate of the data copy / reconstruction stage in the data replication chain, avoiding missed errors in this stage. If it is not possible to determine different replication methods for the target data block and the target checksum, using the same replication method for both aspects can still ensure end-to-end verification of the data replication chain. Figure 6 is a flowchart illustrating a data verification method provided in another exemplary embodiment of this disclosure. This method can be executed by a data verification device, which can be implemented as software, hardware, or a combination of both. The data verification device can be integrated into a computing device. In this embodiment, the target data is stored on multiple data nodes. Each data node stores data blocks generated for the target data and corresponding checksums. Based on this, referring to Figure 6, the method may include: Step 600: When the target data supports multiple replication methods, in response to a data block replication command initiated for the target data, different replication methods are selected for the target data blocks to be replicated and the target checksums corresponding to the target data blocks; Step 601: Based on the data blocks stored for the target data, the required target data blocks are replicated according to the first replication method selected for the target data blocks; and based on the checksums stored for the target data, the target checksums are replicated according to the second replication method selected for the target checksums; Step 602: If the replicated target checksums match the replicated target data blocks, the replicated target data blocks are determined to be reliable. In this embodiment, the replication methods supported by the target data may include, but are not limited to, copying and reconstructing. In this embodiment, step 600 proposes that different replication methods be preferentially selected for the target data blocks and the target checksums. Thus, since the copying operation of the target data block and the copying operation of the target check code in this embodiment are both performed by the same execution entity (that is, the data verification device or computing device in this embodiment), if a certain copying method in the execution entity continues to have problems, by prioritizing the selection of different copying methods in this preferred implementation, it is possible to avoid the copied target data block and the copied target check code from simultaneously becoming erroneous. This can further improve the error detection rate of the data copy / reconstruction link in the data copying chain and avoid the problem of missed error detection in this link.In this embodiment, in step 601, data nodes can be randomly selected from multiple data nodes to provide the stored data blocks according to the number of nodes required for the first replication method, so as to support the replication of the target data block in this embodiment. The selected data nodes are not limited in this embodiment, as long as they can ensure the implementation of the first replication method. For example, if the first replication method uses reconstruction, then in step 601, any k data nodes (k being the number of data blocks required for reconstruction) can be selected from multiple data nodes, and the target data block can be reconstructed using the data blocks stored in the selected k data nodes. Similarly, in step 601, data nodes can be randomly selected from multiple data nodes to provide the stored checksum according to the number of nodes required for the second replication method, so as to support the replication of the target checksum in this embodiment. The selected data nodes are not limited in this embodiment, as long as they can ensure the implementation of the second replication method. For example, if the second replication method employs reconstruction, then in step 601, any k data nodes (k being the number of data blocks required for reconstruction) can be selected from multiple data nodes. The target checksum can be reconstructed using the checksums stored in the selected k data nodes. Preferably, in this embodiment, a first node group and a second node group can be selected from multiple data nodes, wherein the data nodes contained in the first node group and the data nodes contained in the second node group differ. Based on this, in step 601, the required target data block can be replicated according to the first replication method based on the data blocks stored for the target data in the first node group, and the target checksum corresponding to the target data block can be replicated according to the second replication method based on the checksums stored for the target data in the second node group. The inventors discovered during their research that different storage methods support different replication methods. The storage methods in this embodiment may include, but are not limited to, erasure coding mechanisms and multiple copy mechanisms. This embodiment proposes an exemplary replication method selection scheme: if the target data storage method adopts erasure coding mechanism, then the first replication method can be determined as reconstruction, and the second replication method can be determined as copy. Another exemplary replication method selection scheme can be: if the target data storage method adopts erasure coding mechanism, then the first replication method can be determined as copy, and the second replication method can be determined as reconstruction. In summary, this embodiment proposes prioritizing different replication methods for the target data block and the target checksum. This avoids the synchronized errors in the replicated target data block and the replicated target checksum when a certain replication method has persistent problems in the execution entity. This improves the error detection rate of the data copy / reconstruction stage in the data replication chain and mitigates the error omission problem in this stage.Furthermore, the inventors discovered during their research that multiple replication methods for target data are not always guaranteed. For example, under erasure coding, if the data node containing the target data block is disconnected, neither the target data block nor the target checksum can be replicated using the copy method; instead, both replication methods must use the same replication method—reconstruction. Similarly, under a multi-copy mechanism, reconstruction is not supported; therefore, both replication methods must use the same replication method—copy. To address this, this embodiment proposes that when the target data supports only one replication method, the replication method supported by the target data should be selected for both the target data block and the target checksum. That is, the first replication method and the second replication method should be the same. In this case, an exemplary replication method selection scheme could be: if the target data is stored using erasure coding, then the first replication method can be determined as reconstruction, and the second replication method can be determined as reconstruction. Another exemplary replication method selection scheme could be: if the target data is stored using a multi-copy mechanism, then the first replication method can be determined as copy, and the second replication method can be determined as reconstruction. No further examples of selection schemes are provided here. In this case, by combining the aforementioned selection of different first node groups and second node groups, cross-verification of the target data block and target checksum can be achieved through different data sources, ensuring that the data verification method provided in this embodiment covers the entire data replication chain. It is worth noting that some technical details are involved in this embodiment but are not described in detail. For these technical details, please refer to the technical descriptions in the relevant embodiments of Figures 1-5 above, which will not be repeated here, but this should not cause any loss of the protection scope of this embodiment. It should be noted that some processes described in the above embodiments and figures include multiple operations appearing in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear in this document or may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations, and the numbers themselves do not represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the terms "first" and "second" used in this document are used to distinguish different node groups, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types. Figure 7 is a schematic diagram of the structure of a computing device provided in another exemplary embodiment of this disclosure. As shown in Figure 6, the computing device includes: a memory 70 and a processor 71.The target data is stored on multiple data nodes, and each data node stores a data block generated for the target data and a corresponding checksum for the data block. Based on this, in one design concept, a processor 71, coupled to a memory 70, is used to execute a computer program in the memory 70 to: in response to a data block copying instruction initiated for the target data, select a first node group and a second node group from the multiple data nodes, wherein the data nodes in the first node group and the second node group differ; copy the required target data block based on the data blocks stored for the target data in the first node group; and copy a target checksum corresponding to the target data block based on the checksum stored for the target data in the second node group; if the copied target checksum matches the copied target data block, then the copied target data block is determined to be reliable. In an optional embodiment, when the processor 71 selects a first node group and a second node group from the plurality of data nodes, it may specifically be used to: after selecting the first node group or the second node group from the plurality of data node groups, preferentially select data nodes for the other node group from the remaining nodes in the plurality of data nodes; if there are insufficient remaining nodes, supplement the other node group with data nodes from the already selected node groups. In an optional embodiment, the processor 71 may also be used to determine a copying method for the target data block and the target checksum respectively; when the processor 71 copies the required target data block, it may specifically be used to: copy the target data block according to the first copying method determined for the target data block; when the processor 71 copies the target checksum corresponding to the target data block, it may specifically be used to: copy the target checksum according to the second copying method determined for the target checksum. In an optional embodiment, when the processor 71 selects different first node groups and second nodes from the plurality of data nodes, it may specifically be used to: select the first node group from the plurality of data nodes according to the data requirements corresponding to the storage method of the target data and the first replication method; and select the second node group from the plurality of data nodes according to the data requirements corresponding to the storage method of the target data and the second replication method.In an optional embodiment, when the processor 71 selects the first node group from the plurality of data nodes according to the data requirements corresponding to the storage method of the target data and the first replication method, it may specifically be used to: if the storage method adopts erasure coding mechanism and the first replication method is copy, then select the data node where the target data block is located from the plurality of data nodes to form the first node group; if the storage method adopts erasure coding mechanism and the first replication method is reconstruction, then select k data nodes from the remaining nodes other than the data node where the target data block is located from the plurality of data nodes to form the first node group, where k is the number of data blocks required for reconstruction; if the storage method adopts multi-replica mechanism and the first replication method is copy, then select the data node where any replica is located from the plurality of data nodes to form the first node group. In an optional embodiment, when the processor 71 determines the replication method for the target data block and the target checksum respectively, it may specifically be used to: when the target data supports multiple replication methods, select different replication methods for the target data block and the target checksum. In an optional embodiment, when the processor 71 copies the target data block according to the first copying method, it may specifically be used to: if the target data is stored using an erasure coding mechanism and the first copying method is copy, then perform a copy operation on the target data block on the data node where the target data block is located to copy the target data block; if the target data is stored using an erasure coding mechanism and the first copying method is reconstruction, then read the data block that is the target data stored in the first node group; and reconstruct the target data block based on the read data block. In an optional embodiment, when the processor 71 copies the target checksum according to the second copying method, it may specifically be used to: if the target data is stored using an erasure coding mechanism and the second copying method is reconstruction, then read the checksum stored as the target data from the second node group; reconstruct the target checksum based on the read checksum; if the target data is stored using an erasure coding mechanism and the second copying method is copy, then perform a copy operation on the target checksum from the data node where the target checksum is located to copy the target checksum. In an optional embodiment, when the processor 71 reconstructs the target checksum based on the read checksum, it may specifically be used to: if there is an erasure coding relationship between the checksums stored for the target data, then encode the target checksum based on the erasure coding relationship between the read checksum and the target checksum.In an optional embodiment, the processor 71 may also be configured to: select a copying method supported by the target data for the target data block and the target checksum, if the target data supports a single copying method. In an optional embodiment, the copying methods supported by the target data include copying and / or duplication. In another design concept, the processor 71 may be coupled to the memory 70 for executing a computer program in the memory 70 to: select different copying methods for the target data block to be copied and the target checksum to be copied for the target data block, in response to a data block copying instruction initiated for the target data, if the target data supports multiple copying methods; copy the required target data block according to a first copying method selected for the target data block based on the data block stored for the target data; and copy the target checksum according to a second copying method selected for the target checksum based on the checksum stored for the target data; if the copied target checksum matches the copied target data block, then determine that the copied target data block is reliable. In one optional embodiment, the copying methods supported by the target data may include, but are not limited to, copying and reconstruction. In one optional embodiment, if the target data is stored using an erasure coding mechanism, the processor 71 may determine the first copying method as reconstruction and the second copying method as copying. Further, as shown in FIG7, the computing device also includes: a communication component 72, a power supply component 73, and other components. FIG6 only schematically shows some components and does not imply that the computing device only includes the components shown in FIG7. It is worth noting that the technical details of the above embodiments of the computing device can be referred to the relevant descriptions in the foregoing method embodiments. To save space, they will not be repeated here, but this should not cause a loss of the scope of protection of this disclosure. Accordingly, the embodiments of this disclosure also provide a computer-readable storage medium storing a computer program, which, when executed, can implement the steps in the above method embodiments. Accordingly, this disclosure also provides a computer program product, in which the included computer program, when executed, can implement the steps in the above-described method embodiments. The memory in FIG7 is used to store the computer program and can be configured to store various other data to support operation on a computing platform. Examples of such data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, pictures, videos, etc.The memory can be implemented using any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The communication component in Figure 7 above is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as Wi-Fi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IRDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies. The power supply component in Figure 7 above provides power to various components of the device in which the power supply component resides. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component resides. Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart and / or block diagram, and combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart one or more blocks and / or the block diagram one or more blocks.These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams. It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "including a..." does not exclude the presence of other identical elements in the process, method, product, or device that includes said element. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse. The above descriptions are merely embodiments of this disclosure and are not intended to limit this disclosure. For those skilled in the art, this disclosure can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

Claims 1. A data validation method, wherein, The target data is stored on multiple data nodes, and each data node stores a data block generated for the target data and a checksum corresponding to the data block. The method includes: in response to a data block copying instruction initiated for the target data, selecting a first node group and a second node group from the multiple data nodes, wherein the data nodes contained in the first node group and the data nodes contained in the second node group are different; copying the required target data block based on the data block stored for the target data in the first node group; and copying a target checksum corresponding to the target data block based on the checksum stored for the target data in the second node group; if the copied target checksum matches the copied target data block, then the copied target data block is determined to be reliable.

2. The method according to claim 1, wherein, Selecting a first node group and a second node group from the plurality of data nodes includes: after selecting the first node group or the second node group from the plurality of data node groups, prioritizing the selection of data nodes for the other node group from the remaining nodes in the plurality of data nodes; if there are insufficient remaining nodes, supplementing the other node group with data nodes from the already selected node groups.

3. The method according to claim 1 or 2, wherein, Also includes: Determine the copying method for the target data block and the target checksum, respectively; Copying the required target data block includes: copying the target data block according to a first copying method determined for the target data block; copying the target checksum corresponding to the target data block includes: copying the target checksum according to a second copying method determined for the target checksum.

4. The method according to claim 3, wherein, Selecting a first node group and a second node group that are different from the plurality of data nodes includes: selecting the first node group from the plurality of data nodes according to the data requirements corresponding to the storage method of the target data and the first replication method; and selecting the second node group from the plurality of data nodes according to the data requirements corresponding to the storage method of the target data and the second replication method.

5. The method according to claim 4, wherein, According to the storage method of the target data and the data requirements corresponding to the first replication method, the first node group is selected from the plurality of data nodes, including: if the storage method adopts erasure coding mechanism and the first replication method is copy, then the data node where the target data block is located is selected from the plurality of data nodes to form the first node group; if the storage method adopts erasure coding mechanism and the first replication method is reconstruction, then k data nodes are selected from the remaining nodes other than the data node where the target data block is located from the plurality of data nodes to form the first node group, where k is the number of data blocks required for reconstruction; if the storage method adopts multiple replica mechanism and the first replication method is copy, then the data node where any replica is located is selected from the plurality of data nodes to form the first node group.

6. The method according to any one of claims 3-5, wherein, Determining the copying method for the target data block and the target checksum includes: when the target data supports multiple copying methods, selecting different copying methods for the target data block and the target checksum.

7. The method according to any one of claims 3-6, wherein, The target is copied according to the first copying method. The data block includes: if the target data is stored using erasure coding and the first copying method is copy, then a copy operation is performed on the target data block on the data node where the target data block is located to copy the target data block; if the target data is stored using erasure coding and the first copying method is reconstruction, then the data block where the target data is stored is read from the first node group; based on the read data block, the target data block is reconstructed.

8. The method according to any one of claims 3-7, wherein, The second copying method for obtaining the target checksum includes: if the target data is stored using erasure coding and the second copying method is reconstruction, then the checksum stored in the target data is read from the second node group; based on the read checksum, the target checksum is reconstructed; if the target data is stored using erasure coding and the second copying method is copy, then a copy operation is performed on the target checksum from the data node where the target checksum is located to obtain the target checksum.

9. The method according to claim 8, wherein, Based on the read checksum, reconstructing the target checksum includes: if there is an erasure coding relationship between the checksums stored in the target data, then encoding the target checksum based on the erasure coding relationship between the read checksum and the target checksum.

10. The method according to any one of claims 6-9, wherein, It also includes: when the target data supports a single copying method, selecting the copying method supported by the target data for the target data block and the target checksum.

11. The method according to any one of claims 6-10, wherein, The target data supports copying and / or duplication.

12. A data verification method, wherein, The target data is stored on multiple data nodes, each storing data blocks generated for the target data and corresponding checksums. The method includes: when the target data supports multiple replication methods, in response to a data block replication instruction initiated for the target data, selecting different replication methods for the target data blocks to be replicated and the target checksums corresponding to the target data blocks; replicating the required target data blocks according to the first replication method selected for the target data blocks based on the data blocks stored for the target data; and replicating the target checksums according to the second replication method selected for the target checksums based on the checksums stored for the target data; if the replicated target checksums match the replicated target data blocks, then the replicated target data blocks are determined to be reliable.

13. A computing device, wherein, The device includes a memory, a processor, and a communication component; the memory is used to store one or more computer instructions; the processor is coupled to the memory and the communication component and is used to execute the one or more computer instructions to perform the data verification method according to any one of claims 1-12.

14. A computer-readable storage medium for storing a computer program, wherein, When the computer program is executed by one or more processors, it causes the one or more processors to perform the data verification method according to any one of claims 1-12.

15. A computer program product, wherein, The computer program, when executed by one or more processors, causes the one or more processors to perform the data verification method according to any one of claims 1-12.

Citation Information

Patent Citations

  • Cloud migration of file system data hierarchies

    CN110268379A

  • Data recovery verification method, device and equipment

    CN110795269A

  • Data reconstruction method and device

    CN114237967A

  • Data reconstruction method and device

    CN114579352A

  • Direct memory access data path for RAID storage device

    CN116185275A