Data verification method, device, equipment and medium

Abstract

The embodiment of the invention discloses a data verification method, device, equipment and medium, and relates to the technical field of data processing. The method includes: extracting the attribute value of the source data and the attribute value of the backup data, wherein the attribute value of the source data is determined according to the source data, and the data amount of the attribute value of the source data is smaller than the data of the source data amount; compare the attribute value of the source data with the attribute value of the backup data; determine the difference between the source data and the backup data according to the comparison result. Embodiments of the present invention provide a data verification method, device, equipment, and medium, which realize data verification that takes both cost and accuracy into account.

Description

technical field

[0001] The embodiments of the present invention relate to the technical field of data processing, and in particular, to a data verification method, device, device, and medium. Background technique

[0002] In the process of migration or backup of massive data, it is usually necessary to verify the correctness of the data, that is, by reading and comparing two sets of data, it is judged whether there is a difference, and the difference rate is obtained, so as to determine the subsequent operation and processing methods.

[0003] There are usually two ways to verify massive data:

[0004] 1) Sampling

[0005] A certain proportion of data is randomly selected for comparison, and the results are calculated according to the sample proportion.

[0006] 2) Full comparison

[0007] Read all the data and compare them one by one, which can fully reflect the real situation of the data.

[0008] However, the currently used schemes suffer from the following disadvant...

Examples

Embodiment 1

[0038] figure 1 It is a flow chart of a data verification method provided by Embodiment 1 of the present invention. This embodiment is applicable to the case of performing both cost and accuracy verification on the backup data. The method can be executed by a data verification device, and the device can be realized by software and / or hardware. see figure 1 , the data verification method provided in this embodiment includes:

[0039] S110. Extract the attribute value of the source data and the attribute value of the backup data.

[0040] Wherein, the attribute value of the source data is determined according to the source data, and the data amount of the attribute value of the source data is smaller than the data amount of the source data. The attribute value of the backup data is determined according to the backup data, and the data amount of the attribute value of the backup data is smaller than the data amount of the backup data.

[0041] The source data is the source d...

Embodiment 2

[0070] figure 2 It is a flow chart of a data verification method provided by Embodiment 2 of the present invention. This embodiment is an optional solution proposed on the basis of the foregoing embodiments. see figure 2 , the data verification method provided in this embodiment includes:

[0071] S210. Generate an attribute value of the source data according to the source data, and generate an attribute value of the backup data according to the backup data.

[0072] S220. Store the generated attribute value in the attribute storage unit.

[0073] Specifically, the generated attribute value of the source data is stored in the attribute storage unit of the source data, and the generated attribute value of the backup data is stored in the attribute storage unit of the backup data.

[0074] The storage unit may be a correspondingly consistent storage unit with the source data.

[0075] Typically, see image 3 , the attribute storage unit includes attribute location index ...

Embodiment 3

[0121] This embodiment is an optional solution proposed on the basis of the foregoing embodiments. The data verification method provided in this embodiment includes:

[0122] attribute value generation

[0123] Wherein, attribute value generation includes determination of attribute value acquisition method, attribute value storage format and attribute value generation method.

[0124] The attribute acquisition method is as follows: For fields of fixed-length data types such as bool, int, and long, the original value is directly used as the attribute value because it occupies a small space. For variable-length data types such as string and Binary, the space occupied by the actual value may be large, and it is not suitable to use the original value as the attribute value, so the summary of the type of data is used as the attribute value. This attribute acquisition method can not only save space, but also ensure the accuracy of data.

[0125] Wherein, the digest extraction met...

technical field

[0001] The embodiments of the present invention relate to the technical field of data processing, and in particular, to a data verification method, device, device, and medium. Background technique

[0002] In the process of migration or backup of massive data, it is usually necessary to verify the correctness of the data, that is, by reading and comparing two sets of data, it is judged whether there is a difference, and the difference rate is obtained, so as to determine the subsequent operation and processing methods.

[0003] There are usually two ways to verify massive data:

[0004] 1) Sampling

[0005] A certain proportion of data is randomly selected for comparison, and the results are calculated according to the sample proportion.

[0006] 2) Full comparison

[0007] Read all the data and compare them one by one, which can fully reflect the real situation of the data.

[0008] However, the currently used schemes suffer from the following disadvant...

Examples

Embodiment 1

[0038] figure 1 It is a flow chart of a data verification method provided by Embodiment 1 of the present invention. This embodiment is applicable to the case of performing both cost and accuracy verification on the backup data. The method can be executed by a data verification device, and the device can be realized by software and / or hardware. see figure 1 , the data verification method provided in this embodiment includes:

[0039] S110. Extract the attribute value of the source data and the attribute value of the backup data.

[0040] Wherein, the attribute value of the source data is determined according to the source data, and the data amount of the attribute value of the source data is smaller than the data amount of the source data. The attribute value of the backup data is determined according to the backup data, and the data amount of the attribute value of the backup data is smaller than the data amount of the backup data.

[0041] The source data is the source d...

Embodiment 2

[0070] figure 2 It is a flow chart of a data verification method provided by Embodiment 2 of the present invention. This embodiment is an optional solution proposed on the basis of the foregoing embodiments. see figure 2 , the data verification method provided in this embodiment includes:

[0071] S210. Generate an attribute value of the source data according to the source data, and generate an attribute value of the backup data according to the backup data.

[0072] S220. Store the generated attribute value in the attribute storage unit.

[0073] Specifically, the generated attribute value of the source data is stored in the attribute storage unit of the source data, and the generated attribute value of the backup data is stored in the attribute storage unit of the backup data.

[0074] The storage unit may be a correspondingly consistent storage unit with the source data.

[0075] Typically, see image 3 , the attribute storage unit includes attribute location index ...

Embodiment 3

[0121] This embodiment is an optional solution proposed on the basis of the foregoing embodiments. The data verification method provided in this embodiment includes:

[0122] attribute value generation

[0123] Wherein, attribute value generation includes determination of attribute value acquisition method, attribute value storage format and attribute value generation method.

[0124] The attribute acquisition method is as follows: For fields of fixed-length data types such as bool, int, and long, the original value is directly used as the attribute value because it occupies a small space. For variable-length data types such as string and Binary, the space occupied by the actual value may be large, and it is not suitable to use the original value as the attribute value, so the summary of the type of data is used as the attribute value. This attribute acquisition method can not only save space, but also ensure the accuracy of data.

[0125] Wherein, the digest extraction met...

Images