Data reconciliation method and electronic device

Abstract

The application provides a data reconciliation method and an electronic device, which can be applied to the technical field of cloud platforms. The data reconciliation method comprises the following steps: in response to receiving a data reconciliation request from a cloud platform, obtaining backup metadata sets of a backup task according to a task identifier and a backup version identifier in the data reconciliation request, wherein the backup metadata sets comprise first metadata, second metadata and third metadata, the first metadata comprises task layer information of the backup task, the second metadata comprises a physical storage location of backup original data, and the third metadata comprises global storage information of the backup original data and an associated mapping relationship between the physical storage location and the global storage information; determining a backup process state of the backup task according to a task state in the task layer information; and in the case that the backup process state is complete, reconciling the original data and the backup original data based on the first metadata, the second metadata and the third metadata to obtain a reconciliation result.

Claims

1. A data reconciliation method, characterized in that, The method includes: In response to receiving a data reconciliation request from the cloud platform, a backup metadata dataset for the backup task is obtained based on the task identifier and backup version identifier in the data reconciliation request. The backup task is used to back up the original data of the business on the cloud platform via the backup metadata dataset to obtain the backup original data corresponding to the backup version identifier. The backup metadata dataset includes first metadata, second metadata, and third metadata. The first metadata includes task-level information of the backup task. The second metadata includes the physical storage location of the backup original data. The third metadata includes global storage information of the backup original data and the association mapping relationship between the physical storage location and the global storage information. The global storage information is used to characterize the logical index of the file / block of the backup original data. The backup process status of the backup task is determined based on the task status in the task layer information. If the backup process is complete, the original data and the backup original data are reconciled based on the first metadata, the second metadata, and the third metadata to obtain a reconciliation result, including: When the task identifier corresponds to the file identifier of the third metadata, the second metadata and the third metadata are matched according to the association mapping relationship, the global storage information and the number of entries of the third metadata to obtain a matching result. The third metadata also includes the data management information of the backup original data. If the matching result indicates that the second metadata matches the third metadata, the backup original data is retrieved from the physical storage location according to the data management information. The original data and the backup original data are compared for consistency to obtain the reconciliation results.

2. The method according to claim 1, characterized in that, The step of matching the second metadata with the third metadata based on the association mapping relationship in the third metadata, the global storage information, and the number of entries in the third metadata to obtain a matching result includes: Determine the target second metadata that has the associated mapping relationship with the third metadata from the second metadata; If the storage cluster, storage pool, backup disk, and storage node ownership domain corresponding to the global storage information are consistent with the storage cluster, storage pool, backup disk, and storage node ownership information corresponding to the physical storage location associated with the target second metadata, the number of entries of the third metadata is compared with the number of entries of the target second metadata. When the number of entries in the third metadata is the same as the number of entries in the target second metadata, the matching result is obtained, which indicates that the second metadata matches the third metadata.

3. The method according to claim 2, characterized in that, The data management information includes at least one of the following: data parsing method, data compression method, data decryption method, and data encapsulation method.

4. The method according to claim 2, characterized in that, The method further includes: A matching result indicating a mismatch between the second metadata and the third metadata is obtained if any one of the following conditions is met: The task identifier does not correspond to the file identifier of the third metadata. The target second metadata, which is associated with the third metadata, is not present in the second metadata. The global storage information does not match the storage ownership information of the physical storage location of the target second metadata; The number of entries for the third metadata is inconsistent with the number of entries for the target second metadata.

5. The method according to claim 2, characterized in that, The method further includes: If the task identifier does not correspond to the file identifier of the third metadata, the third metadata will be marked as erroneous metadata. If the number of entries in the third metadata is inconsistent with the number of entries in the target second metadata, and the original backup data is normal, the number of entries in the target second metadata and the associated mapping relationship shall be corrected according to the number of entries in the third metadata.

6. The method according to claim 2, characterized in that, The method further includes: In the event of inconsistencies between the target second metadata and the original backup data, an abnormal instance is identified from the original backup data based on the inconsistency information. After clearing the abnormal instance, a full backup is performed. The inconsistency information includes at least one of the following: mismatch in the number of physical addresses or inconsistent data content.

7. The method according to claim 1, characterized in that, The method further includes: If the backup process is in the running state, continue to execute the backup task, and perform reconciliation after the backup process is switched to the completed state. If the backup process is in a pending state, the execution time of the backup task is delayed.

8. The method according to claim 1, characterized in that, The method further includes: In the event of loss of the original data, the original data can be repaired using historical backups.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

Images