Data backup verification method and device, electronic equipment and storage medium

Abstract

The invention provides a data backup verification method and device, electronic equipment and a storage medium. The method comprises the steps that metadata information of a to-be-backed-up table is obtained, a backup file list and a global verification rule of the to-be-backed-up table are generated according to the metadata information, the to-be-backed-up table is partitioned according to the backup file list, an initial data set is obtained, the initial data set comprises at least one data block set, and the initial data set is backed up. The initial backup file group comprises at least one initial backup file, the initial backup files are in one-to-one correspondence with the data block groups, and the initial backup file group is subjected to data backup verification according to a global verification rule of the to-be-backed-up table. Therefore, the data in the to-be-backed-up table is partitioned, and partitioned verification is performed on the data, so that the problems of excessive resource consumption, long verification time, high maintenance cost and the like caused by huge data volume in the data backup verification process of the distributed database can be solved.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to a data backup verification method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of information technology, data reliability and security have become increasingly important. In this context, database backup and recovery mechanisms have become crucial. With the advent of the big data era, distributed databases have gradually become the mainstream way of data storage and processing due to their advantages such as high scalability, high availability and high performance.

[0003] Since distributed databases store data in different physical nodes, and there may be complex interactions and dependencies between different physical nodes, data backup becomes extremely complex, which may lead to incomplete backup data. Therefore, a data backup verification method is needed.

[0004] In the prior art, the verification method of data backup usually relies on the recovery process of part or all of the data to verify the integrity and accuracy of the data backup. However, since the amount of data in a distributed database is usually very large and the distributed database usually spans multiple nodes and computer rooms, the recovery operation often involves complex network transmission and coordination, which further aggravates resource consumption and prolongs the recovery time. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a data backup verification method, device, electronic device and storage medium to solve the problems of excessive resource consumption, long verification time and high maintenance cost caused by the huge amount of data in the data backup verification process of the distributed database. The specific technical solution is as follows:

[0006] In a first aspect of an embodiment of the present application, a data backup verification method is first provided, the method comprising:

[0007] Obtain metadata information of the table to be backed up, and generate a backup file list and a global verification rule for the table to be backed up based on the metadata information;

[0008] Partitioning the table to be backed up according to the backup file list to obtain an initial data group, wherein the initial data group includes at least one data block group;

[0009] Backing up the initial data group to obtain an initial backup file group, wherein the initial backup file group includes at least one initial backup file, and the initial backup file corresponds to the data block group one by one;

[0010] According to the global verification rule of the table to be backed up, the data backup of the initial backup file group is verified.

[0011] In an optional implementation, partitioning the table to be backed up according to the backup file list to obtain the initial data group includes:

[0012] Obtaining the shard information of the table to be backed up from the backup file list;

[0013] According to the shard information of the table to be backed up, obtaining the shard data nodes of the table to be backed up;

[0014] The data blocks of the table to be backed up are partitioned according to the shard data nodes of the table to be backed up to obtain an initial data group.

[0015] In an optional implementation, the verifying of data backup of the initial backup file group according to the global verification rule of the table to be backed up includes:

[0016] According to the global verification rule of the table to be backed up, obtaining a verification file group from the initial backup file group;

[0017] Performing data detection on the verification file group;

[0018] If the initial backup file does not contain data in the verification file group, it is determined that the backup of the data block group corresponding to the initial backup file fails;

[0019] If all the initial backup files in the verification file group have data, verifying the first data backup of the verification file group to obtain a first verification result;

[0020] According to the first verification result, the second data backup is verified on the verification file group.

[0021] In an optional implementation, the verifying the first data backup on the verification file group to obtain a first verification result includes:

[0022] Generate first verification information for each of the initial backup files in the verification file group according to a first algorithm;

[0023] Generate second verification information of the data block group corresponding to each of the initial backup files in the verification file group according to a second algorithm;

[0024] The first verification result is obtained according to the first verification information and the second verification information.

[0025] In an optional implementation, obtaining the first verification result according to the first verification information and the second verification information includes:

[0026] If the first verification information of all the initial backup files in the verification file group matches the second verification information of the data block group corresponding to the initial backup file, determining that the first verification result is that the initial backup is successful;

[0027] If the first verification information of the initial backup file in the verification file group does not match the second verification information of the data block group corresponding to the initial backup file, it is determined that the first verification result is that the initial backup is unsuccessful.

[0028] In an optional implementation, the verifying the second data backup on the verification file group according to the first verification result includes:

[0029] If the first verification result is that the initial backup is successful, generating global verification information according to the order of the data blocks in the table to be backed up and the first verification information corresponding to all the initial backup files in the verification file group;

[0030] Generate initial verification information according to the order of the data blocks in the table to be backed up and the second verification information of the data block group corresponding to all the initial backup files in the verification file group;

[0031] If the global verification information is consistent with the initial verification information, it is determined that the backup of the table data to be backed up is successful;

[0032] If the global verification information is inconsistent with the initial verification information, it is determined that the backup of the data in the table to be backed up has failed.

[0033] In an optional implementation, if the global verification information is consistent with the initial verification information, after determining that the backup of the to-be-backed-up table data is successful, the method further includes:

[0034] The initial verification information is stored in a metadata database.

[0035] In a second aspect of the embodiment of the present application, a data backup verification device is also provided, the device comprising:

[0036] An acquisition module acquires metadata information of the table to be backed up, and generates a backup file list and a global verification rule of the table to be backed up according to the metadata information;

[0037] A partitioning module, partitioning the table to be backed up according to the backup file list to obtain an initial data group, wherein the initial data group includes at least one data block group;

[0038] A backup module, backing up the initial data group to obtain an initial backup file group, wherein the initial backup file group includes at least one initial backup file, and the initial backup file corresponds to the data block group one by one;

[0039] The verification module verifies the data backup of the initial backup file group according to the global verification rule of the table to be backed up.

[0040] In a third aspect of the embodiments of the present application, there is further provided an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0041] Memory, used to store computer programs;

[0042] The processor is used to implement the steps of the data backup verification method described in any one of the first aspects above when executing the program stored in the memory.

[0043] In a fourth aspect of the embodiments of the present application, a storage medium is further provided, wherein instructions are stored in the storage medium, and when the storage medium is run on a computer, the computer executes the steps of the data backup verification method described in any one of the first aspects above.

[0044] In a fifth aspect of the embodiments of the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute the steps of the data backup verification method described in any one of the first aspects above.

[0045] The above technical solution provided by the embodiment of the present application obtains metadata information of the table to be backed up, and generates a backup file list and a global verification rule of the table to be backed up based on the metadata information, partitions the table to be backed up based on the backup file list to obtain an initial data group, which includes at least one data block group, backs up the initial data group to obtain an initial backup file group, which includes at least one initial backup file, and the initial backup file corresponds to the data block group one by one, and verifies the data backup of the initial backup file group according to the global verification rule of the table to be backed up. In this way, the data in the table to be backed up is partitioned and the data is verified by partitions, which can solve the problems of excessive resource consumption, long verification time and high maintenance cost caused by the huge amount of data in the distributed database during the data backup verification process. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0048] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0049] Figure 1 A schematic diagram of an implementation flow of a data backup verification method provided in an embodiment of the present application;

[0050] Figure 2 A schematic diagram of an implementation flow of another data backup verification method provided in an embodiment of the present application;

[0051] Figure 3 A schematic diagram of an implementation flow of a first data backup verification method provided in an embodiment of the present application;

[0052] Figure 4 A schematic diagram of an implementation process for obtaining a first verification result provided in an embodiment of the present application;

[0053] Figure 5 A schematic diagram of an implementation flow of a second data backup verification method provided in an embodiment of the present application;

[0054] Figure 6 A schematic diagram of the structure of a data backup verification device provided in an embodiment of the present application;

[0055] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0057] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0058] To facilitate understanding of the embodiments of the present application, further explanation will be given below with reference to specific embodiments in conjunction with the accompanying drawings. The embodiments do not constitute a limitation on the embodiments of the present application.

[0059] like Figure 1 FIG. 1 is a schematic diagram of an implementation flow of a data backup verification method provided in an embodiment of the present application. The method may specifically include the following steps:

[0060] S101, obtaining metadata information of a table to be backed up, and generating a backup file list and a global verification rule of the table to be backed up according to the metadata information.

[0061] In some embodiments of the present application, metadata information of the table to be backed up is obtained, wherein the metadata information may include information such as the structure of the table, sharding information, data node distribution, size and number of data blocks, etc. According to the sharding information and data node distribution in the metadata information of the table to be backed up, the data blocks contained in each shard are determined, and then a backup file identifier is generated for each data block and stored in the backup file list. At the same time, according to the sharding rules, data types and sizes in the metadata information table, a global verification rule is generated to clarify the verification method of each data block or shard, which is not limited in the present application example.

[0062] For example, by obtaining the metadata information of Table 1, it can be found that there are 2 shards, and the data blocks of Table 1 have a total of 8500 blocks (Shard 1 has 4500 data blocks, and Shard 2 has 4000 data blocks); then the global verification rule of Table 1 is: Shard 1 is responsible for verifying data blocks: 1-1000, 2001-3000, 4001-4500 Shard 2 is responsible for verifying data blocks: 1001-2000, 3001-4000.

[0063] S102, partitioning the table to be backed up according to the backup file list to obtain an initial data group, wherein the initial data group includes at least one data block group.

[0064] In some embodiments of the present application, the table to be backed up is partitioned according to the backup file list, and an initial data group is finally obtained. Partitioning can be to divide the data in the table to be backed up into a number of data blocks according to certain rules, and group the data blocks to obtain multiple data block groups to form an initial data group, and each data block group can contain a certain number of data blocks as the basic unit for subsequent backup and verification. The partitioning rules can be determined according to the table's sharding information, data block size, and backup strategy, or the partitioning rules can be set manually, and the present application example does not limit this.

[0065] For example, according to the backup file list, in Table 1, shard 1 has 4500 data blocks, and shard 2 has 3000 data blocks. After partitioning Table 1, the initial data groups obtained are data block group 1 (1-1000), data block group 2 (1001-2000), data block group 3 (2001-3000), data block group 4 (3001-4000), and data block group 5 (4001-4500) corresponding to shard 1. Data block group 6 (1-1000), data block group 7 (1001-2000), and data block group 8 (2001-3000) corresponding to shard 2.

[0066] S103, backing up the initial data group to obtain an initial backup file group, where the initial backup file group includes at least one initial backup file, and the initial backup file corresponds to the data block group one by one.

[0067] In some embodiments of the present application, different sharding nodes store different data blocks of the table to be backed up, and the initial data group is backed up, that is, the data of the data blocks contained in each data block group is backed up, that is, the data in each data block group is written into the corresponding initial backup file, ensuring a one-to-one correspondence between the initial backup file and the data block group. Thus, an initial backup file group containing all initial backup files is obtained. Among them, during the entire backup process, no matter which sharding node the data block is located on, it will be included in the initial data group and then backed up to the initial backup file group. These backup files will be used in the subsequent verification process to ensure the integrity and accuracy of the backup data.

[0068] For example, Table 1 corresponds to two shards. Shard 1 stores 5000 data blocks, which are divided into data block group 1 (1-1000), data block group 2 (1001-2000), data block group 3 (2001-3000), data block group 4 (3001-4000), and data block group 5 (4001-5000); Shard 2 stores 1000 data blocks, which are divided into data block group 6 (1-1000). By backing up the initial data group, 6 initial backup files corresponding to the 6 data block groups can be obtained.

[0069] S104: verify the data backup of the initial backup file group according to the global verification rule of the table to be backed up.

[0070] In some embodiments of the present application, the initial backup file group is verified according to the global verification rules of the table to be backed up. Among them, the verification process may include reading the data blocks or shard data in each initial backup file, generating corresponding verification information, and comparing it with the original verification information stored in the source database. It may also include reading the database or shard data information in the corresponding initial backup file according to the global verification rules, generating corresponding verification information, if the comparison results are consistent, it means that the backup data is complete and accurate; if they are inconsistent, it means that there may be problems with the backup data, and further inspection and repair are required, which is not limited in the examples of the present application.

[0071] For example, read the data block in the initial backup file 1 (corresponding to data block group 1), generate a hash value, and compare it with the original hash value of the corresponding data block stored in the metadata database. If the two are consistent, continue to verify the next backup file; if they are inconsistent, it is necessary to locate the problem and take corresponding repair measures.

[0072] Through the above description of the technical solution provided by the embodiment of the present application, the metadata information of the table to be backed up is obtained, and according to the metadata information, a backup file list and a global verification rule of the table to be backed up are generated. According to the backup file list, the table to be backed up is partitioned to obtain an initial data group, which includes at least one data block group. The initial data group is backed up to obtain an initial backup file group, which includes at least one initial backup file, and the initial backup file corresponds to the data block group one by one. According to the global verification rule of the table to be backed up, the initial backup file group is verified for data backup. In this way, the data in the table to be backed up is partitioned and the data is verified by partitions, which can solve the problems of excessive resource consumption, long verification time and high maintenance cost caused by the huge amount of data in the distributed database during the data backup verification process.

[0073] like Figure 2 FIG. 1 is a schematic diagram of an implementation flow of another data backup verification method provided in an embodiment of the present application, which may specifically include the following contents:

[0074] S201, obtaining metadata information of a table to be backed up, and generating a backup file list and a global verification rule of the table to be backed up according to the metadata information.

[0075] In the embodiment of the present application, this step is similar to the above-mentioned step S101, and the embodiment of the present application will not be described one by one here.

[0076] S202: Obtain shard information of the table to be backed up from the backup file list.

[0077] In some embodiments of the present application, the backup file list is parsed to extract the shard information of the table to be backed up, and the shard information of the table to be backed up includes the number of shards, shard identifiers, and data block information corresponding to each shard.

[0078] S203: Obtain the shard data nodes of the table to be backed up according to the shard information of the table to be backed up.

[0079] In some embodiments of the present application, based on the shard information of the table to be backed up and the mapping relationship between the shards and the data nodes in the metadata database, the data nodes corresponding to each shard can be queried, so that the specific location of the stored shard data can be obtained based on the data node, and then the data to be backed up can be read from the corresponding data node.

[0080] For example, the shard information of the table to be backed up is known as shard 1: ID is 001, corresponding to data blocks 1-1000; shard 2: ID is 002, corresponding to data blocks 1001-2000. The mapping table between shards and data nodes is: shard ID 001 corresponds to data node A; shard ID 002 corresponds to data node B, so that the data of shard 1 (ID is 001) is stored on data node A, and the data of shard 2 (ID is 002) is stored on data node B, and then the corresponding data can be read.

[0081] S204, partitioning the data blocks of the table to be backed up according to the shard data nodes of the table to be backed up to obtain an initial data group, wherein the initial data group includes at least one data block group.

[0082] In some embodiments of the present application, according to the shard data node of the table to be backed up, data information is obtained from the corresponding data node, and then the data blocks of the table to be backed up are partitioned to obtain an initial data group. Each data block group in the initial data group contains a certain number of data blocks, and these data blocks are located on the same shard data node.

[0083] For example, assuming that shard 1 contains data blocks 1-500 and shard 2 contains data blocks 1-500, the data blocks corresponding to shard 1 and shard 2 are divided into multiple data block groups, such as shard 1 corresponds to data block group 1 (including data blocks 1-250) and data block group 2 (including data blocks 251-500), and shard 2 corresponds to data block group 3 (including data blocks 1-250) and data block group 4 (including data blocks 251-500).

[0084] S205, backing up the initial data group to obtain an initial backup file group, where the initial backup file group includes at least one initial backup file, and the initial backup files correspond to the data block groups one by one.

[0085] In the embodiment of the present application, this step is similar to the above-mentioned step S103, and the embodiment of the present application will not be described one by one here.

[0086] S206: Obtain a verification file group from the initial backup file group according to the global verification rule of the table to be backed up.

[0087] In some embodiments of the present application, according to the global verification rules of the table to be backed up, a verification file group is obtained from the initial backup file group. The global verification rules are used to ensure the integrity and accuracy of the data backup and verify whether the data in the backup file is consistent with the original data. The global verification rules may include verification of data format, data type, data range, etc. The global verification rules may be generated based on the number of data blocks to ensure that the data in the backup file meets the expected format and range requirements, or they may be set manually according to business needs. The verification file group is the initial backup file corresponding to the data block group that needs to be verified for different shards selected from the initial backup file group according to the global verification rules.

[0088] For example, the global verification rule of the table to be backed up is set as follows: if the number of data blocks is greater than 1000, the data is checked in a round-robin manner with a granularity of 1000 to generate verification information, that is, the data blocks are grouped into groups of 1000 blocks, and the data in each group is checked. When the global verification rule of Table 1 is: Shard 1 is responsible for checking data blocks: 1-1000, 2001-3000, 4001-4500; Shard 2 is responsible for checking data blocks: 1001-2000, 3001-4000, the initial backup files corresponding to the verification file group are: the initial backup files corresponding to data blocks 1-1000, 2001-3000, 4001-4500 in Shard 1, and the initial backup files corresponding to data blocks 1001-2000, 3001-4000 in Shard 2.

[0089] S207, performing data detection on the verification file group.

[0090] In some embodiments of the present application, data detection is performed on each initial backup file in the verification file group. Each initial backup file in the verification file group can be read to detect whether data exists therein.

[0091] For example, read the initial backup file 1, read the data of the initial backup file 1, and check whether there is data in the initial backup file 1.

[0092] S208: If the initial backup file does not contain data in the verification file group, it is determined that the backup of the data block group corresponding to the initial backup file has failed.

[0093] In some embodiments of the present application, data detection is performed on each initial backup file in the verification file group. When data does not exist in the initial backup file, it means that the data block group corresponding to this backup file has not been backed up in the backup file. Therefore, the backup of the data block group corresponding to the initial backup file without data fails, which means that an error or abnormality occurred in the backup process.

[0094] For example, when the initial backup file 1 has data in the verification file group, but the initial backup file 2 does not have data, the backup of the data block group corresponding to the initial backup file 2 fails.

[0095] S209: If all the initial backup files in the verification file group have data, verify the first data backup on the verification file group to obtain a first verification result.

[0096] In some embodiments of the present application, when data exists in all initial backup files, it is necessary to verify the first data backup of the verification file group to obtain a first verification result.

[0097] For example, when the initial backup file 1 and the initial backup file 2 have data in the verification file group, it is necessary to verify the first data backup of the initial backup file 1 and the initial backup file 2 to obtain a first verification result.

[0098] Specifically, the first data backup is verified on the verification file group to obtain a first verification result, such as Figure 3 As shown, it is a schematic diagram of an implementation flow of a first data backup verification method provided in an embodiment of the present application, which may include the following steps:

[0099] S301, generating first verification information for each initial backup file in a verification file group according to a first algorithm.

[0100] In some embodiments of the present application, first verification information for each initial backup file in the verification file group is generated according to a first algorithm. The first algorithm may be any algorithm suitable for data block verification, such as MD5, SHA-1, SHA-256 or other more advanced hash algorithms. When the first algorithm is the SHA-256 cryptographic hash algorithm, it can map data of any length to a hash value of a fixed length (e.g., 256 bits, i.e., 32 bytes), and the binary data of the initial backup file is processed by the first algorithm to calculate and generate a hash value, which is used as the first verification information of the initial backup file.

[0101] For example, the text content in the initial backup file 1 is hashed by the first algorithm to obtain a hash value 1 with a fixed length of 32 bytes (i.e., the first verification information of the initial backup file 1); the text content in the initial backup file 2 is hashed by the first algorithm to obtain a hash value 2 with a fixed length of 32 bytes (i.e., the first verification information of the initial backup file 2).

[0102] S302: Generate second verification information of the data block group corresponding to each initial backup file in the verification file group according to a second algorithm.

[0103] In some embodiments of the present application, second verification information of the data block group corresponding to each initial backup file in the verification file group is generated according to the second algorithm. The second algorithm can be any algorithm suitable for data block verification, such as MD5, SHA-1, SHA-256 or other more advanced hash algorithms. When the second algorithm is the SHA-256 cryptographic hash algorithm, it can map data of any length to a hash value of a fixed length (e.g., 256 bits, i.e., 32 bytes), and the binary data of the data block group corresponding to the initial backup file is processed by the second algorithm to calculate and generate a hash value, which is used as the second verification information of the data block group corresponding to the initial backup file.

[0104] For example, a hash calculation is performed on the data in data block group 1 corresponding to the initial backup file 1 through the second algorithm to obtain a hash value 1 with a fixed length of 32 bytes (i.e., the second verification information of data block group 1 corresponding to the initial backup file 1); a hash calculation is performed on the text content in data block group 2 corresponding to the initial backup file 2 through the second algorithm to obtain a hash value 2 with a fixed length of 32 bytes (i.e., the second verification information of data block group 2 corresponding to the initial backup file 2).

[0105] S303: Obtain a first verification result according to the first verification information and the second verification information.

[0106] In some embodiments of the present application, a first verification result can be obtained based on the first verification information and the second verification information, that is, the first verification information corresponding to each initial backup file in the verification file group is compared with the second verification information of the corresponding data block to obtain the first verification result.

[0107] For example, if the first verification information is inconsistent with the second verification information, a first verification result 1 can be obtained, and if the first verification information is consistent with the second verification information, a first verification result 2 can be obtained.

[0108] Specifically, according to the first verification information and the second verification information, a first verification result is obtained, such as Figure 4 As shown, a schematic diagram of an implementation process of obtaining a first verification result provided in an embodiment of the present application may include the following steps:

[0109] S401, if the first verification information of all the initial backup files in the verification file group matches the second verification information of the data block group corresponding to the initial backup file, it is determined that the first verification result is that the initial backup is successful.

[0110] In some embodiments of the present application, when the first verification information of all initial backup files is consistent with the second verification information of the data block group corresponding to the initial backup file, it can be determined that the first verification result is that the initial backup is successful.

[0111] For example, a table to be backed up is divided into multiple data block groups, and each of them is backed up separately. After the backup is completed, according to the global verification rule, a verification file group is obtained from the initial backup file group (initial backup file 1, initial backup file 2, initial backup file 3, initial backup file 4, initial backup file 5, initial backup file 6) (shard 1 is responsible for verifying initial backup file 1 and initial backup file 5; shard 2 is responsible for verifying initial backup file 5), and the first verification information of each initial backup file in the verification file group is compared with the second verification information of the corresponding data block group. If the first verification information of all initial backup files is completely consistent with the second verification information of the corresponding data block group, then it can be determined that the backup of this table is successful.

[0112] S402: If the first verification information of the initial backup file in the verification file group does not match the second verification information of the data block group corresponding to the initial backup file, determine that the first verification result is that the initial backup is unsuccessful.

[0113] In some embodiments of the present application, when the first verification information of the initial backup file is inconsistent with the second verification information of the data block group corresponding to the initial backup file, it indicates that an abnormality occurred during the backup process, and the first verification result can be determined as initial backup failure.

[0114] For example, a table to be backed up is divided into multiple data block groups, and each of them is backed up separately. After the backup is completed, according to the global verification rule, a verification file group is obtained from the initial backup file group (initial backup file 1, initial backup file 2, initial backup file 3, initial backup file 4, initial backup file 5, initial backup file 6) (shard 1 is responsible for verifying initial backup file 1 and initial backup file 5; shard 2 is responsible for verifying initial backup file 5), and the first verification information of each initial backup file in the verification file group is compared with the second verification information of the corresponding data block group. If the first verification information of the initial backup file is inconsistent with the second verification information of the corresponding data block group 1, then the backup of this table fails.

[0115] S210: Verify the second data backup on the verification file group according to the first verification result.

[0116] In some embodiments of the present application, the second data backup is verified on the verification file group according to the first verification result.

[0117] For example, if the first verification result is result 1, the second data backup is verified on the verification file group according to result 1.

[0118] Specifically, according to the first verification result, the second data backup is verified on the verification file group, such as Figure 5 As shown, it is a schematic diagram of an implementation flow of a second data backup verification method provided in an embodiment of the present application, which may include the following steps:

[0119] S501, if the first verification result is that the initial backup is successful, generate global verification information according to the order of the data blocks in the table to be backed up and the first verification information corresponding to all the initial backup files in the verification file group.

[0120] In some embodiments of the present application, if the first verification result is that the initial backup is successful, global verification information is generated according to the order of the data blocks in the table to be backed up and the first verification information corresponding to all the initial backup files in the verification file group, wherein the order of the data blocks in the table to be backed up refers to the order in which the data corresponding to the data blocks are recorded or organized in the table to be backed up. This order can be the order in which the data is written to the table to be backed up, or it can be the order after the data is logically grouped (such as data block group 1, data block group 3, data block group 2, etc.). According to the order of the data blocks in the table to be backed up, each initial backup file in the verification file group is sorted or associated to ensure that each initial backup file corresponds to a specific data block group one by one. Then, the first verification information corresponding to each initial backup file is combined to generate global verification information, that is, the first verification information corresponding to each initial backup file is spliced ​​together according to the order after the initial backup files are sorted or associated.

[0121] For example, the order of data blocks in the table to be backed up is data block group 1, data block group 3, data block group 2, data block group 5, data block group 4, and the verification file group contains the initial backup files corresponding to data block group 1, data block group 3, and data block group 4. Then, according to this order, the initial backup files in the verification file group are sorted, and the arrangement order of the initial backup files is initial backup file 1 (corresponding to data block group 1), initial backup file 3 (corresponding to data block group 3), and initial backup file 4 (corresponding to data block group 4). Therefore, the combination order of the first verification information is first verification information 1, first verification information 3, and first verification information 4, and the global verification information is spliced ​​by these three parts of the first verification information.

[0122] S502 , generating initial verification information according to the order of the data blocks in the table to be backed up and second verification information of the data block groups corresponding to all the initial backup files in the verification file group.

[0123] In some embodiments of the present application, initial verification information is generated based on the order of data blocks in the table to be backed up and the second verification information of the data block groups corresponding to all initial backup files in the verification file group. Among them, the order of data blocks in the table to be backed up refers to the order in which the data corresponding to the data blocks are recorded or organized in the table to be backed up. This order can be the order in which the data is written into the table to be backed up, or it can be the order after the data is logically grouped (such as data block group 1, data block group 3, data block group 2, etc.). According to the order of data blocks in the table to be backed up, the data block groups corresponding to each initial backup file in the verification file group are sorted or associated to ensure that the data block groups correspond to the initial backup files one by one. Then, the second verification information of the data block groups corresponding to each initial backup file is combined to generate the initial verification information, that is, the second verification information of the data block groups corresponding to each initial backup file is spliced ​​according to the order after the data block groups corresponding to the initial backup files are sorted or associated.

[0124] For example, the order of data blocks in the table to be backed up corresponds to data block group 1, data block group 3, data block group 2, data block group 5, and data block group 4. The verification file group includes the initial backup files corresponding to data block group 1, data block group 3, and data block group 4. Then, according to this order, the data block groups corresponding to the initial backup files in the verification file group are sorted, and the arrangement order of the data block groups is data block group 1 (corresponding to initial backup file 1), data block group 3 (corresponding to initial backup file 3), and data block group 4 (corresponding to initial backup file 4). Therefore, the combination order of the second verification information is second verification information 1, second verification information 3, and second verification information 4. The initial verification information is spliced ​​from these three parts of the second verification information.

[0125] S503: If the global verification information is consistent with the initial verification information, it is determined that the backup of the data in the table to be backed up is successful.

[0126] In some embodiments of the present application, the initial verification information of the data block group corresponding to all the initial backup files in the verification file group is matched with the global verification information. If the initial verification information of the data block group corresponding to all the initial backup files in the verification file group is consistent with the global verification information, the data of the table to be backed up has not been tampered with or damaged during the backup process, so it can be determined that the data backup of the table to be backed up is successful. When it is determined that the data backup in the table to be backed up is successful, the initial verification information can also be stored in the metadata database so that the data backup can be directly verified using the initial verification information stored in the metadata database next time.

[0127] For example, the data of the data block group corresponding to all the initial backup files in the verification file group generates initial verification information, and the global verification information is generated during the backup process. If the two are completely consistent, the system will record the status of "backup successful" in the backup log and save the initial verification information to the metadata database for verification next time.

[0128] S504: If the global verification information is inconsistent with the initial verification information, it is determined that the backup of the data in the table to be backed up has failed.

[0129] In some embodiments of the present application, the initial verification information of the data block group corresponding to all the initial backup files in the obtained verification file group is matched with the global verification information. If the global verification information is inconsistent with the initial verification information, the data of the table to be backed up may be erroneous or damaged during the backup process, so it can be determined that the backup of the table to be backed up has failed. At the same time, the record of the backup failure of the table to be backed up can also be notified to the corresponding administrator or the backup operation can be re-initiated.

[0130] For example, if a mismatch is found when verifying the global checksum information after the backup and the initial checksum information, the "backup failed" status can be recorded in the backup log and the administrator can be notified for troubleshooting. At the same time, other related backup tasks can be suspended to prevent the error from spreading. After receiving the notification, the administrator can check the log files, hardware status, etc. during the backup process to determine the cause of the failure and take appropriate measures to repair it.

[0131] Corresponding to the above method embodiment, the present application embodiment also provides a data backup verification device, such as Figure 6 As shown, the device may include an acquisition module 601 , a partition module 602 , a backup module 603 , and a verification module 604 .

[0132] The acquisition module 601 acquires metadata information of the table to be backed up, and generates a backup file list and a global verification rule of the table to be backed up according to the metadata information;

[0133] Partitioning module 602, partitioning the table to be backed up according to the backup file list to obtain an initial data group, wherein the initial data group includes at least one data block group;

[0134] The backup module 603 backs up the initial data group to obtain an initial backup file group, where the initial backup file group includes at least one initial backup file, and the initial backup file corresponds to the data block group one by one;

[0135] The verification module 604 verifies the data backup of the initial backup file group according to the global verification rule of the table to be backed up.

[0136] The present application also provides an electronic device, such as Figure 7As shown, it includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0137] Memory 703, used for storing computer programs;

[0138] In one embodiment of the present application, the processor 701, when used to execute the program stored in the memory 703, implements the following steps:

[0139] Obtain metadata information of the table to be backed up, and generate a backup file list and a global verification rule of the table to be backed up based on the metadata information; partition the table to be backed up based on the backup file list to obtain an initial data group, which includes at least one data block group; back up the initial data group to obtain an initial backup file group, which includes at least one initial backup file, and the initial backup file corresponds to the data block group one by one; verify the data backup of the initial backup file group according to the global verification rule of the table to be backed up.

[0140] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0141] The communication interface is used for communication between the above electronic device and other devices.

[0142] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0143] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0144] In another embodiment provided in the present application, a storage medium is provided, in which instructions are stored. When the storage medium is executed on a computer, the computer executes the data backup verification method described in any one of the above embodiments.

[0145] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the data backup verification method described in any one of the above embodiments.

[0146] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a storage medium, or transmitted from one storage medium to another storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0147] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0148] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0149] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A data backup verification method, characterized in that: The method comprises: Obtain metadata information of the table to be backed up, and generate a backup file list and a global verification rule for the table to be backed up based on the metadata information; Partitioning the table to be backed up according to the backup file list to obtain an initial data group, wherein the initial data group includes at least one data block group; Backing up the initial data group to obtain an initial backup file group, wherein the initial backup file group includes at least one initial backup file, and the initial backup file corresponds to the data block group one by one; According to the global verification rule of the table to be backed up, the data backup of the initial backup file group is verified.

2. The method according to claim 1, characterized in that Partitioning the table to be backed up according to the backup file list to obtain an initial data group includes: Obtaining the shard information of the table to be backed up from the backup file list; According to the shard information of the table to be backed up, obtaining the shard data nodes of the table to be backed up; The data blocks of the table to be backed up are partitioned according to the shard data nodes of the table to be backed up to obtain an initial data group.

3. The method according to claim 1, characterized in that The verifying of data backup of the initial backup file group according to the global verification rule of the table to be backed up includes: According to the global verification rule of the table to be backed up, obtaining a verification file group from the initial backup file group; Performing data detection on the verification file group; If the initial backup file does not contain data in the verification file group, it is determined that the backup of the data block group corresponding to the initial backup file fails; If all the initial backup files in the verification file group have data, verifying the first data backup of the verification file group to obtain a first verification result; According to the first verification result, the second data backup is verified on the verification file group.

4. The method according to claim 3, characterized in that The verifying the first data backup on the verification file group to obtain a first verification result includes: Generate first verification information for each of the initial backup files in the verification file group according to a first algorithm; Generate second verification information of the data block group corresponding to each of the initial backup files in the verification file group according to a second algorithm; The first verification result is obtained according to the first verification information and the second verification information.

5. The method according to claim 4, characterized in that The obtaining the first verification result according to the first verification information and the second verification information includes: If the first verification information of all the initial backup files in the verification file group matches the second verification information of the data block group corresponding to the initial backup file, determining that the first verification result is that the initial backup is successful; If the first verification information of the initial backup file in the verification file group does not match the second verification information of the data block group corresponding to the initial backup file, it is determined that the first verification result is that the initial backup is unsuccessful.

6. The method according to claim 4, characterized in that The verifying the second data backup on the verification file group according to the first verification result includes: If the first verification result is that the initial backup is successful, generating global verification information according to the order of the data blocks in the table to be backed up and the first verification information corresponding to all the initial backup files in the verification file group; Generate initial verification information according to the order of the data blocks in the table to be backed up and the second verification information of the data block group corresponding to all the initial backup files in the verification file group; If the global verification information is consistent with the initial verification information, it is determined that the backup of the table data to be backed up is successful; If the global verification information is inconsistent with the initial verification information, it is determined that the backup of the data in the table to be backed up has failed.

7. The method according to claim 6, characterized in that If the global verification information is consistent with the initial verification information, after determining that the backup of the table data to be backed up is successful, the method further includes: The initial verification information is stored in a metadata database.

8. A data backup verification device, characterized in that: The device comprises: An acquisition module acquires metadata information of the table to be backed up, and generates a backup file list and a global verification rule of the table to be backed up according to the metadata information; A partitioning module, partitioning the table to be backed up according to the backup file list to obtain an initial data group, wherein the initial data group includes at least one data block group; A backup module, backing up the initial data group to obtain an initial backup file group, wherein the initial backup file group includes at least one initial backup file, and the initial backup file corresponds to the data block group one by one; The verification module verifies the data backup of the initial backup file group according to the global verification rule of the table to be backed up.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-7 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images