A data verification method, apparatus, device and medium

Abstract

This application relates to the field of data processing technology, specifically to a data verification method, apparatus, device, and medium. The method is applied in database migration operations and includes: grouping a first database table into at least one first group based on preset rules, and grouping a second database table into at least one second group based on preset rules; forming a first pair with each corresponding second group; summing the hash values ​​of each first record included in each first group to obtain the sum of hash values ​​for each first group, and summing the hash values ​​of each second record included in each second group to obtain the sum of hash values ​​for each second group; determining whether the sums of hash values ​​of the first group and the second group in each first pair are equal; if the sums of hash values ​​of the first group and the second group in any first pair are not equal, then determining that an inconsistent field exists between the first group and the second group in any first pair.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a data verification method, apparatus, device, and medium. Background Technology

[0002] Database migration refers to the transfer of data from a source database to a target database. To assess the quality of the migration, it's necessary to verify that the data before and after the migration is identical. If they differ, it's also necessary to identify the inconsistent fields to help technical personnel pinpoint problems encountered during the migration process. Current technology allows for the location of inconsistent fields through record-by-record, field-by-field comparison, but this method is inefficient. Therefore, improving data verification efficiency has become a pressing issue. Summary of the Invention

[0003] This application provides a data verification method, apparatus, device, and medium to improve data verification efficiency.

[0004] In a first aspect, embodiments of this application provide a data verification method, which is applied in database migration operations, and the method includes:

[0005] Obtain the first database table and the second database table; wherein, the second database table is the database table obtained by migrating the first database table from the source database to the target database;

[0006] The first database table is grouped according to a preset rule to obtain at least one first group, and the second database table is grouped according to the preset rule to obtain at least one second group; the first group and the corresponding second group form a first pair; each first group in the at least one first group includes at least one first record, and each second group in the at least one second group includes at least one second record;

[0007] The sum of hash values ​​of each first record included in each first group is obtained by summing the hash values ​​of each first group, and the sum of hash values ​​of each second record included in each second group is obtained by summing the hash values ​​of each second group.

[0008] Determine whether the sum of the hash values ​​of the first group and the second group in each first pair is equal. If the sum of the hash values ​​of the first group and the second group in any first pair is not equal, then it is determined that there is an inconsistent field in the first group and the second group in any first pair.

[0009] In this scheme, the first and second database tables before and after migration are grouped to obtain at least one first pair. The first group and the second group in each first pair correspond one-to-one. The hash value of each first record in each first group is summed to obtain the hash value sum of each first group. The hash value of each second record in each second group is summed to obtain the hash value sum of each second record. By comparing whether the hash values ​​of the first group and the second group in each first pair are equal, it can be determined whether there are inconsistent fields in the first pair. This judgment method is simple and improves the efficiency of data verification.

[0010] Optionally, the step of grouping the first database table into at least one first group based on preset rules, and grouping the second database table into at least one second group based on the preset rules, includes: grouping the first database table into at least one first group and grouping the second database table into at least one second group according to the primary keys of the first database table and the second database table; wherein, the primary key is used to uniquely identify the records in the first database table and the second database table; and the primary key of at least one first record in the first group of each first pair is the same as the primary key of at least one second record in the second group.

[0011] This method allows for grouping of the first and second database tables based on the primary key used to uniquely identify records in the first and second database tables. This ensures that the second group in each first pair is obtained by migrating the first group from the source database to the target database, making the first and second groups in each first pair comparable and improving the reliability of the solution.

[0012] Optionally, the step of grouping the first database table into at least one first group and the second database table into at least one second group based on the primary keys of the first database table and the second database table includes: grouping first records in the first database table with the same content in the first specified position of the primary key into one group to obtain the at least one first group, and grouping second records in the second database table with the same content in the first specified position of the primary key into one group to obtain the at least one second group.

[0013] This method divides the first and second database tables based on the content of the primary key at the first specified position. This ensures that the first and second groups in each first pair not only have a corresponding relationship, but also that the primary key content at the specified position is the same for each first record in the first group and each second record in the second group. In this way, each first pair can be distinguished based on the content of the primary key at the specified position. After determining which first pair the inconsistent field appears in, it is also possible to know what the content of the primary key at the specified position is in the record where the inconsistent field is located, thus improving the accuracy of data validation.

[0014] Optionally, the step of grouping the first records in the first database table with the same primary key content at the first specified position into a group to obtain at least one first group, and grouping the second records in the second database table with the same primary key content at the first specified position into a group to obtain at least one second group, includes: grouping the first records in the first database table with the same primary key content at the first specified position into a group to obtain at least one sub-first group, and grouping the second records in the second database table with the same primary key content at the first specified position into a group to obtain at least one sub-second group; merging the at least one sub-first group into a preset number of at least one first group, and merging the at least one sub-second group into the preset number of at least one second group.

[0015] In this method, if the content of the primary key at the first specified position has multiple possibilities, the first database table and the second database table are divided according to the content of the primary key at the first specified position. The number of sub-first groups and sub-second groups in the at least one sub-first group may be too large. The at least one sub-first group and the at least one sub-second group are merged into a preset number of at least one first group and at least one second group respectively. In this way, only the sum of the hash values ​​of the preset number of first pair pairs needs to be compared, the number of comparisons is small, and the data verification efficiency is improved.

[0016] Optionally, the method further includes: if the hash values ​​of the target first group and the target second group included in any first pair are not equal, dividing the target first group into at least one third group and the target second group into at least one fourth group according to the primary key; wherein the third group and the corresponding fourth group form a second pair; summing the hash values ​​of each first record included in each third group to obtain the sum of the hash values ​​of each third group, and summing the hash values ​​of each second record included in each fourth group to obtain the sum of the hash values ​​of each fourth group; determining whether the sum of the hash values ​​of the third group and the fourth group in each second pair are equal, and if the sum of the hash values ​​of the third group and the fourth group in any second pair are not equal, then determining that there is an inconsistent field in the third group and the fourth group in any second pair.

[0017] This method, after determining which first pair the inconsistent field appears in, can further divide the first group in the first pair into at least one third group, and the second group in the first pair into at least one fourth group. The third group and the fourth group in each second pair correspond one-to-one. By comparing whether the sum of the hash values ​​of the third group and the fourth group in each second pair are equal, it can be determined whether an inconsistent field appears in the second pair. In this way, the scope of locating inconsistent fields is narrowed, further improving the accuracy of data verification.

[0018] Secondly, embodiments of this application provide a data verification device, which is applied in database migration services. The device includes modules / units / technical means for performing the methods described in the first aspect or any optional implementation of the first aspect.

[0019] For example, the device may include:

[0020] The acquisition module is used to acquire a first database table and a second database table; wherein, the second database table is a database table obtained by migrating the first database table in the source database to the target database;

[0021] The processing module is configured to group the first database table into at least one first group based on preset rules, and to group the second database table into at least one second group based on the preset rules; the first group and the corresponding second group form a first pair; each first group includes at least one first record, and each second group includes at least one second record; the hash values ​​of each first record included in each first group are summed to obtain the sum of hash values ​​of each first group, and the hash values ​​of each second record included in each second group are summed to obtain the sum of hash values ​​of each second group; it is determined whether the sum of hash values ​​of the first group and the second group in each first pair are equal, and if the sum of hash values ​​of the first group and the second group in any first pair are not equal, it is determined that there is an inconsistent field in the first group and the second group in any first pair.

[0022] Optionally, when the processing module groups the first database table to obtain at least one first group based on preset rules, and groups the second database table to obtain at least one second group based on the preset rules, it is specifically used to: group the first database table to obtain at least one first group and group the second database table to obtain at least one second group according to the primary keys of the first database table and the second database table; wherein, the primary key is used to uniquely identify the records in the first database table and the second database table; and the primary key of at least one first record included in the first group in each first pair is the same as the primary key of at least one second record included in the second group.

[0023] Optionally, when the processing module groups the first database table into at least one first group and the second database table into at least one second group based on the primary keys of the first database table and the second database table, it is specifically used to: group the first records in the first database table with the same content in the first specified position as the primary key into one group to obtain the at least one first group, and group the second records in the second database table with the same content in the first specified position as the primary key into one group to obtain the at least one second group.

[0024] Optionally, when the processing module groups the first records in the first database table with the same primary key content at the first specified position into a group to obtain at least one first group, and groups the second records in the second database table with the same primary key content at the first specified position into a group to obtain at least one second group, the specific steps are as follows: grouping the first records in the first database table with the same primary key content at the first specified position into a group to obtain at least one sub-first group, and grouping the second records in the second database table with the same primary key content at the first specified position into a group to obtain at least one sub-second group; merging the at least one sub-first group into a preset number of at least one first group, and merging the at least one sub-second group into the preset number of at least one second group.

[0025] Optionally, the processing module is further configured to: if the hash values ​​of the target first group and the target second group included in any first pair are not equal, divide the target first group into at least one third group and the target second group into at least one fourth group according to the primary key; wherein the third group and the corresponding fourth group form a second pair; sum the hash values ​​of each first record included in each third group to obtain the sum of the hash values ​​of each third group, and sum the hash values ​​of each second record included in each fourth group to obtain the sum of the hash values ​​of each fourth group; determine whether the sum of the hash values ​​of the third group and the fourth group in each second pair are equal, and if the sum of the hash values ​​of the third group and the fourth group in any second pair are not equal, determine that there is an inconsistent field in the third group and the fourth group in any second pair.

[0026] Thirdly, this application provides an electronic device, including: at least one processor; and a memory and a communication interface communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor, by executing the instructions stored in the memory, causes the electronic device to perform the method described in the first aspect or any optional embodiment of the first aspect through the communication interface.

[0027] Fourthly, this application provides a computer-readable storage medium for storing instructions that, when executed, cause the method described in the first aspect or any optional embodiment of the first aspect to be implemented.

[0028] Fifthly, this application provides a computer program product comprising: computer program code, which, when executed on a computer, causes the computer to perform the method described in the first aspect or any optional embodiment of the first aspect.

[0029] The technical effects or advantages of one or more technical solutions provided in the second, third, fourth and fifth aspects of this application can all be explained by the corresponding technical effects or advantages of one or more technical solutions provided in the first aspect, and will not be repeated here. Attached Figure Description

[0030] Figure 1 A flowchart illustrating a data verification method provided in this application embodiment;

[0031] Figure 2 A schematic diagram illustrating a data verification method provided in an embodiment of this application;

[0032] Figure 3 A structural diagram of a data verification device provided in an embodiment of this application;

[0033] Figure 4 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0035] The terms "first" and "second" in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising" and any variations thereof are intended to cover non-exclusive protection. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. The term "multiple" in this application can mean at least two, for example, two, three, or more, and the embodiments of this application do not impose limitations.

[0036] The term "and / or" in the embodiments of this application is merely a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0037] The data collection, dissemination, and use in this application all comply with relevant national laws and regulations.

[0038] To facilitate understanding of the solutions in the embodiments of this application, the possible application scenarios of the embodiments of this application will be introduced below.

[0039] In database migration operations, data in the source database needs to be migrated to the target database. In order to evaluate the quality of data migration, it is necessary to verify whether the data before and after migration are the same. If they are different, it is also necessary to find the inconsistent fields to help technical personnel identify the problems that occurred during the database migration process.

[0040] In existing technologies, inconsistent fields can be located by comparing each record and each field, but this method is inefficient.

[0041] Therefore, this application provides a technical solution to improve data verification efficiency.

[0042] For the above scenarios, the data verification method provided by this invention will be described in detail below with reference to the accompanying drawings. It should be noted that this method is applied to database migration operations.

[0043] See Figure 1 The flowchart below illustrates a data verification method provided in this application. This method can be executed by computer devices, such as laptops, desktop computers, and servers, and can also be applied to various devices with computing capabilities, such as the server containing the source database or the server containing the target database. The above devices are merely illustrative examples and are not intended to limit the scope of this application.

[0044] The following example illustrates how this method is executed by a computer device. The method includes:

[0045] S101: Obtain the first database table and the second database table.

[0046] The first database table is used to store data in the source database, and the second database table is used to store data in the target database. The second database table is obtained by migrating the first database table from the source database to the target database.

[0047] S102: Group the first database table according to preset rules to obtain at least one first group, and group the second database table according to preset rules to obtain at least one second group.

[0048] In this configuration, each of the at least one first group includes at least one first record, and each of the at least one second group includes at least one second record. Each first group and its corresponding second group form a first pair.

[0049] When grouping the first database table and the second database table, the first database table can be grouped into at least one first group and the second database table can be grouped into at least one second group based on the primary keys of the first database table and the second database table.

[0050] The primary key is a combination of one or more columns in a database table. The primary key is used to uniquely identify records in the first database table and the second database table. In each first pair, the primary key of at least one first record in the first group is the same as the primary key of at least one second record in the second group.

[0051] For example, each record in the first and second database tables stores a person's identity information. The field names in the first and second database tables include: name, age, gender, ethnicity, and ID number. The ID number serves as the primary key in both tables, uniquely identifying each person's identity. The first and second database tables are grouped according to the ID number, ensuring that the identity information corresponding to the records in the first group of each first pair is identical to the identity information corresponding to the records in the second group.

[0052] In particular, embodiments of this application can also partition the database table based on a non-empty unique index. The unique index is also used to uniquely identify records in the first database table and the second database table. The methods for partitioning the database table based on a non-empty unique index and the methods for partitioning the database table based on the primary key of the database table will not be described in detail here.

[0053] This method allows for grouping of the first and second database tables based on the primary key used to uniquely identify records in the first and second database tables. This ensures that the second group in each first pair is obtained by migrating the first group from the source database to the target database, making the first and second groups in each first pair comparable and improving the reliability of the solution.

[0054] Grouping the first and second database tables based on the primary key can be done in the following ways.

[0055] In one possible implementation, the first database table is grouped into at least one first group and the second database table is grouped into at least one second group based on the value of the primary key of each record.

[0056] For example, the primary key of both the first and second database tables is a number, and the number of each record indicates which record it is in the database table. For instance, if both the first and second database tables contain 1000 records, the first record with number 0001 indicates that it is the first record in the first database table, and the second record with number 0123 indicates that it is the 123rd record in the second database table. Following the ascending order of the numbers, the first records with numbers 0001 to 0100 are grouped into one first group, the first records with numbers 0101 to 0200 are grouped into another first group, and so on. Similarly, the grouping method for the second database table is similar to that for the first database table, and will not be repeated here. The first and second groups in a pair have the same numbers. Referring to the example above, if the first group in a pair contains records with numbers 0001 to 0100, then the corresponding second group also contains records with numbers 0001 to 0100.

[0057] Understandably, the above is only an example and not a limitation. In practical applications, the number of first records included in different first groups can be the same or different, and the number of second records included in different second groups can be the same or different.

[0058] This method groups the first and second database tables based on the primary key value, resulting in high grouping efficiency and simple, easy-to-implement operations.

[0059] In another possible implementation, first records in the first database table with the same primary key content at the first specified position are grouped together to obtain at least one first group, and second records in the second database table with the same primary key content at the first specified position are grouped together to obtain at least one second group.

[0060] For example, the primary key of the first and second database tables is the ID card number, and each record stores a person's identity information. Assuming the first specified position is the last digit of the ID card number, which can be 0 to 9 or X (10 possibilities), the first and second database tables can be divided into 10 groups. The last digit of the ID card numbers in the first records of each first group is the same, the last digit of the ID card numbers in the second records of each second group is the same, and the last digit of the ID card numbers in the records of both the first and second groups within each first pair is the same.

[0061] It is understood that the above is only an example and not a limitation. The first specified position can also be any combination of one or more characters in the field where the primary key is located (for example, the first three characters of the field where the primary key is located are the first specified position). In addition, the content of the first specified position can be numbers, letters, a combination of numbers and letters, or other forms. This application does not impose any restrictions.

[0062] This method divides the first database table and the second database table based on the content of the primary key at the first specified position. This ensures that the first group and the second group in each first pair not only have a corresponding relationship, but also that the primary key content of each first record in the first group and each second record in the second group in the first pair is the same at the specified position. Thus, after determining which first pair the inconsistent field appears in, we can also know what the content of the primary key of the record containing the inconsistent field is at the specified position, thereby improving the accuracy of data verification.

[0063] Optionally, the first records in the first database table with the same primary key content at the first specified position are grouped together to obtain at least one sub-first group, and the second records in the second database table with the same primary key content at the first specified position are grouped together to obtain at least one sub-second group. It is determined whether the number of sub-first groups included in the at least one sub-first group exceeds a preset number, or whether the number of sub-second groups included in the at least one sub-second group exceeds a preset number. If not, the at least one sub-first group is considered as at least one first group, and the at least one sub-second group is considered as at least one second group. If they exceed the preset number, the at least one sub-first group is merged into at least one preset number of first groups, and the at least one sub-second group is merged into at least one preset number of second groups. Using the example above where the primary key of the first and second database tables is an ID number, assuming the first specified position is the last digit of the ID number, the at least one sub-first group includes 10 sub-first groups, and the at least one sub-second group includes 10 sub-second groups. If the preset number is 5 groups, any two groups from the at least one sub-first group are merged into one group, resulting in 5 first groups. The at least one sub-second group is merged in the same way, resulting in 5 second groups.

[0064] Understandably, the preset quantity can be specified according to actual needs, and this application does not impose any restrictions.

[0065] In this method, if the content of the primary key at the first specified position has multiple possibilities, the first database table and the second database table are divided according to the content of the primary key at the first specified position. The number of sub-first groups and sub-second groups in the at least one sub-first group may be too large. The at least one sub-first group and the at least one sub-second group are merged into a preset number of at least one first group and at least one second group respectively. In this way, the amount of data to be compared is small, which improves the data verification efficiency.

[0066] S103: Summing the hash values ​​of each first record included in each first group yields the sum of hash values ​​for each first group, and summing the hash values ​​of each second record included in each second group yields the sum of hash values ​​for each second group.

[0067] For example, the field values ​​of each field included in each first record are concatenated to obtain the first string corresponding to each first record. The hash value of the first string corresponding to each first record is used as the hash value of each first record. The hash values ​​of each first record included in each first group are summed to obtain the sum of the hash values ​​of each first group. The calculation method for the sum of the hash values ​​of each second group is similar to the calculation method for the sum of the hash values ​​of each first group, and will not be repeated here.

[0068] S104: Determine whether the sum of the hash values ​​of the first group and the second group in each first pair is equal. If the sum of the hash values ​​of the first group and the second group in any first pair is not equal, then determine that there is an inconsistent field in the first group and the second group in any first pair.

[0069] If the sum of the hash values ​​of the first group and the second group in the first pair is equal, then it is determined that there are no inconsistent fields in the first group and the second group in any first pair.

[0070] After determining that the hash values ​​of the target first group and the target second group included in any first pair are not equal, the target first group can be divided into at least one third group and the target second group can be divided into at least one fourth group according to the primary key; wherein, the third group and the corresponding fourth group form a second pair; the hash values ​​of each first record included in each third group are summed to obtain the sum of the hash values ​​of each third group, and the hash values ​​of each second record included in each fourth group are summed to obtain the sum of the hash values ​​of each fourth group; it is determined whether the sum of the hash values ​​of the third group and the fourth group in each second pair are equal. If the sum of the hash values ​​of the third group and the fourth group in any second pair are not equal, it is determined that there is an inconsistent field in the third group and the fourth group in any second pair.

[0071] Understandably, the method of dividing the first target group into at least one third group and the second target group into at least one fourth group based on the primary key can be similar to the method of grouping the first database table and the second database table based on the value of the primary key in S102. It can also be similar to the method of grouping the first database table and the second database table based on the content of the primary key in the first specified position in S102. The difference is that when grouping the first target group and the second target group, it is necessary to group the first database table and the second database table based on the content of the primary key in the second specified position, where the first specified position and the second specified position are different.

[0072] In the above schemes S101 to S104, the first database table and the second database table before and after migration are grouped to obtain at least one first pair. The first group and the second group in each first pair correspond one-to-one. The hash value of each first record included in each first group is summed to obtain the hash value sum of each first group. The hash value of each second record included in each second group is summed to obtain the hash value sum of each second record. By comparing whether the hash values ​​of the first group and the second group in each first pair are equal, it can be determined whether there are inconsistent fields in the first pair. This judgment method is simple and improves the efficiency of data verification.

[0073] See Figure 2 To better understand the technical solutions of the embodiments of this application, a more complete example is given below.

[0074] The primary key of the first database table T1 and the second database table T2 is the ID number F, and each record is used to store a person's identity information.

[0075] Group the first and second database tables according to the last digit of the primary key. For both tables T1 and T2, use the query "F like '%1'" (find records where the last digit of F is 1) to obtain a first pair. The primary keys of all records in this first pair end in 1. The sum of the hash values ​​of the first group in this first pair is denoted as H(T1F:1), and the sum of the hash values ​​of the second group is denoted as H(T2F:1). Similarly, this process is repeated to obtain the first pair where the primary key of the ID card number is any of the other values.

[0076] Determine if the sum of the hash values ​​of the first and second groups in each first pair are equal. If H(T1F:i) = H(T2F:i), then there is no inconsistent field in the first pair where the last digit of the primary key is i. If H(T1F:3) ≠ H(T2F:3), then there is an inconsistent field in the first pair where the last digit of the primary key is 3.

[0077] Group the first pairs of primary key pairs whose last digit is 3 according to the second-to-last digit of the primary key, using the same grouping method as grouping the first and second database tables according to the last digit of the primary key, resulting in at least one second pair. Determine if the sum of the hash values ​​of the third and fourth groups in each second pair is equal. If H(T1F:3j) = H(T2F:3j), then no inconsistent fields appear in the second pair of primary key pairs where the second-to-last digit is j. If H(T1F:34) ≠ H(T2F:34), then an inconsistent field appears in the second pair of primary key pairs where the second-to-last digit is 4.

[0078] Following the order of the primary key from back to front, after identifying the Nth pair where the inconsistent field appears, the Nth pair is then grouped and compared again until the Mth pair appears. In the Mth pair, the first group contains only one record from the first record, and the second group contains only one record from the second record. At this point, the process terminates. This allows us to locate which record the inconsistent field appears in. Here, M and N are both positive integers.

[0079] exist Figure 2 In this example, a record with the third-to-last digit of the primary key being 9, the second-to-last digit being 4, and the last digit being 3 is an example of a record with inconsistent fields.

[0080] After the above process is terminated, a prompt message can be sent to the technician's equipment. The prompt message indicates which record the inconsistent field between the first database table and the second database table appears in, so that the technician can investigate the cause of the error.

[0081] See Figure 3 This application provides a data verification device 300, which is applied in database migration services. The device includes modules / units / technical means for performing the methods executed by computer devices in the above method embodiments.

[0082] For example, the device 300 may include:

[0083] The acquisition module 301 is used to acquire a first database table and a second database table; wherein, the second database table is a database table obtained by migrating the first database table in the source database to the target database;

[0084] Processing module 302 is configured to group the first database table into at least one first group based on preset rules, and to group the second database table into at least one second group based on the preset rules; the first group and the corresponding second group form a first pair; each first group includes at least one first record, and each second group includes at least one second record; the hash values ​​of each first record included in each first group are summed to obtain the sum of hash values ​​of each first group, and the hash values ​​of each second record included in each second group are summed to obtain the sum of hash values ​​of each second group; it is determined whether the sum of hash values ​​of the first group and the second group in each first pair are equal, and if the sum of hash values ​​of the first group and the second group in any first pair are not equal, it is determined that there is an inconsistent field in the first group and the second group in any first pair.

[0085] Optionally, when the processing module 302 groups the first database table to obtain at least one first group based on preset rules, and groups the second database table to obtain at least one second group based on the preset rules, it is specifically used to: group the first database table to obtain at least one first group and group the second database table to obtain at least one second group according to the primary keys of the first database table and the second database table; wherein, the primary key is used to uniquely identify the records in the first database table and the second database table; and the primary key of at least one first record included in the first group in each first pair is the same as the primary key of at least one second record included in the second group.

[0086] Optionally, when the processing module 302 groups the first database table to obtain at least one first group and the second database table to obtain at least one second group based on the primary keys of the first database table and the second database table, it is specifically used to: group the first records in the first database table with the same content of the primary key at a first specified position into a group to obtain the at least one first group, and group the second records in the second database table with the same content of the primary key at the first specified position into a group to obtain the at least one second group.

[0087] Optionally, when the processing module 302 divides the first records in the first database table with the same primary key content at the first specified position into a group to obtain at least one first group, and divides the second records in the second database table with the same primary key content at the first specified position into a group to obtain at least one second group, the specific steps are as follows: dividing the first records in the first database table with the same primary key content at the first specified position into a group to obtain at least one sub-first group, and dividing the second records in the second database table with the same primary key content at the first specified position into a group to obtain at least one sub-second group; merging the at least one sub-first group into a preset number of at least one first group, and merging the at least one sub-second group into the preset number of at least one second group. Optionally, the processing module 302 is further configured to: if the hash values ​​of the target first group and the target second group included in any first pair are not equal, divide the target first group into at least one third group and the target second group into at least one fourth group according to the primary key; wherein the third group and the corresponding fourth group form a second pair; sum the hash values ​​of each first record included in each third group to obtain the sum of the hash values ​​of each third group, and sum the hash values ​​of each second record included in each fourth group to obtain the sum of the hash values ​​of each fourth group; determine whether the sum of the hash values ​​of the third group and the fourth group in each second pair are equal, and if the sum of the hash values ​​of the third group and the fourth group in any second pair are not equal, determine that there is an inconsistent field in the third group and the fourth group in any second pair.

[0088] It should be understood that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0089] As one possible product form of the aforementioned device, see [link to product description]. Figure 4 This application also provides an electronic device 400, comprising:

[0090] At least one processor 401; and a communication interface 403 communicatively connected to the at least one processor 401; the at least one processor 401 causes the electronic device 400 to perform, for example, through the communication interface 403, instructions stored in the memory 402 by executing instructions stored in the memory 402. Figure 1 The method in the illustrated embodiment.

[0091] Optionally, the memory 402 is located outside the electronic device 400.

[0092] Optionally, the electronic device 400 includes the memory 402, which is connected to the at least one processor 401, and stores instructions executable by the at least one processor 401. (Appendix) Figure 4 The dashed line indicates that memory 402 is optional for electronic device 400.

[0093] The processor 401 and the memory 402 can be coupled through an interface circuit or integrated together; no restriction is imposed here.

[0094] This application embodiment does not limit the specific connection medium between the processor 401, memory 402, and communication interface 403. This application embodiment... Figure 4 The processor 401, memory 402, and communication interface 403 are connected via a bus 404. Figure 4 The connections between other components are shown in bold and are for illustrative purposes only, not as limiting information. The bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0095] It should be understood that the processor mentioned in the embodiments of this application can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0096] For example, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0097] It should be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct RAM (DR RAM).

[0098] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) can be integrated into the processor.

[0099] It should be noted that the memories described herein are intended to include, but are not limited to, these and any other suitable types of memories.

[0100] As another possible product form, embodiments of this application also provide a computer-readable storage medium for storing instructions that, when executed, cause a computer to perform actions such as... Figure 1 The method in the illustrated embodiment.

[0101] As another possible product form, this application embodiment also provides a computer program product containing instructions, wherein the computer program product stores instructions that, when run on a computer, cause the computer to perform actions such as... Figure 1 The method in the illustrated embodiment.

[0102] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0103] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0104] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0105] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes. Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, this application also intends to include such modifications and variations if they fall within the scope of the claims of this application and their equivalents.

Claims

1. A data checking method, characterized by, The method is applied to a database migration service, and the method comprises: obtaining a first database table and a second database table; wherein the second database table is a database table obtained by migrating the first database table in a source database to a target database; grouping the first database table based on a preset rule to obtain at least one first group, and grouping the second database table based on the preset rule to obtain at least one second group; the first group and the corresponding second group form a first comparison pair; each first group in the at least one first group comprises at least one first record, and each second group in the at least one second group comprises at least one second record; summing the hash values of each first record included in each first group to obtain a sum of hash values of the first group, and summing the hash values of each second record included in each second group to obtain a sum of hash values of the second group; determining whether the sum of hash values of the first group and the second group in each first comparison pair is equal, and if the sum of hash values of the first group and the second group in any first comparison pair is not equal, determining that an inconsistent field appears in the first group and the second group in the any first comparison pair.

2. The method of claim 1, wherein, The grouping of the first database table based on a preset rule to obtain at least one first group, and the grouping of the second database table based on the preset rule to obtain at least one second group, comprises: grouping the first database table based on a primary key of the first database table and the second database table to obtain at least one first group, and grouping the second database table based on the primary key to obtain at least one second group; wherein the primary key is used to uniquely identify the records in the first database table and the second database table; the primary key of the at least one first record included in the first group in each first comparison pair is the same as the primary key of the at least one second record included in the second group.

3. The method of claim 2, wherein, The grouping of the first database table based on a primary key of the first database table and the second database table to obtain at least one first group, and the grouping of the second database table based on the primary key to obtain at least one second group, comprises: grouping the first records in the first database table with the same content of the primary key in a first specified position into a group to obtain the at least one first group, and grouping the second records in the second database table with the same content of the primary key in the first specified position into a group to obtain the at least one second group.

4. The method of claim 3, wherein, The grouping of the first records in the first database table with the same content of the primary key in a first specified position into a group to obtain the at least one first group, and the grouping of the second records in the second database table with the same content of the primary key in the first specified position into a group to obtain the at least one second group, comprises: grouping the first records in the first database table with the same content of the primary key in a first specified position into a group to obtain at least one first sub-group, and grouping the second records in the second database table with the same content of the primary key in the first specified position into a group to obtain at least one second sub-group; grouping the first records in the first database table with the same content of the primary key in a first specified position into a group to obtain at least one first sub-group, and grouping the second records in the second database table with the same content of the primary key in the first specified position into a group to obtain at least one second sub-group; merge the at least one sub-first group into a preset number of the at least one first group, and merge the at least one sub-second group into a preset number of the at least one second group.

5. The method of claim 1, wherein, The method further comprises: if the hash values of the target first group and the target second group included in any first control pair are not equal, dividing the target first group into at least one third group and the target second group into at least one fourth group according to the primary keys of the first database table and the second database table; wherein the third group and the corresponding fourth group form a second control pair; summing the hash values of each first record included in each third group to obtain the hash value sum of each third group, and summing the hash values of each second record included in each fourth group to obtain the hash value sum of each fourth group; determining whether the hash value sums of the third group and the fourth group in each second control pair are equal, and if the hash value sums of the third group and the fourth group in any second control pair are not equal, determining that inconsistent fields exist in the third group and the fourth group in the any second control pair.

6. A data checking apparatus characterized by comprising: The device is applied to a database migration service, and the device comprises: an acquisition module configured to acquire a first database table and a second database table; wherein the second database table is a database table obtained by migrating a first database table in a source database to a target database; a processing module configured to group the first database table based on a preset rule to obtain at least one first group, and group the second database table based on the preset rule to obtain at least one second group; the first group and the corresponding second group form a first control pair; each first group in the at least one first group includes at least one first record, and each second group in the at least one second group includes at least one second record; summing the hash values of each first record included in each first group to obtain the hash value sum of each first group, and summing the hash values of each second record included in each second group to obtain the hash value sum of each second group; determining whether the hash value sums of the first group and the second group in each first control pair are equal, and if the hash value sums of the first group and the second group in any first control pair are not equal, determining that inconsistent fields exist in the first group and the second group in the any first control pair.

7. The apparatus of claim 6, wherein, The processing module, when grouping the first database table based on a preset rule to obtain at least one first group and grouping the second database table based on the preset rule to obtain at least one second group, is specifically configured to: group the first database table based on the primary keys of the first database table and the second database table to obtain at least one first group and group the second database table based on the primary keys of the first database table and the second database table to obtain at least one second group; wherein the primary keys are used to uniquely identify the records in the first database table and the second database table; the primary keys of the at least one first record included in the first group in each first control pair are the same as the primary keys of the at least one second record included in the second group.

8. The apparatus of claim 7, wherein, The processing module is specifically configured to: divide the first records with the same content of the primary keys in a first specified position in the first database table into a group to obtain the at least one first group, and divide the second records with the same content of the primary keys in the first specified position in the second database table into a group to obtain the at least one second group.

9. The apparatus of claim 8, wherein, The processing module is specifically configured to: divide the first records with the same content of the primary keys in a first specified position in the first database table into a group to obtain the at least one first group, and divide the second records with the same content of the primary keys in the first specified position in the second database table into a group to obtain the at least one second group. The processing module is specifically configured to:

10. The apparatus of claim 6, wherein, divide the first records with the same content of the primary keys in a first specified position in the first database table into a group to obtain the at least one first group, and divide the second records with the same content of the primary keys in the first specified position in the second database table into a group to obtain the at least one second group. The processing module is further configured to: if the hash values of the target first group and the target second group included in the any first comparison pair are not equal, divide the target first group into at least one third group and divide the target second group into at least one fourth group according to the primary keys of the first database table and the second database table, wherein the third group and the corresponding fourth group form a second comparison pair; sum the hash values of each first record included in each third group to obtain the sum of the hash values of each third group, and sum the hash values of each second record included in each fourth group to obtain the sum of the hash values of each fourth group; 11. An electronic device, comprising: determine whether the sum of the hash values of the third group and the fourth group in each second comparison pair is equal, and if the sum of the hash values of the third group and the fourth group in any second comparison pair is not equal, determine that the inconsistent field appears in the third group and the fourth group in the any second comparison pair. comprise: at least one processor; and a memory and a communication interface connected in communication with the at least one processor; 12. A computer-readable storage medium, characterized in that, wherein the memory stores instructions executable by the at least one processor, and the at least one processor, by executing the instructions stored in the memory, causes the electronic device to execute the method of any one of claims 1-5 through the communication interface.

13. A computer program product, characterised in that, The computer readable storage medium stores computer instructions, when the computer instructions run on a computer, causes the computer to execute the method of any one of claims 1-5. The computer program product comprises: computer program code, when the computer program code runs on a computer, causes the computer to execute the method of any one of claims 1-5.

Images