Incremental data verification method and device, equipment, medium and program product

Abstract

The invention discloses an incremental data verification method and device, equipment, a medium and a program product, and the method comprises the steps: respectively obtaining first incremental data and second incremental data of a source database and a target database in a first time period after it is determined that migration and verification of total data from the source database to the target database are completed, splicing each column of incremental data of a target row in a first table in the first database, converting the spliced incremental data into a character string type to obtain character string type data, and mapping the character string type data into integer type data to obtain integer type data; respectively calculating the integer data in each table of the source database and the target database through a Merkel binary tree algorithm to respectively obtain a first Merkel binary tree and a second Merkel binary tree; according to the first Merkel binary tree and the second Merkel binary tree, a verification result is obtained, the problem that in the prior art, an incremental data verification result is inaccurate is solved, and the accuracy of incremental data verification is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an incremental data verification method, device, equipment, medium and program product. Background Art

[0002] In the process of migrating data from the relational database MySQL to other types of databases, there are usually two stages: full data migration and incremental data migration. Full migration focuses on obtaining a complete data snapshot in MySQL at the current time point and migrating it to other types of databases. The goal of incremental migration is to supplement the data that is added or modified after the full migration. In order to ensure the consistency of data before and after migration, the data on the source and destination ends needs to be verified to ensure that the data on the source and destination ends remain consistent.

[0003] The core of the verification of incremental data is to detect the newly added or modified data after comparing the data after the full migration. At present, the common way to verify incremental data is to capture the binary log of MySQL or the master-slave replication log, which records in detail all the changes of data in MySQL, including insertion, update and deletion. Then, the target database can obtain the incremental data by reading these logs and compare it with the source data in the source database.

[0004] However, if the incremental data verification time is too long, some changed data that occurred before the current time point may be missed, resulting in inaccurate comparison results. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide an incremental data verification method, device, equipment, medium and program product, which improves the efficiency and accuracy of incremental data verification.

[0006] The technical solution of this application is as follows:

[0007] In a first aspect, a method for verifying incremental data is provided, the method comprising:

[0008] In the case of determining that the migration and verification of the full data from the source database to the target database are completed, obtaining first incremental data of the source database within a first time period and second incremental data of the target database within the first time period, wherein the first incremental data and the second incremental data are respectively stored in at least one table in their corresponding databases, the first time period is a time period between the current time and the first time, and the first time is the time when the verification of the incremental data is most recently completed from the current time;

[0009] For the incremental data of a target row in a first table in a first database, concatenate the incremental data of each column of the target row and convert them into string type data to obtain string type data corresponding to the incremental data of the target row, wherein the first database is the source database and the target database, the first table is each table in the first database, the target row is each row of the first table, when the first database is the source database, the incremental data is the first incremental data, and when the first database is the target database, the incremental data is the second incremental data;

[0010] Mapping the string data corresponding to the incremental data of each row into integer data of a fixed length to obtain the integer data corresponding to the incremental data of each row;

[0011] The integer data corresponding to each row of incremental data in each table of the source database is calculated by a Merkle binary tree algorithm to obtain a first Merkle binary tree corresponding to the source database, and the integer data corresponding to each row of incremental data in each table of the target database is calculated by the Merkle binary tree algorithm to obtain a second Merkle binary tree corresponding to the target database;

[0012] The first Merkle binary tree is compared with the second Merkle binary tree to obtain a verification result between the first incremental data and the second incremental data.

[0013] In a second aspect, an incremental data verification device is provided, the device comprising:

[0014] A first acquisition module is used to acquire, when it is determined that the migration and verification of the full data from the source database to the target database are completed, first incremental data of the source database within a first time period and second incremental data of the target database within the first time period, wherein the first incremental data and the second incremental data are respectively stored in at least one table in their corresponding databases, the first time period is a time period between the current time and the first time, and the first time is the time when the verification of the incremental data is most recently completed from the current time;

[0015] A first determination module is used for concatenating the incremental data of each column of a target row in a first table in a first database and converting the concatenated incremental data into string type data to obtain string type data corresponding to the incremental data of the target row, wherein the first database is the source database and the target database, the first table is each table in the first database, the target row is each row of the first table, when the first database is the source database, the incremental data is the first incremental data, and when the first database is the target database, the incremental data is the second incremental data;

[0016] A second determination module is used to map the character string data corresponding to the incremental data of each row into integer data of a fixed length to obtain the integer data corresponding to the incremental data of each row;

[0017] A third determination module is used to calculate the integer data corresponding to each row of incremental data in each table of the source database by using a Merkel binary tree algorithm to obtain a first Merkel binary tree corresponding to the source database, and to calculate the integer data corresponding to each row of incremental data in each table of the target database by using the Merkel binary tree algorithm to obtain a second Merkel binary tree corresponding to the target database;

[0018] The fourth determination module is used to compare the first Merkle binary tree with the second Merkle binary tree to obtain a verification result between the first incremental data and the second incremental data.

[0019] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the incremental data verification method described in any one of the embodiments of the present application.

[0020] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the incremental data verification method described in any one of the embodiments of the present application are implemented.

[0021] In a fifth aspect, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device can perform the steps of the incremental data verification method described in any one of the embodiments of the present application.

[0022] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:

[0023] In an embodiment of the present application, when it is determined that the migration and verification of the full amount of data from the source database to the target database is completed, the first incremental data of the source database in the first time period and the second incremental data of the target database in the first time period are obtained, and then the first incremental data and the second incremental data are verified. In this way, when verifying the incremental data, only the incremental data from the last verification to the current time period is verified, rather than verifying all the incremental data. This reduces the time for verification during the migration of incremental data and improves the efficiency of incremental data verification. In addition, when verifying the first incremental data and the second incremental data, the incremental data of each column of the row is concatenated and converted into string type data for each row of incremental data in each table in each database in the source database and the target database, and the string type data corresponding to the incremental data of each row is obtained. In this way, when the incremental data in the source database and the target database are subsequently compared, each row in each table is only compared once, and there is no need to compare multiple times, which further improves the efficiency of incremental data verification. Then, the string type data corresponding to the incremental data of each row is mapped to integer type data of fixed length to obtain the integer type data corresponding to the incremental data of each row, and the Merkel binary tree is used to obtain the integer type data corresponding to the incremental data of each row. The algorithm calculates the integer data corresponding to each row of incremental data in each table of the source database to obtain the first Merkel binary tree corresponding to the source database, and calculates the integer data corresponding to each row of incremental data in each table of the target database through the Merkel binary tree algorithm to obtain the second Merkel binary tree corresponding to the target database. The first Merkel binary tree and the second Merkel binary tree are compared to obtain the verification result between the first incremental data and the second incremental data. In this way, the first incremental data and the second incremental data are converted, mapped and compared into string types, thereby ensuring the consistency of the first incremental data and the second incremental data and improving the verification accuracy of the first incremental data and the second incremental data.

[0024] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application, and do not constitute improper limitations on the present application.

[0026] Figure 1 It is a flowchart of an incremental data verification method provided in an embodiment of the present application;

[0027] Figure 2 is a schematic diagram of a first table provided in an embodiment of the present application;

[0028] Figure 3It is a flowchart of an incremental data verification method provided in an embodiment of the present application;

[0029] Figure 4 It is a structural schematic diagram of an incremental data verification device provided in an embodiment of the present application;

[0030] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make those of ordinary skill in the art better understand the technical scheme of the present application, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are only examples consistent with some aspects of the present application as detailed in the attached claims.

[0033] As described in the background technology section, there is a problem of inaccurate incremental data verification results in the prior art. In order to solve this problem, the embodiments of the present application provide an incremental data verification method, apparatus, equipment, medium and program product. When it is determined that the migration and verification of the full data from the source database to the target database are completed, the first incremental data of the source database within the first time period and the second incremental data of the target database within the first time period are obtained, and then the first incremental data and the second incremental data are verified. In this way, when verifying the incremental data, only the incremental data from the last verification to the current time period is verified, rather than verifying all the incremental data. This reduces the verification time during incremental data migration and improves the efficiency of incremental data verification. In addition, when verifying the first incremental data and the second incremental data, the incremental data of each row in each table in each database in the source database and the target database are concatenated and converted into string type data to obtain the string type data corresponding to the incremental data of each row. In this way, when the incremental data in the source database and the target database are subsequently compared, each row in each table is only compared once, and there is no need to compare multiple times, which further improves the efficiency of incremental data verification. Then, the string type data corresponding to the incremental data of each row is mapped to integer type data of fixed length to obtain the integer type data corresponding to the incremental data of each row. Through the Merkle binary tree The algorithm calculates the integer data corresponding to each row of incremental data in each table of the source database to obtain the first Merkel binary tree corresponding to the source database, and calculates the integer data corresponding to each row of incremental data in each table of the target database through the Merkel binary tree algorithm to obtain the second Merkel binary tree corresponding to the target database. The first Merkel binary tree and the second Merkel binary tree are compared to obtain the verification result between the first incremental data and the second incremental data. In this way, the first incremental data and the second incremental data are converted, mapped and compared into string types, thereby ensuring the consistency of the first incremental data and the second incremental data and improving the verification accuracy of the first incremental data and the second incremental data.

[0034] The incremental data verification method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in combination with the accompanying drawings.

[0035] Figure 1 is a flow chart of an incremental data verification method provided by an embodiment of the present application, such as Figure 1 As shown, the incremental data verification method provided in the embodiment of the present application may include steps 110 to 150.

[0036] Step 110: When it is determined that the migration and verification of the full data from the source database to the target database are completed, obtain the first incremental data of the source database within the first time period and the second incremental data of the target database within the first time period.

[0037] The source data may be a database to be migrated.

[0038] The target database may be the database to which the data is to be migrated.

[0039] In some embodiments of the present application, the source database may be a relational database, such as a MySQL database, and the target database may be a non-relational database, such as a key-value database, a graph database, a time series database, a column-family database, and a document-oriented database. The specific target database may be selected according to user needs and is not limited in the embodiments of the present application.

[0040] The first time period may be a time period between the current time and the first time, where the first time may be the time when the incremental data verification was completed most recently from the current time, that is, the first time is the time when the incremental data verification was completed last time.

[0041] In an example, the last time the incremental data verification was completed was 9:00 am on July 21, 2024, and the current time is 9:30 am on July 21, 2024. The first time period is half an hour from 9:00 am on July 21, 2024 to 9:30 am on July 21, 2024.

[0042] It should be noted that, when the first time period is small enough, real-time verification of the incremental data can be achieved through steps 110 to 150 of the embodiment of the present application.

[0043] The first incremental data may be incremental data of the source database within a first time period.

[0044] The second incremental data may be incremental data of the target database within the first time period.

[0045] It should be noted that the first incremental data and the second incremental data can be stored in at least one table in their corresponding databases, that is, the first incremental data can be stored in at least one table in the source database, and the second incremental data can be stored in at least one table in the target database. For each table in the source database and the target database, the table is a table with M rows and N columns, in which the incremental data is stored, where M and N are both positive integers, and specifically can be positive integers greater than 1.

[0046] In some embodiments of the present application, in order to accurately obtain the incremental data in the source database and the target database, the obtaining of the first incremental data of the source database in the first time period and the second incremental data of the target database in the first time period in step 110 may specifically include:

[0047] Get historical target incremental data;

[0048] Parsing log information in a first time period in a logical log and a physical log of a source database to obtain third incremental data of the first time period;

[0049] The third incremental data and the first data in the source database are used as the first incremental data;

[0050] The data in the target database within the first time period except the historical target incremental data is used as the second incremental data.

[0051] Among them, the historical target incremental data can be the inconsistent data in the source database and the target database obtained after the incremental data of the source database and the target database are verified at the first time, that is, the historical target incremental data is the inconsistent data in the source database and the target database obtained when the incremental data was verified last time.

[0052] Both the logical log and the physical log of the source database may contain execution statements describing data update operations. For example, the logical log of the source database may be the binlog of the source database, and the physical log of the source database may be the redo log of the source database.

[0053] It should be noted that the data recorded in the logical log and physical log of the source database are consistent in terms of coarse granularity, but the storage format is different. However, strictly speaking, the data stored in the redo log is more detailed than that in the binlog, so when parsing log information to obtain incremental data, both logs need to be parsed, so that data that is not parsed in the binlog can be obtained by parsing the redo log.

[0054] The third incremental data may be the incremental data within the first time period obtained by parsing log information in the logical log and the physical log of the source database within the first time period.

[0055] Continuing to refer to the above example, taking the first time period as the half hour from 9:00 am on July 21, 2024 to 9:30 am on July 21, 2024 as an example, the third incremental data is the incremental data within the half hour from 9:00 am on July 21, 2024 to 9:30 am on July 21, 2024.

[0056] The first data may be data in the source database within the first time period excluding the historical target incremental data, that is, the first data may be data in the source database within the first time period excluding the target incremental data.

[0057] In an example, taking the first time period of half an hour from 9:00 a.m. on July 21, 2024 to 9:30 a.m. on July 21, 2024 as an example, there are 1,000 data in the source database within the first time period of half an hour from 9:00 a.m. on July 21, 2024 to 9:30 a.m. on July 21, 2024. The data that are historical target incremental data among the 1,000 data are excluded. For example, if 100 of the 1,000 data are historical target incremental data, the first data is the remaining 900 data.

[0058] In some embodiments of the present application, the inconsistent target incremental data between the source database and the target database obtained from the last incremental data verification is obtained, and then the log information in the logical log and physical log of the source database within the first time period is parsed to obtain the third incremental data of the first time period. In this way, the third incremental data and the first data in the source database can be used as the first incremental data, and the data in the target database within the first time period except the historical target incremental data can be used as the second incremental data.

[0059] It should be noted that when parsing the log information of the logical log and physical log of the source database, considering the impact of log records on database performance, you can consider using asynchronous methods to process log information, or use a dedicated log processing system to reduce the burden on the database.

[0060] In addition to real-time parsing of logical logs and physical logs, incremental data can be stored in specialized storage systems, such as non-relational databases or distributed storage systems, to improve the efficiency and accuracy of data processing.

[0061] In an embodiment of the present application, by obtaining the inconsistent historical target incremental data in the source database and the target database after performing incremental data verification on the source database and the target database at the first time, and then parsing the log information in the logical log and physical log of the source database within the first time period to obtain the third incremental data of the first time period, and the data in the source database within the first time period except the historical target incremental data are used as the first incremental data. In this way, when the first incremental data and the second incremental data are subsequently verified, the historical target incremental data that has been verified last time does not need to be verified, but only the incremental data in the first time period of the source database and the target database are verified, thereby improving the efficiency of incremental data verification.

[0062] Step 120 , for the incremental data of the target row in the first table in the first database, concatenate the incremental data of each column of the target row and convert them into string type data to obtain the string type data corresponding to the incremental data of the target row.

[0063] The first database may be a source database or a target database.

[0064] The first table may be every table in the first database.

[0065] For the incremental data of the target row, the character string data corresponding to the incremental data of the row may be data obtained by converting the incremental data of the row into a character string.

[0066] In some embodiments of the present application, for the incremental data of the target row in each table in each database in the source database and the target database, the incremental data of each column of the row can be spliced, and then the spliced ​​data can be converted into a string type, so that the string data corresponding to the incremental data of the row can be obtained.

[0067] In an example, the first database is a source database, and the first table is table 1 among multiple tables in the source database. Figure 2 , Figure 2 It is a schematic diagram of Table 1 in the source database. Table 1 has 4 rows and 5 columns. Incremental data is stored in the table. For each row of incremental data in Table 1, taking the first row of incremental data as an example, the incremental data of each column in the first row are spliced, that is, the first row of incremental data 3, 4, 2, 7, and 6 are spliced ​​to obtain the spliced ​​data, that is, the first row of incremental data is spliced ​​from 5 data into one data, that is to say, the first row of incremental data is changed from the original 5 columns of data to 1 column of data, and then the spliced ​​data is converted to a string type to obtain the string data corresponding to the first row of incremental data, that is, the string data is obtained.

[0068] In some embodiments of the present application, before the incremental data of each column are concatenated and converted into string type data, the incremental data in each table can be sorted, specifically, sorted according to the operation time of the incremental data. In this way, the incremental data can be compared according to the operation time of the incremental data to avoid missing incremental data and causing inaccurate comparison results.

[0069] Step 130 : Map the string data corresponding to the incremental data of each row to integer data of a fixed length to obtain the integer data corresponding to the incremental data of each row.

[0070] For each row of incremental data, the integer data corresponding to the row of incremental data may be data obtained by mapping the string data corresponding to the row of incremental data to integer data of a fixed length.

[0071] In some embodiments of the present application, when mapping the string data corresponding to each row of incremental data to integer data of a fixed length, a string matching algorithm can be used to map the string data corresponding to each row of incremental data to integer data of a fixed length. For example, a Rabin-Karp algorithm can be used to map the string data corresponding to each row of incremental data to integer data of a fixed length.

[0072] In some embodiments of the present application, in order to accurately obtain the integer data corresponding to each row of incremental data, step 130 may specifically include:

[0073] Perform hash calculation on each character in the string data corresponding to each row of incremental data to obtain the hash value of each character in the string data;

[0074] According to the hash value of each character in the string data, the integer data corresponding to the incremental data of each row is obtained.

[0075] In some embodiments of the present application, for each row of incremental data, a hash calculation can be performed on each character in the string data corresponding to the incremental data of that row to obtain a hash value for each character in the string data. Then, based on the hash value of each character in the string data, the integer data corresponding to each row of incremental data can be obtained according to the following formula (1).

[0076] R(S)=(S[0]×p ∧ (n-1)+S[1]×p ∧ (n-2)+…S[n-2]×p+S[n-1])%q (1)

[0077] In the above formula (1), ∧ represents a power operation, % represents a modulo operation, S[i] represents the hash value of the i-th character in the string data, p and q are both prime numbers, and q is greater than n.

[0078] It should be noted that there is no limit on the values ​​of p and q, as long as both are prime numbers and q is greater than n. In the actual calculation process, within the computing power, the larger the values ​​of p and q, the better.

[0079] Step 140: Calculate the integer data corresponding to each row of incremental data in each table of the source database by using the Merkle binary tree algorithm to obtain a first Merkle binary tree corresponding to the source database, and calculate the integer data corresponding to each row of incremental data in each table of the target database by using the Merkle binary tree algorithm to obtain a second Merkle binary tree corresponding to the target database.

[0080] The first Merkel binary tree may be obtained by calculating the integer data corresponding to each row of incremental data in each table of the source database through the Merkel binary tree algorithm to obtain a Merkel binary tree corresponding to the source database.

[0081] The second Merkle binary tree may be obtained by calculating the integer data corresponding to each row of incremental data in each table of the target database through the Merkle binary tree algorithm to obtain a Merkle binary tree corresponding to the target database.

[0082] In some embodiments of the present application, the integer data corresponding to each row of incremental data in each table of the source database can be obtained first, and then two adjacent integer data R(S) are calculated by the Merkle binary tree algorithm to obtain a leaf node R(S) value, and then the two adjacent leaf node R(S) values ​​are calculated to obtain the previous leaf node R(S) value of the leaf node R(S) value, and the root node value of the Merkle binary tree can be obtained by repeatedly calculating in this way, and then the first Merkle binary tree can be obtained. Correspondingly, the acquisition process of the second Merkle binary tree is consistent with the acquisition process of the first Merkle binary tree, which will not be repeated here.

[0083] In some embodiments of the present application, before step 140, the above method may further include:

[0084] The integer data corresponding to the incremental data of each row in each table of the first database is represented in a preset description format;

[0085] storing integer data represented in a preset description format into a table to obtain a second table;

[0086] Step 140 may specifically include:

[0087] Using the first identifier, respectively filter out first integer data belonging to the source database and second integer data belonging to the target database from each integer data in the second table;

[0088] The first integer data is calculated by the Merkle binary tree algorithm to obtain a first Merkle binary tree corresponding to the source database, and the second integer data is calculated by the Merkle binary tree algorithm to obtain a second Merkle binary tree corresponding to the target database.

[0089] Among them, the preset description format can be a description form of pre-set integer data, and the preset description format can include a first identifier, a second identifier and integer data, where the first identifier can be used to represent the database corresponding to the integer data, the second identifier can be used to represent the table corresponding to the integer data, and the position of the integer data in the corresponding table.

[0090] The second table may be a table obtained by describing the data in all tables according to a preset description format and then putting them into a table, that is, the second table contains a plurality of integer data described in the preset description format.

[0091] In one example, after each row of incremental data in the first table is mapped to integer data R(S), the integer data can be described in the form of {first identifier}|{second identifier}|{R(S)}, and then all integer data in the first table are described in the form of {first identifier}|{second identifier}|{R(S)} and stored in the form of a table to obtain a second table, that is, the second table has multiple integer data described in the form of {first identifier}|{second identifier}|{R(S)}.

[0092] In some embodiments of the present application, after obtaining the second table, the first integer data belonging to the source database and the second integer data belonging to the target database can be distinguished by using the first identifier from the integer data described in the form of multiple {first identifier}|{second identifier}|{R(S)} in the second table, and then the Merkel binary tree algorithm is used to calculate the first integer data to obtain a first Merkel binary tree corresponding to the source database, and the Merkel binary tree algorithm is used to calculate the second integer data to obtain a second Merkel binary tree corresponding to the target database.

[0093] In an embodiment of the present application, the integer data after mapping the incremental data in different databases is described in a preset description format and stored in a table. In this way, the database to which the integer data belongs can be quickly distinguished only by the first identifier in this table, and then the integer data in the source database and the target database can be compared, thereby improving the efficiency of obtaining the integer data in the source database and the target database.

[0094] In some embodiments of the present application, after obtaining the second table, the second table can be stored in a message queue, such as RocketMQ, and then the second table is obtained from RocketMQ, and then the data in the second table is processed. In this way, when certain data needs to be processed, it can be obtained from RocketMQ. When certain data does not need to be processed, it can be queued in RocketMQ, thereby avoiding confusion in the processing of data in the second table.

[0095] Step 150: Compare the first Merkle binary tree with the second Merkle binary tree to obtain a verification result between the first incremental data and the second incremental data.

[0096] In some embodiments of the present application, the verification result between the first incremental data and the second incremental data can be obtained by comparing the root node of the first Merkle binary tree with the root node of the second Merkle binary tree.

[0097] In some embodiments of the present application, in order to accurately obtain the verification result between the first incremental data and the second incremental data, step 150 may specifically include:

[0098] Compare the first root node value of the first Merkle binary tree with the second root node value of the second Merkle binary tree;

[0099] When it is determined that the first root node value and the second root node value are consistent, determining that a verification result between the first incremental data and the second incremental data is that the first incremental data and the second incremental data are consistent;

[0100] When it is determined that the first root node value and the second root node value are inconsistent, it is determined that the verification result between the first incremental data and the second incremental data is that the first incremental data and the second incremental data are inconsistent.

[0101] The first root node value may be the value of the root node of the first Merkle binary tree.

[0102] The second root node value may be the value of the root node of the second Merkle binary tree.

[0103] It should be noted that a Merkle binary tree has a root node, which represents the integrity of the data of any node in the Merkle binary tree. Multiple layers of leaf nodes are deployed below the root node. The bottom-level leaf nodes of the Merkle binary tree store integer data and the hash of the integer data, while each upper-level node stores the hash of two child nodes. Finally, the hash of the root node guarantees the integrity of the data of any node in the Merkle binary tree.

[0104] In some embodiments of the present application, the first root node value of the first Merkle binary tree and the second root node value of the second Merkle binary tree can be compared first. If the first root node value and the second root node value are consistent, it means that the integrity of the first Merkle binary tree and the second Merkle binary tree are consistent, so it can be determined that the verification result between the first incremental data and the second incremental data is that the first incremental data and the second incremental data are consistent.

[0105] If the first root node value is inconsistent with the second root node value, it means that the integrity of the first Merkle binary tree and the second Merkle binary tree is inconsistent, and it can be determined that the verification result between the first incremental data and the second incremental data is that the first incremental data and the second incremental data are inconsistent.

[0106] In an embodiment of the present application, by comparing the first root node value of the first Merkle binary tree with the second root node value of the second Merkle binary tree, the verification result between the first incremental data and the second incremental data can be quickly determined without comparing all node values ​​of the first Merkle binary tree and all nodes of the second Merkle binary tree, thereby improving the efficiency of determining the verification result between the first incremental data and the second incremental data.

[0107] In some embodiments of the present application, in order to further determine the verification result between the first incremental data and the second incremental data, when it is determined that the first root node value and the second root node value are inconsistent, determining that the verification result between the first incremental data and the second incremental data is that the first incremental data and the second incremental data are inconsistent may specifically include:

[0108] When it is determined that the first root node value and the second root node value are inconsistent, comparing the first leaf node of the first Merkle binary tree and the second leaf node of the second Merkle binary tree;

[0109] The inconsistent leaf nodes between the first leaf node and the second leaf node are used as inconsistent target incremental data in the first incremental data and the second incremental data.

[0110] Among them, the first leaf node can be a leaf node of the first Merkle binary tree.

[0111] The second leaf node may be a leaf node of the second Merkle binary tree.

[0112] The target incremental data may be the incremental data that is inconsistent between the first incremental data and the second incremental data.

[0113] In some embodiments of the present application, when it is determined that the first root node value and the second root node value are inconsistent, the first leaf node of the first Merkle binary tree and the second leaf node of the second Merkle binary tree may be compared again, so that the inconsistent leaf nodes can be selected out, and the inconsistent leaf nodes between the first leaf node and the second leaf node can be used as the inconsistent target incremental data in the first incremental data and the second incremental data.

[0114] In an embodiment of the present application, when it is determined that the first root node value and the second root node value are inconsistent, the first leaf node of the first Merkle binary tree and the second leaf node of the second Merkle binary tree are compared, and the inconsistent leaf nodes between the first leaf node and the second leaf node are used as the inconsistent target incremental data in the first incremental data and the second incremental data. In this way, the inconsistent target incremental data in the first incremental data and the second incremental data can be accurately determined.

[0115] In some embodiments of the present application, after obtaining the target incremental data, the above method may further include:

[0116] Output target incremental data.

[0117] In some embodiments of the present application, after the target incremental data is obtained, the target incremental data may be output and displayed to the user, so that the user can intuitively view the verification result between the first incremental data and the second incremental data.

[0118] In some embodiments of the present application, when comparing incremental data, in addition to using the method provided in the embodiments of the present application to perform comparisons, a data comparison method based on machine learning can also be developed to reduce the time and resources required for comparison.

[0119] In the prior art, the analysis of the logical log and physical log of the source database is performed in the target database. After the target database obtains the log, the recording and processing of the log may impose a certain burden on the performance of the target database.

[0120] In order to solve the above problems, the executor of the incremental data verification method in the embodiment of the present application can be an incremental data verification device, which can be independent of the source database and the target database, thereby reducing the performance burden of the target database.

[0121] In some embodiments of the present application, the incremental data verification device can be provided with a management unit, which can be developed through the SpringBoot framework and run on the server in the form of a jar. The management unit accurately monitors the status and anomalies of data verification, reports any potential problems in a timely manner, and also monitors the backlog of the message queue RocketMQ in real time to ensure the smoothness and efficiency of data processing. During the incremental verification process, the management unit needs to fully grasp the verification progress, record the verification speed and difference results in detail, and provide strong support for subsequent data analysis and optimization. In addition, the management unit also provides users with an intuitive, real-time visual monitoring interface, so that they can obtain the latest status and progress of data verification at any time. When anomalies or errors are found, the management unit can quickly start the reporting mechanism to assist in quickly locating problems and formulating effective solutions. The implementation of these functions not only improves the overall efficiency of data verification, but also provides a solid barrier to ensure data accuracy and integrity.

[0122] For the rapid location and resolution of exceptions and errors mentioned above, we can consider developing an automated exception handling system to reduce manual intervention and response time.

[0123] In some embodiments of the present application, when implementing the solutions of the embodiments of the present application, attention should be paid to the following issues:

[0124] 1. Data backup and recovery: Before performing incremental migration, you should ensure that the database is fully backed up to prevent data loss or corruption. At the same time, a detailed recovery plan should be developed so that data can be quickly restored in the event of a problem.

[0125] 2. Performance monitoring and tuning: During the implementation of this application proposal, the performance indicators of the database, such as CPU, memory, and disk usage, should be closely monitored. Once performance bottlenecks or problems are found, they should be tuned or adjusted in a timely manner.

[0126] 3. Testing and verification: Before formal implementation, this application proposal should be fully tested and verified to ensure that its functions and performance meet expectations. At the same time, various possible abnormal situations and edge cases should be considered and tested accordingly.

[0127] 4. Permissions and access control: To ensure data security, access rights to the database should be strictly controlled. Only authorized personnel can access and operate the database, and a multi-level identity authentication and audit mechanism should be implemented.

[0128] 5. Continuous monitoring and iterative optimization: After completing the incremental migration, the consistency and integrity of the data should be continuously monitored, and iterative optimization and adjustment should be performed according to the actual situation. At the same time, the database security strategy and technical architecture should be regularly reviewed and updated.

[0129] In order to more clearly understand the solution of the embodiment of the present application, the incremental data verification method provided by the embodiment of the present application is discussed as a whole below, referring to Figure 3 , Figure 3 The following is a flow chart of an incremental data verification method. The incremental data verification method may specifically include:

[0130] First, the logical log and the physical log are obtained from the source database. Specifically, the logical log and the physical log within the first time period are obtained, and then the logical log and the physical log are parsed to obtain the third incremental data. According to the third incremental data, the first incremental data of the source database and the second incremental data of the target database are obtained. Then, the first incremental data and the second incremental data are respectively stored in at least one table of their corresponding databases, and the incremental data in the table are sorted. Then, for each row of incremental data in each table, each column of the row of incremental data is concatenated and converted into string data. Then, the string data is mapped to integer data, and the integer data is stored in RocketMQ. Then, the first integer data of the source database and the second integer data of the target database are obtained from RocketMQ, and the first integer data and the second integer data of the target database are verified by the Merkel binary tree method to obtain the verification result, which is displayed, and the result file is saved. Specifically, the target incremental data is stored in the result file.

[0131] It should be noted that the incremental data verification method provided in the embodiment of the present application can be executed by an incremental data verification device, or a control module in the incremental data verification device for executing the incremental data verification method.

[0132] Based on the same inventive concept as the above-mentioned incremental data verification method, the present application also provides an incremental data verification device. Figure 4 The incremental data verification device provided in the embodiment of the present application is described in detail.

[0133] Figure 4 It is a schematic structural diagram of an incremental data verification device according to an exemplary embodiment.

[0134] like Figure 4 As shown, the incremental data verification device 400 may include:

[0135] A first acquisition module 410 is configured to, when it is determined that migration and verification of the full data from the source database to the target database are completed, acquire first incremental data of the source database within a first time period and second incremental data of the target database within the first time period, wherein the first incremental data and the second incremental data are respectively stored in at least one table in their corresponding databases, the first time period is a time period between the current time and the first time, and the first time is a time when the latest incremental data verification is completed from the current time;

[0136] A first determination module 420 is configured to concatenate the incremental data of each column of a target row in a first table in a first database and convert the concatenated incremental data into string type data to obtain string type data corresponding to the incremental data of the target row, wherein the first database is the source database and the target database, the first table is each table in the first database, the target row is each row of the first table, and when the first database is the source database, the incremental data is the first incremental data, and when the first database is the target database, the incremental data is the second incremental data;

[0137] The second determination module 430 is used to map the string data corresponding to the incremental data of each row into integer data of a fixed length to obtain the integer data corresponding to the incremental data of each row;

[0138] A third determination module 440 is used to calculate the integer data corresponding to each row of incremental data in each table of the source database by using a Merkel binary tree algorithm to obtain a first Merkel binary tree corresponding to the source database, and to calculate the integer data corresponding to each row of incremental data in each table of the target database by using the Merkel binary tree algorithm to obtain a second Merkel binary tree corresponding to the target database;

[0139] The fourth determination module 450 is used to compare the first Merkle binary tree with the second Merkle binary tree to obtain a verification result between the first incremental data and the second incremental data.

[0140] In an embodiment of the present application, when it is determined that the migration and verification of the full amount of data from the source database to the target database is completed, the first incremental data of the source database in the first time period and the second incremental data of the target database in the first time period are obtained, and then the first incremental data and the second incremental data are verified. In this way, when verifying the incremental data, only the incremental data from the last verification to the current time period is verified, rather than verifying all the incremental data. This reduces the time for verification during the migration of incremental data and improves the efficiency of incremental data verification. In addition, when verifying the first incremental data and the second incremental data, the incremental data of each column of the row is concatenated and converted into string type data for each row of incremental data in each table in each database in the source database and the target database, and the string type data corresponding to the incremental data of each row is obtained. In this way, when the incremental data in the source database and the target database are subsequently compared, each row in each table is only compared once, and there is no need to compare multiple times, which further improves the efficiency of incremental data verification. Then, the string type data corresponding to the incremental data of each row is mapped to integer type data of fixed length to obtain the integer type data corresponding to the incremental data of each row, and the Merkel binary tree is used to obtain the integer type data corresponding to the incremental data of each row. The algorithm calculates the integer data corresponding to each row of incremental data in each table of the source database to obtain the first Merkel binary tree corresponding to the source database, and calculates the integer data corresponding to each row of incremental data in each table of the target database through the Merkel binary tree algorithm to obtain the second Merkel binary tree corresponding to the target database. The first Merkel binary tree and the second Merkel binary tree are compared to obtain the verification result between the first incremental data and the second incremental data. In this way, the first incremental data and the second incremental data are converted, mapped and compared into string types, thereby ensuring the consistency of the first incremental data and the second incremental data and improving the verification accuracy of the first incremental data and the second incremental data.

[0141] In some embodiments of the present application, the fourth determining module 450 may specifically include:

[0142] A first comparing unit, configured to compare a first root node value of the first Merkle binary tree with a second root node value of the second Merkle binary tree;

[0143] a first determining unit, configured to determine, when determining that the first root node value and the second root node value are consistent, that a verification result between the first incremental data and the second incremental data is that the first incremental data and the second incremental data are consistent;

[0144] A second determination unit is configured to determine, when it is determined that the first root node value and the second root node value are inconsistent, that a verification result between the first incremental data and the second incremental data is that the first incremental data and the second incremental data are inconsistent.

[0145] In some embodiments of the present application, the second determining unit is specifically configured to:

[0146] When it is determined that the first root node value and the second root node value are inconsistent, comparing the first leaf node of the first Merkle binary tree and the second leaf node of the second Merkle binary tree;

[0147] The inconsistent leaf nodes between the first leaf node and the second leaf node are used as inconsistent target incremental data between the first incremental data and the second incremental data.

[0148] In some embodiments of the present application, the first acquisition module 410 may specifically include:

[0149] A first acquisition unit is used to acquire historical target incremental data, wherein the historical target incremental data is inconsistent data between the source database and the target database obtained after performing incremental data verification on the source database and the target database at the first time;

[0150] A first parsing unit is used to parse the log information in the first time period in the logical log and the physical log of the source database to obtain third incremental data in the first time period;

[0151] A third determining unit is configured to use the third incremental data and the first data in the source database as the first incremental data, wherein the first data is data in the source database within the first time period except the historical target incremental data;

[0152] The fourth determining unit is configured to use the data in the target database within the first time period except the historical target incremental data as the second incremental data.

[0153] In some embodiments of the present application, the second determining module 430 is specifically used to:

[0154] Performing hash calculation on each character in the string data corresponding to each row of incremental data to obtain a hash value of each character in the string data;

[0155] According to the hash value of each character in the string data, the integer data corresponding to the incremental data of each row is obtained.

[0156] In some embodiments of the present application, the above-mentioned device may further include:

[0157] A preset description unit, used to represent the integer data corresponding to the incremental data of each row in each table of the first database in a preset description format, wherein the preset description format includes a first identifier, a second identifier and the integer data, the first identifier is used to represent the database corresponding to the integer data, the second identifier is used to represent the table corresponding to the integer data, and the position of the integer data in the corresponding table;

[0158] A fifth determining unit, configured to store the integer data represented in the preset description format into a table to obtain a second table;

[0159] The third determination module 440 is specifically used for:

[0160] Using the first identifier, respectively filter out first integer data belonging to the source database and second integer data belonging to the target database from each of the integer data in the second table;

[0161] The first integer data is calculated by a Merkle binary tree algorithm to obtain a first Merkle binary tree corresponding to the source database, and the second integer data is calculated by the Merkle binary tree algorithm to obtain a second Merkle binary tree corresponding to the target database.

[0162] The incremental data verification device provided in the embodiment of the present application can be used to execute the incremental data verification method provided in the above-mentioned method embodiments. Its implementation principle and technical effect are similar, and for the sake of simplicity, they will not be repeated here.

[0163] Based on the same inventive concept, an embodiment of the present application also provides an electronic device.

[0164] Figure 5 Schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device may include a processor 501 and a memory 502 storing computer programs or instructions.

[0165] Specifically, the processor 501 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0166] The memory 502 may include a large capacity memory for data or instructions. For example, but not limitation, the memory 502 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 502 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 502 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 502 is a non-volatile solid-state memory. The memory may include a read-only memory (ROM), a random access memory (RAM), a disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described in the incremental data verification method provided in the above-mentioned embodiments.

[0167] The processor 501 implements any one of the incremental data verification methods in the above embodiments by reading and executing computer program instructions stored in the memory 502 .

[0168] In one example, the electronic device may further include a communication interface 503 and a bus 510. Figure 5 As shown, the processor 501, the memory 502, and the communication interface 503 are connected via a bus 510 and communicate with each other.

[0169] The communication interface 503 is mainly used to implement the communication between the modules, devices, units and / or devices in the embodiment of the present invention.

[0170] Bus 510 includes hardware, software or both, and the parts of electronic equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 510 may include one or more buses. Although the embodiment of the present invention describes and shows a specific bus, the present invention considers any suitable bus or interconnection.

[0171] The electronic device can execute the incremental data verification method in the embodiment of the present invention, thereby achieving Figure 1 Describes the incremental data verification method.

[0172] In addition, in combination with the incremental data verification method in the above embodiment, the embodiment of the present invention can provide a readable storage medium to implement. The readable storage medium stores program instructions, and when the program instructions are executed by the processor, any incremental data verification method in the above embodiment is implemented.

[0173] In addition, in combination with the incremental data verification method in the above embodiment, an embodiment of the present invention may provide a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes any one of the incremental data verification methods in the above embodiment.

[0174] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0175] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0176] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0177] The above reference is according to the method of the embodiment of the present application, the flow chart of the device (system) and the computer program product and / or the block diagram described various aspects of the present application.It should be understood that each square box in the flow chart and / or the block diagram and the combination of each square box in the flow chart and / or the block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the realization of the function / action specified in one or more square boxes of the flow chart and / or the block diagram.Such a processor can be but is not limited to a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit.It can also be understood that each square box in the block diagram and / or the flow chart and the combination of the square boxes in the block diagram and / or the flow chart can also be realized by the dedicated hardware that performs the specified function or action, or can be realized by the combination of dedicated hardware and computer instructions.

[0178] The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system, module and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.

Claims

1. A method for verifying incremental data, characterized in that: The method is applied to an incremental data verification device, and the method comprises: In the case of determining that the migration and verification of the full data from the source database to the target database are completed, obtaining first incremental data of the source database within a first time period and second incremental data of the target database within the first time period, wherein the first incremental data and the second incremental data are respectively stored in at least one table in their corresponding databases, the first time period is a time period between the current time and the first time, and the first time is the time when the verification of the incremental data is most recently completed from the current time; For the incremental data of a target row in a first table in a first database, concatenate the incremental data of each column of the target row and convert them into string type data to obtain string type data corresponding to the incremental data of the target row, wherein the first database is the source database and the target database, the first table is each table in the first database, the target row is each row of the first table, when the first database is the source database, the incremental data is the first incremental data, and when the first database is the target database, the incremental data is the second incremental data; Mapping the string data corresponding to the incremental data of each row into integer data of a fixed length to obtain the integer data corresponding to the incremental data of each row; The integer data corresponding to each row of incremental data in each table of the source database is calculated by a Merkle binary tree algorithm to obtain a first Merkle binary tree corresponding to the source database, and the integer data corresponding to each row of incremental data in each table of the target database is calculated by the Merkle binary tree algorithm to obtain a second Merkle binary tree corresponding to the target database; The first Merkle binary tree is compared with the second Merkle binary tree to obtain a verification result between the first incremental data and the second incremental data.

2. The method according to claim 1, characterized in that The comparing the first Merkle binary tree with the second Merkle binary tree to obtain a verification result between the first incremental data and the second incremental data includes: Compare the first root node value of the first Merkle binary tree with the second root node value of the second Merkle binary tree; In a case where it is determined that the first root node value and the second root node value are consistent, determining that a verification result between the first incremental data and the second incremental data is that the first incremental data and the second incremental data are consistent; In the case where it is determined that the first root node value and the second root node value are inconsistent, it is determined that a verification result between the first incremental data and the second incremental data is that the first incremental data and the second incremental data are inconsistent.

3. The method according to claim 2, characterized in that The step of determining, when it is determined that the first root node value and the second root node value are inconsistent, that a verification result between the first incremental data and the second incremental data is that the first incremental data and the second incremental data are inconsistent comprises: When it is determined that the first root node value and the second root node value are inconsistent, comparing the first leaf node of the first Merkle binary tree and the second leaf node of the second Merkle binary tree; The inconsistent leaf nodes between the first leaf node and the second leaf node are used as inconsistent target incremental data between the first incremental data and the second incremental data.

4. The method according to claim 3, characterized in that The acquiring first incremental data of the source database within a first time period and second incremental data of the target database within the first time period includes: Acquire historical target incremental data, wherein the historical target incremental data is inconsistent data between the source database and the target database obtained after performing incremental data verification on the source database and the target database at the first time; Parsing the log information in the first time period in the logical log and the physical log of the source database to obtain third incremental data of the first time period; The third incremental data and the first data in the source database are used as the first incremental data, wherein the first data is the data in the source database within the first time period except the historical target incremental data; The data in the target database within the first time period except the historical target incremental data is used as the second incremental data.

5. The method according to claim 1, characterized in that The step of mapping the string data corresponding to the incremental data of each row into integer data of a fixed length to obtain the integer data corresponding to the incremental data of each row includes: Performing hash calculation on each character in the string data corresponding to each row of incremental data to obtain a hash value of each character in the string data; According to the hash value of each character in the string data, the integer data corresponding to the incremental data of each row is obtained.

6. The method according to claim 1, characterized in that After obtaining the integer data corresponding to the incremental data of each row, the method further includes: The integer data corresponding to the incremental data of each row in each table of the first database is represented in a preset description format, wherein the preset description format includes a first identifier, a second identifier and the integer data, the first identifier is used to represent the database corresponding to the integer data, the second identifier is used to represent the table corresponding to the integer data, and the position of the integer data in the corresponding table; The integer data represented in the preset description format is stored in a table to obtain a second table; The step of calculating the integer data corresponding to each row of incremental data in each table of the source database by using the Merkel binary tree algorithm to obtain a first Merkel binary tree corresponding to the source database, and calculating the integer data corresponding to each row of incremental data in each table of the target database by using the Merkel binary tree algorithm to obtain a second Merkel binary tree corresponding to the target database includes: Using the first identifier, respectively filter out first integer data belonging to the source database and second integer data belonging to the target database from each of the integer data in the second table; The first integer data is calculated by a Merkle binary tree algorithm to obtain a first Merkle binary tree corresponding to the source database, and the second integer data is calculated by the Merkle binary tree algorithm to obtain a second Merkle binary tree corresponding to the target database.

7. An incremental data verification device, characterized in that: The device comprises: A first acquisition module is used to acquire, when it is determined that the migration and verification of the full data from the source database to the target database are completed, first incremental data of the source database within a first time period and second incremental data of the target database within the first time period, wherein the first incremental data and the second incremental data are respectively stored in at least one table in their corresponding databases, the first time period is a time period between the current time and the first time, and the first time is the time when the verification of the incremental data is most recently completed from the current time; A first determination module is used for concatenating the incremental data of each column of a target row in a first table in a first database and converting the concatenated incremental data into string type data to obtain string type data corresponding to the incremental data of the target row, wherein the first database is the source database and the target database, the first table is each table in the first database, the target row is each row of the first table, when the first database is the source database, the incremental data is the first incremental data, and when the first database is the target database, the incremental data is the second incremental data; A second determination module is used to map the character string data corresponding to the incremental data of each row into integer data of a fixed length to obtain the integer data corresponding to the incremental data of each row; A third determination module is used to calculate the integer data corresponding to each row of incremental data in each table of the source database by using a Merkel binary tree algorithm to obtain a first Merkel binary tree corresponding to the source database, and to calculate the integer data corresponding to each row of incremental data in each table of the target database by using the Merkel binary tree algorithm to obtain a second Merkel binary tree corresponding to the target database; The fourth determination module is used to compare the first Merkle binary tree with the second Merkle binary tree to obtain a verification result between the first incremental data and the second incremental data.

8. An electronic device, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the incremental data verification method as described in any one of claims 1 to 6.

9. A readable storage medium, characterized in that: The readable storage medium stores programs or instructions, and when the programs or instructions are executed by the processor, the steps of the incremental data verification method as described in any one of claims 1-6 are implemented.

10. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the steps of the incremental data verification method as described in any one of claims 1-6.

Images