Data verification method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention provides a data verification method and device, equipment, a storage medium and a program product. The method comprises the steps of pausing data storage of a target database after a data verification trigger event is detected, and obtaining a first transaction stored in a source database; after the first transaction is stored in the source database, creating a first snapshot table of the source database, and obtaining a second transaction stored in the source database after the first snapshot table is created; restarting data storage of the target database, pausing data storage of the target database when a second transaction stored in the target database is detected, and creating a second snapshot table in the target database; and generating a data verification result based on the first snapshot table, the second snapshot table, the first transaction and the second transaction. The full data of the snapshot table is in the same stage on the premise that the source database is not blocked, and the verification accuracy of final data verification is higher.

Description

Technical Field

[0001] This application relates to the field of data processing, and in particular, to a data verification method, apparatus, device, storage medium, and program product. Background Art

[0002] To meet the requirements of different application scenarios, an application system may include multiple databases, and the data of each database needs to be consistent to achieve specific functions, such as data backup, data sharing, etc. Data synchronization is used to ensure the consistency of data in different databases. During the data synchronization process, it is necessary to regularly verify the synchronized data, that is, to verify whether the data in the source database is consistent with the data synchronized to the target database, and determine whether data is lost or incorrect during the synchronization process.

[0003] However, during the data synchronization process, since the source database continuously obtains or stores new transactions, the data in the source database and the data in the target database cannot be in the same stage. For example, at the current moment, there is new data in the source database, but the target database has not yet synchronized this new data. If data verification is performed at this moment, the verification result will be inaccurate.

[0004] Currently, to address the above problems, the method of blocking the source database is usually used, that is, not obtaining and storing new transactions. After all the transactions in the source database are synchronized to the target database, snapshot tables are generated at both ends for verification or directly verified. By blocking the source database, it is ensured that the data in the source database and the data in the target database are in the same stage, ensuring data consistency, and then verification is performed. However, blocking the source database will affect the operation of the source database, which may cause system service interruption or delay. Summary of the Invention

[0005] Embodiments of this application provide a data verification method, apparatus, device, storage medium, and program product. Without blocking the source database, by determining the transactions recorded before and after the generated snapshot table, the remaining data in the snapshot table is in the same stage, and synchronous verification is achieved based on the comparison result of the data in the same stage, with high verification accuracy.

[0006] In a first aspect, an embodiment of the present application provides a data verification method, including: after detecting a data verification trigger event, pausing the data entry of the target database, and obtaining a first transaction stored in the source database; a transaction is an event for adding, deleting, or modifying data in the source database; the first transaction is the transaction with the latest occurrence time stored in the source database when the data verification trigger event is detected; after storing the first transaction in the source database, creating a first snapshot table of the source database, and obtaining a second transaction stored in the source database after creating the first snapshot table; restarting the data entry of the target database, and when detecting the second transaction stored in the target database, pausing the data entry of the target database, and creating a second snapshot table in the target database; generating a data verification result based on the first snapshot table, the second snapshot table, the first transaction, and the second transaction.

[0007] In a possible implementation manner, generating a data verification result based on the first snapshot table, the second snapshot table, the first transaction, and the second transaction includes: generating a preliminary data verification result based on the first snapshot table and the second snapshot table; determining the data that has changed in the source database between the first transaction and the second transaction as incremental data; generating a data verification result based on the preliminary data verification result and the incremental data.

[0008] In a possible implementation manner, the preliminary data verification result is used to indicate whether the data items in the first snapshot table and the second snapshot table are consistent; generating a data verification result based on the preliminary data verification result and the incremental data includes: determining whether the inconsistent data in the preliminary data verification result belongs to the incremental data; if so, deleting the inconsistent data in the preliminary data verification result to obtain the data verification result.

[0009] In a possible implementation manner, generating a data verification result based on the preliminary data verification result and the incremental data includes: comparing the incremental data with the corresponding data in the second snapshot table to generate an incremental verification result; updating the preliminary data verification result of the incremental data in the preliminary data verification result to the incremental verification result to obtain the data verification result.

[0010] In a possible implementation manner, generating a data verification result based on the preliminary data verification result and the incremental data includes: comparing the incremental data with the corresponding data in the second snapshot table to generate an incremental verification result; if the inconsistent data in the preliminary data verification result belongs to the incremental data, updating the preliminary data verification result of the incremental data in the preliminary data verification result to the incremental verification result to obtain the data verification result; if the inconsistent data in the preliminary data verification result does not belong to the incremental data, determining the preliminary data verification result as the data verification result.

[0011] In a possible implementation, obtaining a second transaction after creating a first snapshot table includes: after creating a first snapshot table of a source server, obtaining a log number of the first snapshot table; determining a transaction whose corresponding log number is greater than or equal to the log number of the first snapshot table as the second transaction.

[0012] In a second aspect, an embodiment of the present application provides a data verification device, including: a first transaction obtaining module, configured to pause data entry into a target database after detecting a data verification trigger event, and obtain a first transaction stored in a source database, where the transaction is an event for adding, deleting, or modifying data in the source database, and the first transaction is the transaction with the latest occurrence time stored in the source database when the data verification trigger event is detected; a first snapshot table creating module, configured to create a first snapshot table of the source database after storing the first transaction in the source database, and obtain a second transaction stored in the source database after creating the first snapshot table; a second snapshot table creating module, configured to restart data entry into the target database, pause data entry into the target database when detecting the second transaction stored in the target database, and create a second snapshot table in the target database; a data verification module, configured to generate a data verification result based on the first snapshot table, the second snapshot table, the first transaction, and the second transaction.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, so that the processor executes the method provided in the first aspect above.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored, and when the computer execution instructions are executed by a processor, they are used to implement the method provided in the first aspect above.

[0015] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the method provided in the first aspect above.

[0016] The data verification method, device, equipment, storage medium and program product provided by the embodiments of the present application provide a data verification strategy without blocking the source database. Specifically, when performing data verification, the synchronization service of the target database is paused, that is, the data entry of the target database is paused, and the transaction with the latest occurrence time stored in the source database is obtained and recorded as the first transaction; then, a first snapshot table is created in the source database. When the snapshot table is created, the source database can continuously store new transactions, and the second transaction stored in the source database after the creation of the first snapshot table is obtained; the synchronization service of the target database is restarted. When the target database synchronizes to the second transaction, the synchronization service of the target database is paused, and a second snapshot table is created in the target database; finally, based on the data recorded in the first snapshot table and the second snapshot table and the first transaction and the second transaction, data synchronization verification between the target database and the source database is realized, and a data verification result is generated. Based on the transactions stored before and after the created snapshot tables, the data in the snapshot tables is processed, so that the full amount of data in the two snapshot tables is in the same stage. Using the comparison result of the full amount of data in the same stage to realize data synchronization verification overcomes the problem of inaccurate data verification caused by the synchronization data not being in the same stage and improves the accuracy of data verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present application and, together with the specification, are used to explain the principles of the present application.

[0018] Figure 1 It is a schematic diagram of data synchronization provided by an embodiment of the present application;

[0019] Figure 2 It is a schematic flowchart of a data verification method provided by an embodiment of the present application;

[0020] Figure 3 It is a schematic diagram of database data parsing provided by an embodiment of the present application;

[0021] Figure 4 It is a schematic diagram of creating a snapshot table provided by an embodiment of the present application;

[0022] Figure 5 It is a schematic diagram of data verification provided by an embodiment of the present application;

[0023] Figure 6 It is a schematic diagram of a method for determining an incremental verification result provided by an embodiment of the present application;

[0024] Figure 7 It is a schematic flowchart of another data verification method provided by an embodiment of the present application;

[0025] Figure 8Structural schematic diagram of a data verification device provided by an embodiment of the present application;

[0026] Figure 9 Structural schematic diagram of an electronic device provided by an embodiment of the present application.

[0027] Through the above-mentioned drawings, specific embodiments of the present application have been shown, and there will be more detailed descriptions hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Specific embodiments

[0028] Here, exemplary embodiments will be described in detail, and their examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0029] With the development of information technology, a system can include multiple databases, and the data in these databases often needs to be kept accurate and consistent to achieve specific functions, such as data backup, data sharing, etc. Data synchronization can maintain the consistency and coordination of data between multiple systems or databases to ensure that the data in all systems or databases is consistent, reduce data conflicts and redundancy, and improve data quality.

[0030] Figure 1 Schematic diagram of data synchronization provided by an embodiment of the present application. As Figure 1 shown, data synchronization is mainly divided into three stages. The first stage is the initialization loading of data, storing transactions into the source database, and making additions, deletions, and modifications to the data in the source database according to the transactions, while obtaining the basic point of data synchronization; the second stage is to perform data synchronization based on the basic point, that is, synchronizing the data in the source database to the target database; the third stage is to periodically verify the data, that is, comparing and verifying the data in the source database and the data in the target database to confirm whether data is lost during data synchronization.

[0031] When performing data synchronization in the second stage, by parsing the database log, obtaining the events of adding, deleting, and modifying the data in the source database, converting the events into a specific message format inside the synchronization software in units of transactions, sending them to the synchronization software of the target database through the data synchronization software, and finally the synchronization software of the target database restoring the events in units of transactions into statements supported by the target database and executing them on the target database, ultimately achieving real-time data synchronization and making the data in the source database and the target database consistent.

[0032] During data verification in the third stage, the data of the source database and the full data of the target database are obtained respectively, and it is verified whether the data of the source database and the full data of the target database are consistent. If they are consistent, it means that the data is successfully synchronized; if they are inconsistent, it means that an error occurs during the data synchronization process. Among them, the full data refers to all the data stored in the entire database at a specific time point or during a specific stage.

[0033] Combined with the above scenario, since the first stage is continuously running, that is, continuously storing the incremental data of the transaction into the source database. The incremental data refers to the data that has changed during a certain time period, and the incremental data of the transaction refers to the data that has changed during the process of the transaction. If the full data of the source database and the full data of the target database are obtained at a certain moment, there may be incremental data of the transaction that has been stored in the source database but not synchronized to the target database in the full data of the source database, resulting in the full data of the source database and the full data of the target database not being in the same stage. The same stage means that the status or version of each piece of data in different databases is consistent, that is, the final content of each piece of data at this node is consistent. For example, the last transaction stored or entered in the two databases is the same transaction. If the data is not in the same stage and the verification results are inconsistent, there are two reasons. One is that an error occurs during the data synchronization process, resulting in inconsistent data between the source database and the target database. The other is that due to the data not being in the same stage, there is some incremental data in one database that is more than that in the other database. Eventually, the data verification result is inaccurate.

[0034] Exemplarily, continue to refer to Figure 1, in the first stage, 2 transactions were loaded into the source database, namely Transaction 1 and Transaction 2. The incremental data of Transaction 1 is Data 1, and the incremental data of Transaction 2 is Data 2. In data synchronization, the synchronization is performed in the order of transaction loading. If verification is performed at the moment when Transaction 1 is synchronized to the target database while Transaction 2 is not synchronized, the data in the source database is the data after Transaction 2 occurred, that is, including Data 1 and Data 2. And when there is no error in data synchronization, the full amount of data in the target database only contains the incremental data of Transaction 1, that is, Data 1. Therefore, the full amount of data in the source database and the full amount of data in the target database are not in the same stage, and the verification results are still inconsistent when there is no error in data synchronization. If verification is performed at the moment when Transaction 2 is synchronized to the target database, but if Transaction 3 is loaded into the source database before verification, then the full amount of data in the source database includes the incremental data of Transaction 3 in addition to Data 1 and Data 2, and it is still not in the same stage as the full amount of data in the target database, and the verification results are still inconsistent when there is no error in data synchronization. If verification is performed when Transaction 2 is synchronized to the target database and no other transactions are loaded into the source database before verification, then when there is no error in data synchronization, the full amount of data in both the source database and the target database is Data 1 and Data 2, in the same stage, and the verification is consistent.

[0035] Regarding the above problems, the currently common data verification strategy is to block the first stage of the source database, that is, not to obtain and store new transactions or incremental data. After all the transactions in the source database are synchronized to the target database, snapshot tables are generated at both ends for verification or verification is directly performed. By blocking the source database, it is ensured that the full amount of data in the source database and the full amount of data in the target database are in the same stage. At this time, when the verification results are inconsistent, it is only caused by errors in the data synchronization process.

[0036] However, blocking the source database from obtaining and storing new transactions or incremental data will affect the operation of the source database, and thus may affect the operation of the entire system. For example, it may cause delays in data writing, and thus may cause delays in system response. Or, if the data is in the process of being written when the source database is blocked, it may cause data loss and system service interruption.

[0037] An embodiment of the present application provides a data verification method. For the data verification process, without blocking the source database, snapshot tables are created in the source database and the target database respectively, and the transactions before and after the snapshot tables are obtained. The incremental data between the transactions before and after the snapshot tables includes the data corresponding to different stages. By analyzing the snapshot tables of the source database and the target database and the transactions before and after the snapshot tables, the data of the source database and the target database in the same stage can be obtained, and then the data verification result can be determined. The verification result only represents whether there is an error in data synchronization. It realizes data verification without blocking the source database, improves the accuracy of data verification, and avoids blocking the database and thus affecting the operation of the system.

[0038] The following specific embodiments will be used to describe in detail the technical solutions of the present application and how the technical solutions of the present application solve the above technical problems. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0039] Figure 2 It is a schematic flowchart of a data verification method provided by an embodiment of the present application. The data verification method provided in this embodiment can be executed by a device in the system with corresponding data processing capabilities, which can be a device specifically used for data verification. As Figure 2 shown, the data verification method provided in this embodiment includes the following steps:

[0040] Step S201: After detecting a data verification trigger event, suspend the data entry of the target database and obtain the first transaction stored in the source database.

[0041] The source database refers to the database that is the initial source of data during the data synchronization process, that is, data changes (such as addition, deletion, and modification) occur in the source database. Data synchronization refers to copying and transmitting data from the source database to another database, where the other database is the target database.

[0042] A transaction is an event used to add, delete, or modify data in the source database. It can be an event of adding data to the source database, deleting data from the source database, modifying or updating data in the source database, or any combination of events of adding, deleting, and modifying data in the source database. One transaction can correspond to multiple events for adding, deleting, or modifying data in the source database.

[0043] Figure 3 It is a schematic diagram of database data parsing provided by an embodiment of the present application. As Figure 3As shown in the figure, when data synchronization is performed in the second stage, by parsing the database log of the source database used to record data changes in the database, events of adding, deleting, or modifying data in the source database are obtained and encapsulated into a transaction. The changed data corresponding to this transaction is the incremental data of this transaction. Each parsed transaction is cached in the transaction queue according to the occurrence time, such as Figure 3 Transactions SEQ-1 to SEQ-Y in

[0044] Exemplarily, assume that the full amount of data in the source database is Data 1 and Data 2. At a certain moment, Data 3 is inserted into the source database and Data 2 is deleted. This event is stored as a transaction in the database log of the source database. During data synchronization, through the database log, inserting Data 3 and deleting Data 2 can be recorded as a transaction, and this transaction is synchronized to the target database to insert Data 3 and delete Data 2 in the target database, completing the synchronization of the incremental data (Data 3 and Data 2) of this transaction.

[0045] The first transaction is the transaction with the latest occurrence time when a data verification trigger event is detected. The occurrence time refers to the time when the incremental data of this transaction is stored or takes effect in the database. Specifically, the latest event of adding, deleting, or modifying data in the source database parsed from the source database when a data verification trigger event is detected is the first transaction, that is, the last transaction in the transaction queue when a data verification trigger event is detected. Exemplarily, when parsing the source database, each transaction can be bound with a serial number, and this serial number increases with the occurrence time of the incremental data of this transaction in the source database. This serial number can represent the order in which the transaction is synchronized. Since the order in which the transaction is synchronized is the same as the order in which the incremental data of the transaction occurs in the source database, the transaction with the largest serial number is the first transaction.

[0046] It is also possible to determine that the transaction with the largest log number or the latest log generation time of the database log corresponding to the transaction is the first transaction by obtaining the log number or log generation time of the database log corresponding to the transaction.

[0047] A data verification trigger event refers to an event that automatically performs data verification when the event occurs. Data verification can be performed periodically. The data verification trigger event can be when a preset data verification time is reached. For example, the system is set to perform data verification at a cycle of time T. Every time time T elapses, it is considered that a data verification trigger event occurs. The data verification trigger event can also be when the system receives a control instruction. For example, a user selects the data verification function on the user terminal, and the user terminal sends a control instruction to the system. After the system detects the control instruction, it performs data verification.

[0048] Specifically, after detecting the data verification trigger event, suspend the data entry of the target database, that is, suspend synchronizing transactions to the target database, and obtain the first transaction with the latest occurrence time. Exemplarily, the transaction is bound with a serial number, and the transaction with the largest corresponding serial number in the transaction queue is determined as the first transaction.

[0049] Step S202, after storing the first transaction in the source database, create a first snapshot table of the source database, and obtain the second transaction stored in the source database after creating the first snapshot table.

[0050] A snapshot table is a table or database table for storing data, used to record and capture the full amount of data stored in the database at a certain moment. The first snapshot table is a snapshot table that records the full amount of data in the source database. Among them, the second snapshot table is a snapshot table that records the full amount of data in the target database.

[0051] Different databases can create snapshot tables using the characteristics of their own databases. For example, the Oracle database can use flashback queries or serialized transactions to generate snapshot tables; the KingBase database can use database snapshots or repeatable read transactions as snapshot tables; the Mysql database can use repeatable read transactions as snapshot tables; the SQLserver database can use repeatable read transactions as snapshot tables, etc. Creating a snapshot table does not block the source database, that is, creating a snapshot table and adding, deleting, or modifying data in the source database are in a parallel state.

[0052] Specifically, after storing the first transaction in the source database, create a first snapshot table. The data recorded in the first snapshot table includes the full amount of data in the source database, and after creating the first snapshot table, obtain the second transaction stored in the source database after creating the first snapshot table.

[0053] The second transaction is the transaction corresponding to the event of adding, deleting, or modifying data in the source database after creating the first snapshot table.

[0054] The acquisition method of the second transaction can be to determine, through the transaction queue after creating the first snapshot table, a transaction in the transaction queue whose occurrence time in the source database is later than the creation time of the first snapshot table as the second transaction. Specifically, the last transaction in the transaction queue can be determined as the second transaction after creating the first snapshot table, or the database log of the source database can be read to obtain the log generation time corresponding to each transaction, and the transaction whose corresponding log generation time is later than the creation time of the first snapshot table is determined as the second transaction.

[0055] In addition to the data of the source database before the first transaction, the data recorded in the first snapshot table may also include other incremental data. Figure 4 A schematic diagram of creating a snapshot table provided by an embodiment of the present application is as follows. Figure 4 As shown in the figure, since creating the snapshot table and adding, deleting, or modifying the data in the source database are in a parallel state, during the creation process of the first snapshot table (snapshot table t1), after the first transaction (transaction SEQ-X), the source database may undergo addition, deletion, or modification changes, that is, after the first transaction, there may be other transactions when creating the snapshot table. Similarly, after creating the first snapshot table and before the second transaction (transaction SEQ-Y), the source database may undergo other addition, deletion, or modification changes. That is, during the time period from the occurrence time of the first transaction to the creation time of the first snapshot table, there may be other transactions, and during the time period from the creation time of the first snapshot table to the occurrence time of the second transaction, there may also be other transactions. Therefore, the snapshot table t1 includes the data corresponding to transactions SEQ-1 to SEQ-X, and also includes some of the incremental data between the first transaction (transaction SEQ-X) and the second transaction (transaction SEQ-Y).

[0056] In order to minimize the amount of incremental data (incremental data of the transactions occurring between the first transaction and the second transaction) between the first transaction and the second transaction and avoid excessive data volume for subsequent processing due to too much incremental data between the first transaction and the second transaction, a transaction whose occurrence time in the source database is later than and close to the creation time of the first snapshot table can be preferentially selected as the second transaction.

[0057] In some embodiments, if the data in the source database does not undergo addition, deletion, or modification changes after creating the first snapshot table, the second transaction and the first transaction can be the same transaction.

[0058] Optionally, acquiring the second transaction after creating the first snapshot table includes: after creating the first snapshot table of the source database, obtaining the log number of the first snapshot table; determining a transaction whose corresponding log number is greater than or equal to the log number of the first snapshot table as the second transaction.

[0059] During the database operation process, the database log is used to record all events that add, delete, and change the database, so as to facilitate event tracing and data recovery in the event of a failure. Each database log has a unique log number, which is used to identify the log. The format and generation method of the log number vary from system to system, and usually include a timestamp, sequence number, or other identification information. Therefore, based on the log number, the order in which the events of the database occurred can be determined.

[0060] The log number of the first snapshot table refers to the log number that records the creation of the snapshot table. The log number of a transaction refers to the log number of the event corresponding to the transaction. For example, at a certain moment, data 3 is inserted into the source database and data 2 is deleted. By recording the database log of the event, the insertion of data 3 and the deletion of data 2 can be recorded as one transaction, and the log number corresponding to the transaction is the log number of the database log that records the event.

[0061] Specifically, after creating the first snapshot table of the source database, the log number of the first snapshot table is obtained, and the log number can represent the time when the first snapshot table is created. Then, based on the log number, a transaction of the source database after the first snapshot table is created is determined to be the second transaction, that is, a transaction whose corresponding log number is greater than or equal to the log number of the first snapshot table is the second transaction.

[0062] Determining the second transaction that is later than the first snapshot table by log number is simple to implement and highly efficient. Compared with determining the last transaction in the transaction queue as the second transaction, this method avoids the existence of too much incremental data due to a long time between the creation time of the first snapshot table and the occurrence time of the second transaction, thereby reducing the amount of data for subsequent data processing.

[0063] Step S203: restart the target database data storage, and when the second transaction stored in the target database is detected, suspend the target database data storage, and create a second snapshot table in the target database.

[0064] In order to prevent the data from being synchronized to the second transaction when the first snapshot table is created, resulting in missing the opportunity to generate the second snapshot table in the target database, the target database data entry can be temporarily suspended when a data verification trigger event is detected. After obtaining the second transaction, the target database data entry can be restarted.

[0065] After restarting the target database data storage, the synchronization software of the target database obtains transactions from the transaction queue in sequence and synchronizes the transactions to the target database. In order to ensure that there are no other transactions between the creation time of the second snapshot table and the data storage time of the second transaction, and there is no need to use the synchronization tool to create the second snapshot table immediately after the second transaction, when the second transaction stored in the target database is detected, the target database data storage is suspended, and then the second snapshot table is created in the target database.

[0066] Continue to refer to Figure 4 When generating the snapshot table t2 (the second snapshot table) in the target database, there are no other transactions during the period from the data entry time of the second transaction (SEQ-Y) to the creation time of the second snapshot table. Therefore, the full data of the second snapshot table t2 includes the incremental data between transaction SEQ-1 and transaction SEQ-X, and all the incremental data between the first transaction (transaction SEQ-X) and the second transaction (transaction SEQ-Y).

[0067] In some embodiments, in order not to affect data synchronization and avoid overloading the synchronization software of the target database due to excessive unsynchronized data in the source database, after creating the second snapshot table in the target database, the data entry of the target database can be restarted.

[0068] Step S204: Generate a data verification result based on the first snapshot table, the second snapshot table, the first transaction, and the second transaction.

[0069] As described above, when data synchronization is successful, the data in the first snapshot table includes the full data of the source database before the first transaction, and part of the incremental data between the first transaction and the second transaction; the data in the second snapshot table includes the full data of the target database before the first transaction, and all the incremental data corresponding to the first transaction to the second transaction. The full data refers to all the data in the database or snapshot table.

[0070] Since when data synchronization is error-free, the data in the first snapshot table and the data in the second snapshot table may not be in the same stage, resulting in inconsistent data. Therefore, when comparing and verifying the data in the first snapshot table and the data in the second snapshot table, the inconsistent data will lead to inaccurate verification results. For example, if there are no other transactions between the first transaction and the second transaction, the full data of the first snapshot table is the data changed before the first transaction, and the full data of the second snapshot table is the data synchronized to the target database before the second transaction. When comparing the first snapshot table with the second snapshot table, the second snapshot table has more incremental data of the second transaction than the first snapshot table. Therefore, the verification between the first snapshot table and the second snapshot table is inconsistent, and this inconsistency is not caused by data synchronization errors.

[0071] In this embodiment, the data corresponding to the inconsistent verification results in the above examples are all caused by the incremental data between the first transaction and the second transaction. The incremental data between the first transaction and the second transaction can be determined based on the first transaction and the second transaction. And based on this incremental data, the data in the first snapshot table and the data in the second snapshot table are adjusted to data in the same stage. Exemplarily, in the first snapshot table and the second snapshot table, the incremental data is deleted so that the remaining data in the first snapshot table and the second snapshot table are both the data before the first transaction; or, the incremental data is supplemented in the first snapshot table so that the data in the first snapshot table and the second snapshot table are both the data before the second transaction. The full amount of data in the same stage in the first snapshot table and the second snapshot table is subjected to data comparison verification. If the data verification result shows consistency, it indicates that the data synchronization is successful before the first transaction or the second transaction, and there is no loss or error; if the data verification result shows inconsistency, it indicates that the data synchronization is incorrect before the first transaction or the second transaction, and the inconsistent data needs to be repaired to ensure that the data in the target database is consistent with the data in the source database.

[0072] The data repair method can be to obtain each transaction synchronized in the transaction queue, compare the transaction queue of the source database and the transaction queue of the target database, and determine the lost transaction or the incorrect transaction. Based on the lost transaction or the incorrect transaction, the data in the target database is repaired. It can also be to determine the events of the incremental, deleted, and modified data that cause the inconsistency through the database logs of the source database and the target database, and based on this event, repair the data in the target database.

[0073] The data verification method provided by the embodiment of the present application provides a data verification strategy without blocking the source database. Specifically, when performing data verification, the synchronization service of the target database is paused, that is, data entry is paused, and the transaction with the latest occurrence time stored in the source database is obtained and recorded as the first transaction; a first snapshot table is created in the source database. When the snapshot table is created, the source database can continuously store new transactions, and the second transaction stored in the source database after the creation of the first snapshot table is obtained; the synchronization service of the target database is restarted. When the target database synchronizes to the second transaction, the synchronization service of the target database is paused, and a second snapshot table adjacent to the second transaction is created in the target database; finally, based on the data recorded in the first snapshot table and the second snapshot table and the first transaction and the second transaction, the data is verified to generate a data verification result. Based on the transactions stored before and after the created snapshot table, the data in the snapshot table is processed, which can make the data in the same stage and avoid the problem of inaccurate data verification caused by the synchronization data not being in the same stage, improving the accuracy of data verification.

[0074] Optionally, generate a data verification result based on the first snapshot table, the second snapshot table, the first transaction, and the second transaction, including: generating a preliminary data verification result based on the first snapshot table and the second snapshot table; determining the data that has changed in the source database between the first transaction and the second transaction as incremental data; generating a data verification result based on the preliminary data verification result and the incremental data.

[0075] The preliminary data verification result is used to indicate whether the full amount of data in the first snapshot table is consistent and accurate with the full amount of data in the second snapshot table. Since the full amount of data in the first snapshot table and the second snapshot table includes multiple pieces of data, the preliminary data verification result can include the verification results of each piece of data. Incremental data refers to the data that has changed in the source database between the first transaction and the second transaction, that is, the incremental data corresponding to the transactions existing between the first transaction and the second transaction. It should be noted that the incremental data does not include the incremental data of the first transaction.

[0076] The preliminary data verification result is divided into consistent or inconsistent according to whether the data is consistent. Among them, the inconsistent verification can be further divided according to the data conditions of the first snapshot table and the second snapshot table. For example, if the data content of this item in the first snapshot table and the second snapshot table is different, the preliminary data verification result of this item is data inconsistent; if this item of data only exists in the first snapshot table, the preliminary data verification result of this item is that there is more data in the source database; if this item of data only exists in the second snapshot table, the preliminary data verification result of this item is that there is more data in the target database.

[0077] Each piece of data in the database is composed of multiple fields (columns), and each field stores specific information, and all fields constitute the complete data. The multiple fields include but are not limited to identifiers, data attributes, timestamps, etc. Among them, each piece of data contains a primary key value that uniquely identifies the data. The primary key value has uniqueness, non-nullability, and immutability, that is, two pieces of data cannot have the same primary key value in a database, each piece of data must have a valid primary key value, and it is not changed. The primary key value is usually the identifier field of the data.

[0078] Therefore, the primary key values of each piece of data in the full amount of data in the first snapshot table can be used to retrieve the corresponding data in the second snapshot table, and the content of each piece of data in the first snapshot table is compared with the content of each piece of data in the second snapshot table to determine the preliminary data verification result. Through the primary key value of the data, the row where the data is located can be accurately and efficiently located, providing a more efficient path to access the data, and avoiding the problem that the same data cannot be determined in the two tables after the data content changes.

[0079] Exemplarily, Figure 5 is a schematic diagram of data verification provided by an embodiment of the present application. As Figure 5As shown in the figure, when a data verification trigger event is detected, data 1 and data 2 are stored in the source database. After the first transaction, a snapshot table t1 (the first snapshot table) is created. Between the first transaction and the snapshot table t1, the source database stores transaction 1 (inserting data 3 and data 4). Then, the data in the snapshot table t1 are data 1, data 2, data 3, and data 4. From transaction 2 (inserting data 5, deleting data 4, and updating data 3) and transaction 3 (updating data 1) after the creation of the snapshot table t1, transaction 2 is determined as the second transaction. During data synchronization, when the target database detects transaction 2, it pauses data entry into the database and creates a snapshot table t2. The data in the snapshot table t2 are data 1, data 2, data 3' (data 3' is the updated data 3), and data 5. Among them, transaction 1 is included between the first transaction and the second transaction (transaction 2), and the incremental data are the incremental data of transaction 1 and transaction 2, that is, data 3, data 4, and data 5 are the data that have changed in the source database between the first transaction and the second transaction.

[0080] Table 1 shows the preliminary data verification results of the snapshot table t1 and the snapshot table t2. By comparing the data in the snapshot table t1 and the snapshot table t2, the preliminary data verification results are obtained. If there is no error in data synchronization, because the data are not in the same stage, the data in the snapshot table t1 are the full amount of data in the source database at the time of the occurrence of transaction 1, and the data in the snapshot table t2 are the full amount of data in the target database at the time of the occurrence of transaction 2. Obviously, data 3, data 4, and data 5 are still inconsistent.

[0081] Table 1 Preliminary Data Verification Results of Snapshot Table t1 and Snapshot Table t2

[0082]

[0083] After generating the data verification results, it is necessary to determine whether the data with inconsistent verification results are caused by data synchronization errors. Therefore, the data verification results can be determined based on the incremental data. That is, in the preliminary data verification results, the items corresponding to the incremental data can be deleted, and only the verification results of the data other than the incremental data are included. That is, in the first snapshot table, some of the incremental data included in the first snapshot table are deleted, and in the second snapshot table, the incremental data are deleted; or, in the preliminary data verification results, the verification results of the incremental data can be further verified, such as incremental verification, to correct the preliminary data verification results to obtain the data verification results.

[0084] Exemplarily, the data in the first snapshot table and the data in the second snapshot table can be verified first to obtain a preliminary data verification result. Then, the incremental data is subtracted from the preliminary data verification result, and the remaining data is the verification result of the data before the first transaction. It is also possible to determine the incremental data verification result based on the data changes in the source database and the target database for each piece of incremental data after obtaining the preliminary data verification result. According to the incremental data verification result, the preliminary data verification result is corrected to obtain the data verification result.

[0085] Based on the data verification result determined from the preliminary data verification result and the incremental data, it can be ensured that the data in the determined data verification result is in the same stage, so that the finally obtained verification result only indicates whether the data synchronization is successful, improving the accuracy of data verification.

[0086] Optionally, the preliminary data verification result is used to indicate whether the data in the first snapshot table and the second snapshot table is consistent; generating a data verification result based on the preliminary data verification result and the incremental data includes: determining whether the inconsistent data in the preliminary data verification result belongs to the incremental data; if so, deleting the inconsistent data in the preliminary data verification result to obtain the data verification result.

[0087] Since the full amount of data includes at least one piece of data, the preliminary data verification result can include the preliminary data verification results of each piece of data (whether each piece of data is consistent), that is, the preliminary data verification results of each piece of data constitute the preliminary data verification result of the full amount of data.

[0088] Among them, the consistent data in the first snapshot table and the second snapshot table indicates that the data has been successfully synchronized to the target database. The inconsistent data may be due to data synchronization errors or data not being in the same stage. The data not in the same stage can only belong to the incremental data. Therefore, the inconsistent data in the preliminary data verification result belonging to the incremental data can be deleted, so that the remaining inconsistent data is only caused by data synchronization errors.

[0089] Specifically, obtain the inconsistent data in the preliminary data verification result, and determine whether the inconsistent data belongs to the incremental data according to the primary key value of the inconsistent data. Exemplarily, according to the primary key value of the inconsistent data, retrieve in the primary key values of the incremental data. If the data is retrieved, it means that the data belongs to the incremental data.

[0090] If it belongs to the incremental data, it means that the inconsistency of each piece of data may be due to the data not being in the same stage. Then, delete the inconsistent data in the preliminary data verification result to obtain the data verification result; if it does not belong to the incremental data, it means that the inconsistency of each piece of data is caused by data synchronization errors, and then the preliminary data verification result is the data verification result.

[0091] Exemplarily, continuing to refer to Figure 5 As shown, when it is determined that Transaction 2 is the second transaction, the first incremental data between the first transaction and the second transaction is Data 3, Data 4, and Data 5, and the incremental data is Data 3, Data 4, and Data 5. Continuing to refer to the preliminary data verification result shown in Table 1, the inconsistent data is Data 3, Data 4, and Data 5, and all the inconsistent data belongs to the incremental data. Then, in the preliminary data verification result, delete the inconsistent data item, and the obtained final data verification result is the data in columns 1 and 2 of Table 1, and the verification results are all consistent.

[0092] If Data 1 and Data 2 are in error during the data synchronization process, that is, in the preliminary data verification result in Table 1, all of Data 1 to Data 5 are inconsistent, and Data 1 and Data 2 do not belong to the incremental data, then the data verification results of Data 1 and Data 2 are inconsistent.

[0093] In some embodiments, if the final data verification result is inconsistent or the verification result of some item data is inconsistent, then according to the data in the source database, repair the data in the target database for this item. The data repair method refers to the data repair method in the above embodiments and will not be elaborated here.

[0094] In this embodiment, by deleting in the preliminary data verification result the items that may be inconsistent due to the data not being in the same stage, ensuring that the remaining item data are all in the same stage, so that the finally obtained data verification result only indicates whether the data is successfully synchronized before the first transaction occurs, avoiding re-verifying the inconsistent data, with simple logic, easy implementation, and higher data verification efficiency.

[0095] Optionally, based on the preliminary data verification result and the incremental data, generate a data verification result, including: comparing the incremental data with the corresponding data in the second snapshot table to generate an incremental verification result; updating the preliminary data verification result of the incremental data in the preliminary data verification result to the incremental verification result to obtain the data verification result.

[0096] The data verification result provided in the above embodiment can determine whether there are synchronization errors such as missing data and abnormal storage in the data synchronization before the first transaction. For the incremental data generated during the verification process, it needs to be verified during the next data verification. If the data changes frequently, it may not be verified all the time. For example, if this item of data is still incremental data during the next data verification, then this data is still not verified during the next data verification.

[0097] Therefore, this embodiment provides another data verification result, which can determine whether there are synchronization errors such as missing data and abnormal storage in the data synchronization before the second transaction.

[0098] Specifically, according to the above content, the full data of the first snapshot table only contains partial incremental data, while the second snapshot table contains all incremental data. Due to the different incremental data included, their verification results are inconsistent. Therefore, the incremental data between the first transaction and the second transaction can be compared separately with all the incremental data in the second snapshot table, that is, compare the incremental data with the corresponding data in the second snapshot table, and generate an incremental verification result based on the comparison result. This incremental verification result determines whether the incremental data of the transaction parsed by the source database is consistent with the data in the target database. Therefore, the verification result of the incremental data in the preliminary data verification result can be updated to the incremental verification result to obtain the data verification result.

[0099] Among them, the specific method for determining the incremental verification result is as follows:

[0100] Construct the incremental data into the first incremental data in the form of key-value pairs (Key-Value Pair), where the Key is the primary key value, and the Value is other information and identifiers, such as other column information in a piece of data except the primary key value. If the change corresponding to this piece of data is an insertion, the other information of the Value is the other columns of the data, and the identifier is INSERT; if the change corresponding to this piece of data is a deletion, the other information of the Value of this piece of data is empty, and the identifier is DELETE; if the change corresponding to this piece of data is an update, when the Key in the memory does not exist, the Key is the primary key value, and the other information of the Value is the updated column. When the Key in the memory exists, the existing Value is overwritten. Among them, the identifier is UPDATE for all, and some parsing methods cannot obtain the values of all columns after update, and only retain or replace the existing information.

[0101] Figure 6 It is a schematic diagram of the method for determining the incremental verification result provided by the embodiment of the present application. As Figure 6 shown, first, obtain the data corresponding to the incremental data in the second snapshot table item by item, denoted as Figure 6The second incremental data in []. Next, obtain the corresponding data in the incremental data through the Key of each piece of data in the second incremental data, denoted as the first incremental data; when the corresponding first incremental data is obtained through the Key, judge the other information of the corresponding first incremental data and the corresponding second incremental data. If the other information is the same, the data is consistent; if the other information is different, the data is inconsistent; if the two ends of the other information are marked as DELETE, the data is deleted in the source database and the data is consistent. When comparing each piece of data, delete the piece of data in the first incremental data and the second incremental data, indicating that it has been retrieved. When each piece of data in the second incremental data has been retrieved and there are still remaining first incremental data, it means that there are remaining first incremental data in the source database that do not exist in the target database. In this scenario, obtain the identifier of the remaining first incremental data. If the identifier is DELETE, it means that it has been deleted in the source database, so the data has not been synchronized to the target database, and the data is consistent; if the identifier is not DELETE, such as INSERT or UPDATE, it means that the data does not exist in the target database, and the data is inconsistent.

[0102] Exemplarily, Table 2 is provided by an embodiment of the present application Figure 5 The incremental verification results in the data verification process of []. As shown in Table 2, the incremental data (Data 3, Data 4, and Data 5) of Transaction 1 and Transaction 2 are compared. Data 3 and Data 5 in the incremental data are retrieved according to Data 3 and Data 5 in the second snapshot table. If there is no error in data synchronization, the Values of the data are the same, and the incremental verification result is consistent; there is Data 4 in the incremental data, but Data 4 does not exist in the second snapshot table. That is, after traversing the second incremental data, the remaining incremental data is Data 4, but the identifier of Data 4 is DELETE, indicating that the data has been deleted in the source database, and the incremental verification result is consistent.

[0103] Table 2 Figure 5 The incremental verification results in the data verification process

[0104]

[0105] After obtaining the incremental verification results, replace the corresponding items in the preliminary data verification results of Table 1 with the incremental verification results of Table 2 to obtain the final data verification results, that is, the data verification results of Data 1 to Data 5 are all consistent.

[0106] By verifying the incremental data, the verification of the incremental data (which is process data) and the full - volume data is achieved. It can accurately determine whether the incremental data in the source database is successfully synchronized to the target database. At the same time, by using the incremental data verification result to replace the preliminary data verification result, the verification based on the second snapshot table is realized, that is, the full - volume data before the second transaction is verified, avoiding the situation where the data to be synchronized is not verified, and the verification result is more accurate and effective.

[0107] Optionally, based on the preliminary data verification result and the incremental data, a data verification result is generated, including: comparing the incremental data with the corresponding data in the second snapshot table to generate an incremental verification result; if the inconsistent data in the preliminary data verification result belongs to the incremental data, then updating the verification result of the incremental data in the preliminary data verification result to the incremental verification result to obtain the data verification result; if the inconsistent data in the preliminary data verification result is not the incremental data, then determining the preliminary data verification result as the data verification result.

[0108] In order to further improve the speed and efficiency of data verification, during the process of updating the preliminary data verification result, only the inconsistent data can be updated. That is, it is judged whether the inconsistent data in the preliminary data verification result belongs to the incremental data. If it belongs, then updating the preliminary data verification result of this item in the preliminary data verification result to the incremental verification result. If it does not belong, then this item of data is caused by an error in data synchronization, and the preliminary data verification result is the data verification result. By this method, some incremental data can be updated, improving the speed of data verification.

[0109] Figure 7 It is a schematic flowchart of another data verification method provided by the embodiment of the present application. As Figure 7 shown, the data verification method mainly includes the following steps:

[0110] Step S701: After detecting a data verification trigger event, suspend the data entry of the target database and obtain the first transaction stored in the source database.

[0111] Step S702: After storing the first transaction in the source database, create a first snapshot table of the source database.

[0112] Step S703: Obtain the log number of the first snapshot table, and determine a transaction whose corresponding log number is greater than or equal to the log number of the first snapshot table as the second transaction.

[0113] Step S704: Restart the data entry of the target database. When detecting the second transaction stored in the target database, suspend the data entry of the target database and create a second snapshot table in the target database.

[0114] Step S705: Generate a preliminary data verification result based on the first snapshot table and the second snapshot table.

[0115] Step S706: Determine the data that has changed in the source database between the first transaction and the second transaction as incremental data.

[0116] Step S707: Determine whether the inconsistent data in the preliminary data verification result belongs to the incremental data. If so, execute Step S708; if not, execute Step S709.

[0117] Step S708: Delete the inconsistent data from the preliminary data verification result to obtain the data verification result.

[0118] Step S709: Determine that the preliminary data verification result is the data verification result.

[0119] The data verification method provided in this embodiment has a similar implementation principle and technical effect, which will not be elaborated here.

[0120] Figure 8 It is a structural schematic diagram of a data verification device provided in an embodiment of the present application. As Figure 8 shown, the data verification device provided in this embodiment includes: a first transaction acquisition module 801, a first snapshot table creation module 802, a second snapshot table creation module 803, and a data verification module 804.

[0121] The first transaction acquisition module 801 is used to pause the data entry of the target database after detecting a data verification trigger event, and acquire the first transaction stored in the source database. A transaction is an event used to add, delete, or modify data in the source database, and the first transaction is the transaction with the latest occurrence time stored in the source database when the data verification trigger event is detected; the first snapshot table creation module 802 is used to create a first snapshot table of the source database after storing the first transaction in the source database, and acquire the second transaction stored in the source database after creating the first snapshot table; the second snapshot table creation module 803 is used to restart the data entry of the target database, pause the data entry of the target database when detecting the second transaction stored in the target database, and create a second snapshot table in the target database; the data verification module 804 is used to generate a data verification result based on the first snapshot table, the second snapshot table, the first transaction, and the second transaction.

[0122] Optionally, the data verification module 804 includes a preliminary data verification result unit, an incremental data determination unit, and a data verification result determination unit, where:

[0123] The preliminary data verification result unit is used to generate a preliminary data verification result based on the first snapshot table and the second snapshot table; the incremental data determination unit is used to determine the data that has changed in the source database between the first transaction and the second transaction as incremental data; the data verification result determination unit is used to generate a data verification result based on the preliminary data verification result and the incremental data.

[0124] Optionally, the preliminary data verification result is used to indicate whether the data items in the first snapshot table and the second snapshot table are consistent. The data verification result determination unit is specifically used for: determining whether the inconsistent data in the preliminary data verification result belongs to the incremental data; if so, deleting the inconsistent data in the preliminary data verification result to obtain the data verification result.

[0125] Optionally, the data verification result determination unit is specifically used for: comparing the incremental data with the corresponding data in the second snapshot table to generate an incremental verification result; updating the preliminary data verification result of the incremental data in the preliminary data verification result to the incremental verification result to obtain the data verification result.

[0126] Optionally, the data verification result determination unit is specifically used for: comparing the incremental data with the corresponding data in the second snapshot table to generate an incremental verification result; if the inconsistent data in the preliminary data verification result belongs to the incremental data, updating the preliminary data verification result of the incremental data in the preliminary data verification result to the incremental verification result to obtain the data verification result; if the inconsistent data in the preliminary data verification result does not belong to the incremental data, determining the preliminary data verification result as the data verification result.

[0127] Optionally, the first snapshot table creation module 802 is specifically used for: after storing the first transaction in the source database, creating a first snapshot table of the source database; after creating the first snapshot table of the source database, obtaining the log number of the first snapshot table; determining a corresponding transaction whose log number is greater than or equal to the log number of the first snapshot table as the second transaction.

[0128] The data verification device provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here in this embodiment.

[0129] Figure 9 It is a schematic structural diagram of an electronic device provided in an embodiment of the present application. As Figure 9 shown, the electronic device 90 provided in this embodiment includes: at least one processor 901 and a memory 902. Optionally, the device 90 further includes a communication component 903. Among them, the processor 901, the memory 902, and the communication component 903 are connected through a bus.

[0130] In the specific implementation process, at least one processor 901 executes the computer-executable instructions stored in the memory 902, so that at least one processor 901 executes the above-mentioned method.

[0131] For the specific implementation process of the processor 901, reference can be made to the above method embodiments. Their implementation principles and technical effects are similar, and will not be elaborated here in this embodiment.

[0132] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly embodied as being executed and completed by a hardware processor, or by a combination of hardware and software modules in the processor.

[0133] The memory may include a high-speed random access memory (RAM), and may also include non-volatile memory (NVM), such as at least one disk memory.

[0134] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0135] This application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0136] This application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the processor executes the computer-executable instructions, the above method is implemented.

[0137] The above-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0138] An exemplary readable storage medium is coupled to the processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in a device.

[0139] The division of units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings or direct couplings or communication connections shown or discussed between each other can be indirect couplings or communication connections through some interfaces, devices, or units, and can be in electrical, mechanical, or other forms.

[0140] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0141] In addition, the functional units in various embodiments of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0142] If a function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs, etc., various media that can store program codes.

[0143] Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When this program is executed, it executes the steps including the above method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disks, or optical discs, etc., various media that can store program codes.

[0144] Finally, it should be noted that: After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily think of other implementation manners of the present invention. The present invention is intended to cover any variations, uses, or adaptive changes of the present invention, which follow the general principles of the present invention and include the common general knowledge or conventional technical means in the technical field not disclosed in the present invention. It is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A data verification method, characterized in that, Including: After detecting a data verification trigger event, suspend data entry into the target database and obtain the first transaction stored in the source database; A transaction is an event for adding, deleting, or modifying data in the source database; the first transaction is the transaction with the latest occurrence time stored in the source database when the data verification trigger event is detected; After storing the first transaction in the source database, create a first snapshot table of the source database and obtain the second transaction stored in the source database after creating the first snapshot table; Restart data entry into the target database. When detecting the second transaction stored in the target database, suspend data entry into the target database and create a second snapshot table in the target database; Generate a data verification result based on the first snapshot table, the second snapshot table, the first transaction, and the second transaction.

2. The method according to claim 1, wherein Generating a data verification result based on the first snapshot table, the second snapshot table, the first transaction, and the second transaction includes: Generate a preliminary data verification result based on the first snapshot table and the second snapshot table; Determine the data that has changed in the source database between the first transaction and the second transaction as incremental data; Generate the data verification result based on the preliminary data verification result and the incremental data.

3. The method according to claim 2, characterized in that, The preliminary data verification result is used to indicate whether the data items in the first snapshot table and the second snapshot table are consistent; Generating the data verification result based on the preliminary data verification result and the incremental data includes: Judge whether the inconsistent data in the preliminary data verification result belongs to the incremental data; If so, delete the inconsistent data from the preliminary data verification result to obtain the data verification result.

4. The method according to claim 2, wherein Generating the data verification result based on the preliminary data verification result and the incremental data includes: Compare the incremental data with the corresponding data in the second snapshot table to generate an incremental verification result; Update the preliminary data verification result of the incremental data in the preliminary data verification result to the incremental verification result to obtain the data verification result.

5. The method according to claim 2, wherein Generating a data verification result based on the preliminary data verification result and the incremental data includes: Compare the incremental data with the corresponding data in the second snapshot table to generate an incremental verification result; If the inconsistent data in the preliminary data verification result belongs to the incremental data, update the preliminary data verification result of the incremental data in the preliminary data verification result to the incremental verification result to obtain the data verification result; If the inconsistent data in the preliminary data verification result does not belong to the incremental data, determine the preliminary data verification result as the data verification result.

6. The method according to any one of claims 1-5, characterized in that Obtaining the second transaction after creating the first snapshot table includes: After creating the first snapshot table of the source database, obtain the log number of the first snapshot table; Determine a transaction whose corresponding log number is greater than or equal to the log number of the first snapshot table as the second transaction.

7. A data verification device, characterized in that, Including: A first transaction acquisition module, configured to pause data entry into the target database after detecting a data verification trigger event, and acquire a first transaction stored in the source database, where the transaction is an event for adding, deleting, or modifying data in the source database, and the first transaction is the transaction with the latest occurrence time stored in the source database when the data verification trigger event is detected; A first snapshot table creation module, configured to create a first snapshot table of the source database after storing the first transaction in the source database, and acquire a second transaction stored in the source database after creating the first snapshot table; A second snapshot table creation module, configured to restart data entry into the target database, pause data entry into the target database when detecting the second transaction stored in the target database, and create a second snapshot table in the target database; A data verification module, configured to generate a data verification result based on the first snapshot table, the second snapshot table, the first transaction, and the second transaction.

8. An electronic device, characterized in that, Comprising: A memory and a processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that Computer execution instructions are stored in the computer-readable storage medium, and when the computer execution instructions are executed by the processor, they are used to implement the method according to any one of claims 1-6.

10. A computer program product, characterized in that, Comprising a computer program, which when executed by the processor implements the method according to any one of claims 1-6.

Images