Data consistency verification method, electronic equipment and program product

Abstract

The invention provides a data consistency verification method, electronic equipment and a program product, and relates to the technical field of data processing. The method comprises the following steps: creating at least one verification task according to primary key fragment information; executing each verification task in parallel, and determining a current first verification primary key of the first data table and a current second verification primary key of the second data table in each primary key region in a sliding manner according to the primary key region corresponding to each verification task and a primary key sorting mode in each primary key region; obtaining a plurality of first to-be-compared column data of a row where the current first verification primary key is located from the first data table, and obtaining a plurality of second to-be-compared column data of a row where the current second verification primary key is located from the second data table; and sequentially comparing the first column data to be compared with the corresponding second column data to be compared to obtain a comparison result of the first column data to be compared and the corresponding second column data to be compared. According to the method, the efficiency of data reading and data verification is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data consistency verification method, electronic equipment, and program product. Background Art

[0002] In large-scale enterprise applications, there are many business scenarios that require migrating data from a database table to another heterogeneous database. There are many ways to migrate. No matter which migration method is used, there will be the problem of data consistency verification after migration. If no verification is performed, the data may be inconsistent, which will affect the use of the data and the business system.

[0003] In the existing method, an encryption algorithm is usually used to generate a simpler verification code and primary key hash value for each table data to be verified in the source database and the target database and store them in the corresponding temporary files, and the two temporary files are compared for data consistency using the verification value and primary key hash value.

[0004] The calculation process of the above method is relatively complicated, occupies a large amount of system resources, and has low efficiency in data verification. Summary of the invention

[0005] The purpose of this application is to provide a data consistency verification method, electronic device and program product to improve the efficiency and accuracy of data consistency verification in heterogeneous databases in order to address the deficiencies in the above-mentioned prior art.

[0006] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows: In a first aspect, an embodiment of the present application provides a data consistency verification method, including: creating at least one verification task between a first data table in a source database and a second data table in a target database according to primary key sharding information, each verification task being used to compare the consistency of data in corresponding primary key regions in the first data table and the second data table; Execute various verification tasks in parallel, and according to the primary key areas corresponding to the various verification tasks and the primary key sorting methods in the primary key areas, slide to determine the current first verification primary key of the first data table and the current second verification primary key of the second data table in each primary key area; Acquire multiple first to-be-compared column data of the row where the current first verification primary key is located from the first data table, and acquire multiple second to-be-compared column data of the row where the current second verification primary key is located from the second data table; According to the order of each of the first column data to be compared and the order of each of the second column data to be compared, each of the first column data to be compared is compared with the corresponding second column data to be compared in turn to obtain a comparison result between each of the first column data to be compared and the corresponding second column data to be compared.

[0007] In a second aspect, an embodiment of the present application further provides a data consistency verification device, including: a creation module, a determination module, an acquisition module, and a processing module; The creation module is used to create at least one verification task between the first data table in the source database and the second data table in the target database according to the primary key sharding information, and each verification task is used to compare the consistency of the data in the corresponding primary key area of ​​the first data table and the second data table; The determination module is used to execute various verification tasks in parallel, and according to the primary key areas corresponding to the verification tasks and the primary key sorting methods in the primary key areas, slide to determine the current first verification primary key of the first data table and the current second verification primary key of the second data table in each primary key area; The acquisition module is used to acquire a plurality of first to-be-compared column data of the row where the current first verification primary key is located from the first data table, and acquire a plurality of second to-be-compared column data of the row where the current second verification primary key is located from the second data table; The processing module is used to compare each of the first column data to be compared with the corresponding second column data to be compared in sequence according to the order of each of the first column data to be compared and the order of each of the second column data to be compared, so as to obtain a comparison result between each of the first column data to be compared and the corresponding second column data to be compared.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate through the bus, and the processor executes the machine-readable instructions to implement the data consistency verification method provided in the first aspect.

[0009] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the data consistency verification method provided in the first aspect is executed.

[0010] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program code. When the computer program code is executed by a computing device, the computing device executes the data consistency verification method provided in the first aspect.

[0011] The beneficial effects of this application are: The present application provides a data consistency verification method, electronic device and program product, including: creating at least one verification task between a first data table in a source database and a second data table in a target database according to primary key sharding information; executing various verification tasks in parallel, sliding to determine the current first verification primary key of the first data table in each primary key area and the current second verification primary key of the second data table according to the primary key area corresponding to each verification task and the primary key sorting method in each primary key area; obtaining multiple first to-be-compared column data of the row where the current first verification primary key is located from the first data table, and obtaining multiple second to-be-compared column data of the row where the current second verification primary key is located from the second data table; comparing each first to-be-compared column data with the corresponding second to-be-compared column data in turn according to the order of each first to-be-compared column data and the order of each second to-be-compared column data, and obtaining the comparison result of each first to-be-compared column data with the corresponding second to-be-compared column data. The present method creates multiple verification tasks through primary key sharding information, thereby dividing the global verification task into multiple subtasks for parallel execution, which can improve the efficiency of data verification. During the data verification process, based on the principle that the primary keys of the database are arranged in order, the primary keys can be slid in sequence to quickly and accurately determine the row where the data to be verified is located. Based on the row where the data is located, the data in each column of the row can be read from the data table in real time, so that the data in each column of the data row can be compared with each other to complete data verification. The efficiency of data reading and data verification is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0013] Figure 1 A schematic diagram of a data consistency verification method provided in an embodiment of the present application; Figure 2 A schematic diagram of another data consistency verification method provided in an embodiment of the present application; Figure 3 A schematic diagram of a verification task creation result provided in an embodiment of the present application; Figure 4 A flowchart of another data consistency verification method provided in an embodiment of the present application; Figure 5 A schematic diagram of primary key sharding point division provided in an embodiment of the present application; Figure 6A schematic diagram of another data consistency verification method provided in an embodiment of the present application; Figure 7 A flowchart of another data consistency verification method provided in an embodiment of the present application; Figure 8 A schematic diagram of another data consistency verification method provided in an embodiment of the present application; Fig. 9 A flowchart of another data consistency verification method provided in an embodiment of the present application; Fig.10 A schematic diagram of a main key display provided in an embodiment of the present application; Fig.11 A schematic diagram of another data consistency verification method provided in an embodiment of the present application; Fig.12 A schematic diagram of a data consistency verification device provided in an embodiment of the present application; Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0014] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0015] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0016] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0017] The core goal of heterogeneous database data consistency check is to ensure the integrity and accuracy of data. During data migration, synchronization or sharing, data inconsistency between databases may occur due to various reasons (such as synchronization program bugs, SQL (Structured Query Language) statement execution failure, etc.). Through consistency check, these inconsistencies can be discovered and corrected in a timely manner, thereby ensuring the accuracy and reliability of data. This is of great significance for business decision-making, data analysis and other aspects.

[0018] At present, the commonly used data consistency verification methods can be roughly divided into the following categories: First: Compare by parsing the database log: Query each row of data from the source database and convert it into the target database format; Query the corresponding data from the target database; Compare the data in the source database and the target database to see if they are consistent.

[0019] This method requires multiple queries to the target database, which affects the performance of the target database.

[0020] Second: Query data from the source database, and then query and compare to the target database based on the primary key: Generate a temporary table; The source database is migrated to the target database, and the CRC value of the calculated block is written into the temporary table; Calculate the CRC value of the target database for comparison.

[0021] This method focuses on the data quality of data migration and cannot perform consistency verification on the migrated database.

[0022] Third: Query the source database and the target database, generate intermediate tables or files, and then process and compare the files: Query the source database; Hash the primary key to generate a hash value, then concatenate all field values ​​into a string and calculate the checksum; Hash the primary key into a temporary file according to the hash value; Compare two temporary files.

[0023] This method has lower performance and higher resource usage.

[0024] Based on the defects of the above methods, this solution proposes a data consistency verification method. By calculating the primary key sharding information, the verification tasks of the data table in the database are divided to improve the efficiency of data verification by executing tasks in parallel. By determining the primary key to be verified, the data of each column under the primary key to be verified is obtained from the data table in real time to compare the column data. There is no need to store the data to be compared in a temporary file, so it takes up less resources and has high calculation efficiency.

[0025] Figure 1 A schematic diagram of a data consistency verification method provided in an embodiment of the present application; the execution subject of the method may be an electronic device, such as Figure 1 As shown, the method may include: S101. Create at least one verification task between a first data table in a source database and a second data table in a target database according to primary key sharding information.

[0026] Each verification task is used to compare the consistency of data in the corresponding primary key area of ​​the first data table and the second data table.

[0027] First, in order to perform database operations on the source database and the target database, you can use JDBC (Java Database Connectivity, a standard Java API (Application Programming Interface)) to connect to the source database and the target database respectively.

[0028] JDBC provides a set of interfaces for executing SQL statements, processing result sets, and managing database connections. By using JDBC, Java applications can access and operate various relational database management systems (RDBMS), such as MySQL, PostgreSQL, Oracle, SQL Server, etc.

[0029] You can load the JDBC driver class of the source database and the JDBC driver class of the target database, and then call the database connection method to connect to the source database through the URL (Uniform Resource Locator), user name and password of the source database; and connect to the target database through the URL (Uniform Resource Locator), user name and password of the target database.

[0030] It is worth noting that the above is to connect to the source database through JDBC, and to connect to the target database through JDBC. After the source database is successfully connected and the target database is successfully connected, the JDBC interface can be called through the client to operate the source database and the target database respectively.

[0031] Optionally, the source database and the target database may be heterogeneous databases, and the target database may receive data migrated, synchronized or copied from the source database.

[0032] Typically, both the source database and the target database may contain multiple data tables. When performing data consistency verification, the two corresponding data tables in the source database and the target database may be verified. The first data table in the source database corresponds to the second data table in the target database. The developer may also select the data tables to be compared based on the verification requirements.

[0033] Usually a data table contains multiple rows and multiple columns. The primary key is used to uniquely identify each row in the data table. The primary key can be used to quickly and accurately locate a row in the data table.

[0034] When a data table contains more rows and columns, the amount of data also increases. In order to improve the processing efficiency of data consistency verification, the data table can be divided by primary key to create multiple verification tasks.

[0035] The primary key sharding information is used to indicate the specific division method when dividing the first data table and the second data table. According to the primary key sharding information, the first data table and the second data table can be divided into corresponding multiple primary key areas, so that according to the primary key areas, various verification tasks between the first data table and the second data table can be created. Each primary key area can correspond to a verification task, so the verification task is used to perform consistency comparison on the data in the primary key area corresponding to the verification task.

[0036] By creating verification tasks based on primary key sharding information, the global verification task can be decomposed into multiple subtasks to reduce the amount of data processing and resource usage of each verification task.

[0037] S102, executing various verification tasks in parallel, and slidingly determining the current first verification primary key of the first data table and the current second verification primary key of the second data table in each primary key area according to the primary key area corresponding to each verification task and the primary key sorting method in each primary key area.

[0038] In some embodiments, the verification tasks created above can be executed in parallel. During the parallel execution process, according to the primary key area corresponding to each verification task and the primary key sorting method in the primary key area, the current first verification primary key of the first data table in each primary key area and the current second verification primary key of the second data table can be determined in turn.

[0039] Based on the principle that the primary key of the database is ordered, the current first check primary key of the first data table and the current second check primary key of the second data table are determined by sliding according to the primary key sorting method, so that the primary key can be quickly located, thereby facilitating rapid query of data.

[0040] The efficiency of data verification can be improved by executing multiple verification tasks concurrently.

[0041] S103: Acquire multiple first to-be-compared column data of the row where the current first verification primary key is located from the first data table, and acquire multiple second to-be-compared column data of the row where the current second verification primary key is located from the second data table.

[0042] Based on the current first verification primary key, the row in the first data table where the data currently need to be verified is located can be directly determined. Similarly, based on the current second verification primary key, the row in the second data table where the data currently need to be verified is located can be determined. Based on the determined row, the column data in the row can be compared to achieve data verification.

[0043] In this embodiment, based on the current first verification primary key, multiple first column data to be compared in the row where the current first verification primary key is located can be directly obtained in real time from the first data table. Similarly, based on the current second verification primary key, multiple second column data to be compared in the row where the current second verification primary key is located can be directly obtained from the second data table.

[0044] Table 1 below schematically shows the data storage status of the first data table: Table 1

[0045] Assuming that the current first verification primary key of the determined first data table is primary key 2, then the multiple first to-be-compared column data of the row where the current first verification primary key is located are data 6, data 7, data 8, data 9 and data 10 respectively.

[0046] The method for reading the second to-be-compared column data of the second data table may refer to the first data table.

[0047] By reading the to-be-compared column data from the first data table and the second data table in real time, the efficiency of data comparison can be improved, and there is no need to temporarily store data through intermediate files, thereby improving computing efficiency.

[0048] S104 , comparing each first column data to be compared with the corresponding second column data to be compared in sequence according to the sequence of each first column data to be compared and the sequence of each second column data to be compared, to obtain a comparison result between each first column data to be compared and the corresponding second column data to be compared.

[0049] For the comparison of column data, the same columns are compared with each other. Based on this, the first data to be compared and the second data to be compared in the same column as the first data to be compared can be compared in turn according to the order of each first data to be compared column and the order of each second data to be compared column, so as to obtain the comparison result of each first data to be compared column and the corresponding second data to be compared column.

[0050] Table 2 below schematically shows the data storage status of the second data table: Table 2

[0051] In combination with Table 1 and Table 2, assuming that it is determined that the data comparison is currently being performed on the row where the primary key 2 of the first data table is located and the row where the primary key 2 of the second data table is located. Since the row where the primary key 2 of the first data table is located and the row where the primary key 2 of the second data table is located both contain 5 columns of data, then according to the order of the column data, data 6 of the first data table can be compared with data 21 of the second data table, and data 7 of the first data table can be compared with data 22 of the second data table, and so on, until all column data of the row where the primary key 2 of the first data table is located is compared with the corresponding column data of the row where the primary key 2 of the second data table is located, then the comparison result of the row where the primary key 2 of the first data table is located and the row where the primary key 2 of the second data table is located is obtained.

[0052] By sliding and updating the current first verification primary key and the current second verification primary key, data verification can be completed for all rows in the primary key area of ​​a verification task in turn, and the verification results of the first data table and the second data table under a verification task can be obtained. By combining the verification results of each verification task, the complete verification results of the first data table and the second data table can be obtained.

[0053] In summary, the data consistency verification method provided by this embodiment includes: creating at least one verification task between the first data table in the source database and the second data table in the target database according to the primary key sharding information; executing various verification tasks in parallel, and slidingly determining the current first verification primary key of the first data table in each primary key area and the current second verification primary key of the second data table according to the primary key area corresponding to each verification task and the primary key sorting method in each primary key area; obtaining multiple first column data to be compared in the row where the current first verification primary key is located from the first data table, and obtaining multiple second column data to be compared in the row where the current second verification primary key is located from the second data table; according to the order of each first column data to be compared and the order of each second column data to be compared, each first column data to be compared is compared with the corresponding second column data to be compared in turn, and obtaining the comparison result of each first column data to be compared with the corresponding second column data to be compared. This method creates multiple verification tasks through primary key sharding information, thereby dividing the global verification task into multiple subtasks for parallel execution, which can improve the efficiency of data verification. During the data verification process, based on the principle that the primary keys of the database are arranged in order, the primary keys can be slid in sequence to quickly and accurately determine the row where the data to be verified is located. Based on the row where the data is located, the data in each column of the row can be read from the data table in real time, so that the data in each column of the data row can be compared with each other to complete data verification. The efficiency of data reading and data verification is effectively improved.

[0054] Optionally, Figure 2 A flowchart of another data consistency verification method provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, in the above step S101, creating at least one verification task between the first data table in the source database and the second data table in the target database according to the primary key sharding information may include: S201. According to the global primary key information, a primary key sharding strategy is adopted to determine the primary key sharding information.

[0055] The global primary key information is used to characterize the value range of the primary key of the first data table and the second data table.

[0056] It is worth noting that the value range of the primary key of the first data table and the second data table is the same. For example, the primary key is from 0 to 19, so the global primary key information is [0-19].

[0057] In some embodiments, the primary key sharding information refers to the primary key sharding point. Then, according to the global primary key information, the primary key sharding point can be determined, and the global primary key is divided according to the primary key sharding point to obtain multiple primary key areas.

[0058] In other embodiments, the primary key sharding information may refer to a primary key region under the global primary key information, that is, the primary key sharding information itself corresponds to a small primary key region under the global primary key information, and the primary key region may also be divided according to the primary key sharding information.

[0059] S202: Divide the first data table and the second data table into primary key regions according to the primary key sharding information, and create at least one verification task between the first data table and the second data table according to the primary key region division result.

[0060] Figure 3 A schematic diagram of a verification task creation result provided in an embodiment of the present application is shown in FIG. Figure 3 As shown in the figure, assuming that the global primary key information is [0,19], and the primary key sharding information is the primary key sharding point, assuming that the primary key sharding point is 9, the global primary key can be divided into [0,9] and [10,19], so that a verification task is created for the primary key area [0,9] and a verification task is created for the primary key area [10,19].

[0061] When the primary key sharding information is region information, the determined primary key sharding information may include: [0,9] and [10,19]. Then, a verification task may be directly created for the primary key region [0,9], and a verification task may be directly created for the primary key region [10,19].

[0062] Figure 4 A flowchart of another data consistency verification method provided in an embodiment of the present application; refer to Figure 4 As shown, optionally, in the above step S201, according to the global primary key information, the primary key sharding strategy is adopted to determine the primary key sharding information, which may include: S401. Determine a current first primary key end value and a current second primary key end value according to global primary key information.

[0063] This embodiment is explained with primary key sharding information as the primary key sharding point. Usually, the current primary key end value can be found from the global primary key information, so that the global primary key information is searched multiple times by binary search step by step through the binary search method, and the primary key value range indicated by the global primary key information is divided into multiple primary key areas.

[0064] S402: Determine at least one primary key sharding point according to the first primary key end value and the second primary key end value.

[0065] For the current first primary key end value and the current second primary key end value, since the two primary key end values ​​can determine a primary key area, then at least one primary key sharding point can be determined between the two primary key end values; through at least one primary key sharding point, the primary key area between the two end values ​​can be further divided into smaller primary key areas.

[0066] S403: Determine primary key sharding information corresponding to the current first primary key end value and the current second primary key end value according to each primary key sharding point.

[0067] Then, the at least one primary key sharding point determined above may be used as the primary key sharding information corresponding to the current first primary key end value and the current second primary key end value.

[0068] When the current first primary key end value and the current second primary key end value change, the corresponding primary key fragmentation point will also change.

[0069] That is, at least one primary key sharding point can be determined between every two primary key end values ​​to re-divide the primary key area between the two primary key end values, thereby dividing the global primary key range into multiple small primary key areas level by level.

[0070] Optionally, in the above step S402, determining at least one primary key sharding point based on the first primary key end value and the second primary key end value may include: if the types of the first primary key end value and the second primary key end value are digital types, rounding down the average value of the first primary key end value and the second primary key end value to obtain at least one primary key sharding point.

[0071] Usually, primary keys have different representations. Primary keys can be either numeric or string types.

[0072] When the primary key is a numeric type, the primary key sharding point between any two primary key end values ​​can be calculated by calculating the average value between the two primary key end values. When the average value between the two primary key end values ​​has a decimal part, the primary key sharding point can be obtained by rounding down.

[0073] In some embodiments, multiple primary key sharding points may be determined between two primary key end values, and the sharding may be performed by three-partitioning, four-partitioning, etc.

[0074] In this embodiment, a primary key sharding point is determined between two primary key end values ​​as an example.

[0075] Figure 5 A primary key sharding point division diagram provided in the embodiment of the present application, refer to Figure 5 As shown, since the primary key area division method of the first data table and the second data table is completely consistent, the first data table is used as an example here.

[0076] Assume that the global primary key information of the first data table is [0, 19]. During the first-level split, the current first primary key end value is 0, and the current second primary key end value is 19. Through the above-mentioned averaging method, it can be obtained that the primary key sharding point between the current first primary key end value 0 and the current second primary key end value 19 is 9.

[0077] The second level split is performed based on the result of the first level split. During the second level split, the current first primary key end value can be 0 or 9. When the current first primary key end value is 0, the current second primary key end value is 9. When the current first primary key end value is 9, the current second primary key end value is 19. If the current first primary key end value is 0 and the current second primary key end value is 9, the primary key fragmentation point between the current first primary key end value 0 and the current second primary key end value 9 is 4; if the current first primary key end value is 9 and the current second primary key end value is 19, the primary key fragmentation point between the current first primary key end value 9 and the current second primary key end value 19 is 14.

[0078] By analogy, at the third level of splitting, the current first primary key end value can be 0, 4, 9, or 14; when the current first primary key end value is 0, the current second primary key end value can be 4; when the current first primary key end value is 4, the current second primary key end value can be 9; when the current first primary key end value is 9, the current second primary key end value can be 14; when the current first primary key end value is 14, the current second primary key end value can be 19.

[0079] The split stop condition can be determined according to the number of required primary key sharding points or the number of primary key regions to be split.

[0080] It can be seen that through the first-level splitting, one primary key sharding point can be obtained, thus obtaining two primary key regions; through the second-level splitting, two primary key sharding points can be obtained, thus obtaining four primary key regions.

[0081] Of course, in some embodiments, the primary key region may not be split by determining the primary key sharding point, but may be split by sliding from front to back in accordance with the global primary key information and the order of the primary keys. For example, if five primary keys are divided into one region, then primary keys 0-4 may be divided into one primary key region, primary keys 5-9 may be divided into one primary key region, and so on.

[0082] Figure 6 A flowchart of another data consistency verification method provided in an embodiment of the present application; refer to Figure 6 As shown, in the above step S402, determining at least one primary key sharding point according to the first primary key end value and the second primary key end value may include: S601. If the types of the first primary key end value and the second primary key end value are string types, determine the target characters at the corresponding positions in the primary key fragmentation points to be generated according to the characters at the corresponding positions in the first primary key end value and the second primary key end value.

[0083] When the primary key is of string type, the primary key sharding point between any two primary key end values ​​can be calculated by calculating the middle characters of the characters at corresponding positions between the two strings to generate a new string, thereby obtaining the primary key sharding point.

[0084] Among them, in the calculation of the string type, in the first level calculation, the current first primary key end value and the current second primary key end value can be respectively taken as the maximum primary key value and the minimum primary key value in the global primary key information. The maximum primary key value and the minimum primary key value can be determined from the global primary key information by calculating the size of the string.

[0085] For example, assuming that the global primary key information is arranged in ascending order as follows: [abc; bce; bde; efg; ehf]; then, based on the character size, it can be determined that the maximum primary key value is ehf and the minimum primary key value is abc. Therefore, it can be determined that the current first primary key end value is abc and the current second primary key end value is ehf.

[0086] Then, the character a in the current first primary key end value corresponds to the character e in the current second primary key end value, the character b in the current first primary key end value corresponds to the character h in the current second primary key end value, and the character c in the current first primary key end value corresponds to the character f in the current second primary key end value.

[0087] Then, according to the character a in the current first primary key end value and the character e in the current second primary key end value, the middle character between the character a and the character e can be determined. Since the middle characters of the character a and the character e include: b, c, d, respectively, the character c can be taken as the character at the first position in the primary key fragmentation point to be generated; similarly, according to the character b in the current first primary key end value and the character h in the current second primary key end value, the middle character between the character b and the character h can be determined. Since the middle characters of the character b and the character h include: c, d, e, f, g, respectively, the character e can be taken as the character at the second position in the primary key fragmentation point to be generated; according to the character c in the current first primary key end value and the character f in the current second primary key end value, the middle character between the character c and the character f can be determined. Since the middle characters of the character c and the character f include: d, e, respectively, in the downward way, the character d can be taken as the character at the third position in the primary key fragmentation point to be generated.

[0088] S602: Obtain at least one primary key fragmentation point according to the target characters at the corresponding positions in the primary key fragmentation points to be generated.

[0089] Based on the above determined character c at the first position of the primary key fragmentation point to be generated, the character e at the second position of the primary key fragmentation point to be generated, and the character d at the third position of the primary key fragmentation point to be generated, a primary key fragmentation point can be obtained: ced.

[0090] In some embodiments, the global primary key information can be divided into multiple primary key areas according to the current first primary key end value abc, the primary key sharding point ced, and the current second primary key end value ehf, wherein the primary key between abc-ced is divided into one primary key area, and the primary key between ced-ehf is divided into another primary key area.

[0091] By analogy, during the second level division, the current first primary key end value can be abc or ced; when the current first primary key end value is abc, the current second primary key end value can be ced; when the current first primary key end value is ced, the current second primary key end value can be ehf.

[0092] Figure 7 A flowchart of another data consistency verification method provided in an embodiment of the present application; refer to Figure 7 As shown, optionally, in the above step S102, according to the primary key areas corresponding to each verification task and the primary key sorting method in each primary key area, sliding to determine the current first verification primary key of the first data table and the current second verification primary key of the second data table in each primary key area may include: S701: Perform primary key synchronous sliding on a first data table and a second data table according to a primary key region and a primary key sorting method in the primary key region.

[0093] The primary key can be sorted in ascending order or descending order. The primary key can be slid from small to large in the primary key area in ascending order, or can be slid from large to small in the primary key area in descending order.

[0094] In this embodiment, an ascending sliding mode is used to describe the situation, and two cursors are initialized, and the primary keys of the first data table and the second data table are slid synchronously.

[0095] S702: Determine the current primary key value of the first data table and the current primary key value of the second data table according to the current sliding position.

[0096] Initially, the two cursors point to the first row of data in the first data table and the first row of data in the second data table in the primary key area respectively. Since each row of data corresponds to a unique primary key, the value of the primary key corresponding to the current position of the cursor can be determined.

[0097] As the cursor continues to slide, each time the cursor pointing to the first data table slides, the cursor pointing to the second data table will also slide synchronously. Therefore, the current primary key value of the first data table and the current primary key value of the second data table will be determined correspondingly at each current sliding position.

[0098] S703: Determine a current first verification primary key of the first data table and a current second verification primary key of the second data table according to the current primary key value of the first data table and the current primary key value of the second data table.

[0099] In some embodiments, due to the possibility of data loss during the data migration process, the primary keys of the first data table and the second data table do not completely correspond. In this case, if the data is directly compared with the row where the current primary key corresponding to the current position of the cursor is slid, it will cause deviations in the data comparison and inaccurate comparison results.

[0100] Based on this, in this embodiment, according to the position to which the cursor currently slides, the current primary key value of the first data table and the current primary key value of the second data table can be determined first, and then based on the current primary key value, another judgment can be made to determine the current first verification primary key of the first data table and the current second verification primary key of the second data table.

[0101] When actually performing data comparison, the data in the row where the current first verification primary key of the first data table is located is compared with the data in the row where the current second verification primary key of the second data table is located.

[0102] Optionally, in the above step S703, the current first verification primary key of the first data table and the current second verification primary key of the second data table are determined according to the value of the current primary key of the first data table and the value of the current primary key of the second data table, including: if the value of the current primary key of the first data table is the same as the value of the current primary key of the second data table, then the current primary key of the first data table is used as the current first verification primary key of the first data table, and the current primary key of the second data table is used as the current second verification primary key of the second data table.

[0103] In the first case, if the value of the current primary key in the first data table to which the cursor currently slides is equal to the value of the current primary key in the second data table to which the cursor currently slides, the current primary key of the first data table can be directly used as the current first verification primary key of the first data table, and the current primary key of the second data table can be used as the current second verification primary key of the second data table.

[0104] At this time, the data of the row where the current first verification primary key is located can be directly read from the first data table, and the data of the row where the current second verification primary key is located can be directly read from the second data table, and the data of the row where the current first verification primary key is located can be compared one by one with the data of each corresponding column in the row where the current second verification primary key is located.

[0105] Figure 8 A flow chart of another data consistency verification method provided in an embodiment of the present application; optionally, in step S102, according to the value of the current primary key of the first data table and the value of the current primary key of the second data table, determining the current first verification primary key of the first data table and the current second verification primary key of the second data table, including: S801. If the value of the current primary key of the first data table is greater than the value of the current primary key of the second data table, use the next primary key of the current primary key of the second data table as the new current primary key; and mark the row where the current primary key of the second data table is located as a difference.

[0106] In the second case, if the value of the current primary key in the first data table to which the cursor currently slides is greater than the value of the current primary key in the second data table to which the cursor currently slides, it means that the row where the current primary key is located in the second data table is a unique row. At this time, the cursor pointing to the second data table will continue to slide backward once, sliding to the next primary key of the current primary key of the second data table to obtain a new current primary key; at the same time, the data in the row where the current primary key of the second data table is located needs to be differentially marked to mark it as a unique row.

[0107] S802: Use the current primary key of the first data table as the current first verification primary key of the first data table, and use the new current primary key of the second data table as the current second verification primary key of the second data table.

[0108] In this case, the current primary key of the first data table is used as the current first verification primary key of the first data table, and the new current primary key of the second data table is used as the current second verification primary key of the second data table.

[0109] Fig. 9 A flow chart of another data consistency verification method provided in an embodiment of the present application; optionally, in step S102, according to the value of the current primary key of the first data table and the value of the current primary key of the second data table, determining the current first verification primary key of the first data table and the current second verification primary key of the second data table includes: S901. If the value of the current primary key of the first data table is smaller than the value of the current primary key of the second data table, use the next primary key of the current primary key of the first data table as the new current primary key; and mark the row where the current primary key of the first data table is located as a difference.

[0110] In the third case, if the value of the current primary key in the first data table to which the cursor currently slides is less than the value of the current primary key in the second data table to which the cursor currently slides, it means that the row where the current primary key is located in the first data table is a unique row. At this time, the cursor pointing to the first data table will continue to slide backward once, sliding to the next primary key of the current primary key of the first data table to obtain a new current primary key; at the same time, the data in the row where the current primary key of the first data table is located needs to be differentially marked to mark it as a unique row.

[0111] S902: Use the new current primary key of the first data table as the current first verification primary key of the first data table, and use the current primary key of the second data table as the current second verification primary key of the second data table.

[0112] In this case, the new current primary key of the second data table is used as the current first verification primary key of the first data table, and the current primary key of the second data table is used as the current second verification primary key of the second data table.

[0113] Fig.10 A schematic diagram of a primary key display provided in an embodiment of the present application, referring to Figure 1 As shown, the primary key of the first data table ranges from 0 to 19, and the primary key of the second data table ranges from 0 to 19.

[0114] According to the above primary key sliding method, initially, the cursor points to the current primary key 0 of the first data table and the current primary key 0 of the second data table respectively. Since the value of the current primary key of the first data table is the same as the value of the current primary key of the second data table at this time, the row data of the primary key 0 of the first data table and the row data of the primary key 0 of the second data table are directly read for comparison. After the row data of the primary key 0 of the first data table and the row data of the primary key 0 of the second data table are completely compared, the cursor continues to slide backward, and the cursor points to the current primary key 1 of the first data table and the current primary key 1 of the second data table respectively. Similarly, since the value of the current primary key of the first data table is the same as the value of the current primary key of the second data table, the row data of the primary key 1 of the first data table and the row data of the primary key 0 of the second data table are directly read. The data in the row where the primary key 1 of the data table is located is compared; the cursor is gradually sliding backward in sequence, and when the cursor points to the current primary key 4 of the first data table and the current primary key 4 of the second data table respectively, after the comparison of the data in the row where the primary key 4 of the first data table is located with the data in the row where the primary key 4 of the second data table is located is completed, the cursor points to the current primary key 6 of the first data table and the current primary key 5 of the second data table. Since the value of the current primary key of the first data table is greater than the value of the current primary key of the second data table, at this time, the row where the current primary key 5 of the second data table is located is marked as a unique row, and the cursor pointing to the second data table continues to slide backward to obtain the new current primary key 6 of the second data table, thereby reading the data in the row where the primary key 6 of the first data table is located with the data in the row where the primary key 6 of the second data table is located for comparison.

[0115] Similarly, when the cursor points to the current primary key 12 of the first data table and the current primary key 12 of the second data table respectively, after the comparison of the row data of the primary key 12 of the first data table and the row data of the primary key 12 of the second data table is completed, the cursor points to the current primary key 13 of the first data table and the current primary key 14 of the second data table. Since the value of the current primary key of the first data table is less than the value of the current primary key of the second data table, at this time, the row where the current primary key 13 of the first data table is located is marked as a unique row, and the cursor pointing to the first data table continues to slide backward to obtain the new current primary key 13 of the first data table, thereby reading the row data where the primary key 13 of the first data table is located and comparing it with the row data where the primary key 13 of the second data table is located.

[0116] Fig.11 A flow chart of another data consistency verification method provided by an embodiment of the present application; Optionally, in step S104, according to the order of each first column data to be compared and the order of each second column data to be compared, each first column data to be compared is compared with the corresponding second column data to be compared in turn, and a comparison result of each first column data to be compared with the corresponding second column data to be compared is obtained, including: S1101 . Determine current first column data to be compared according to the order of each first column data to be compared.

[0117] Referring to Table 1 and Table 2, assuming that the current first verification primary key of the first data table is primary key 2 and the current second verification primary key of the second data table is primary key 2, then the first column data to be compared in the row where the current first verification primary key is located are data 6, data 7, data 8, data 9 and data 10 respectively; the second column data to be compared in the row where the current second verification primary key is located are data 21, data 22, data 23, data 24 and data 25 respectively.

[0118] The current first column data to be compared may be determined in sequence according to the column order of each first column data to be compared.

[0119] S1102 : Determine the current second column data to be compared corresponding to the current first column data to be compared according to the order of each second column data to be compared.

[0120] Similarly, the current second column data to be compared corresponding to the current first column data to be compared may be determined in sequence according to the column order of each second column data to be compared.

[0121] It can be seen that when the current first column data to be compared is data 6, the corresponding current second column data to be compared is data 21; when the current first column data to be compared is data 7, the corresponding current second column data to be compared is data 22; when the current first column data to be compared is data 8, the corresponding current second column data to be compared is data 23; when the current first column data to be compared is data 9, the corresponding current second column data to be compared is data 24; when the current first column data to be compared is data 10, the corresponding current second column data to be compared is data 25.

[0122] S1103 , compare the current first column data to be compared with the current second column data to be compared, and obtain a comparison result between the current first column data to be compared and the current second column data to be compared.

[0123] Based on this, data 6 can be compared with data 21 to determine whether data 6 and data 21 are equal. Similarly, data 7 and data 22 can be compared to determine whether data 7 and data 22 are equal. This process continues until all corresponding column data are compared to obtain the data comparison result of the row where the primary key 2 of the first data table is located and the row where the primary key 2 of the second data table is located.

[0124] Among them, when all the current first column data to be compared in the row where the primary key 2 of the first data table is located are completely equal to the current second column data to be compared corresponding to the row where the primary key 2 of the second data table is located, it is considered that the data in the row where the primary key 2 of the first data table is located is completely equal to the data in the row where the primary key 2 of the second data table is located; otherwise, if there is a current first column data to be compared that is not equal to the corresponding current second column data to be compared, it is considered that there is a difference between the data in the row where the primary key 2 of the first data table is located and the data in the row where the primary key 2 of the second data table is located.

[0125] That is, only when data 6 is equal to data 21, data 7 is equal to data 22, data 8 is equal to data 23, data 9 is equal to data 24, and data 10 is equal to data 25, it is considered that the data in the row where the primary key 2 of the first data table is located is completely equal to the data in the row where the primary key 2 of the second data table is located.

[0126] It is worth noting that, for any verification task, when the cursor slides to the end in the primary key area, the current verification task is considered to be completed.

[0127] For example, if the primary key area corresponding to the verification task is 0-9, when the cursor slides to 9, after the data of the row where the primary key 9 of the first data table is located is compared with the row where the primary key 9 of the second data table is located, the verification task is considered to be completed.

[0128] In some embodiments, the verification results of each verification task may be combined to obtain a complete comparison result of the first data table and the second data table.

[0129] In summary, the data consistency verification method provided by this embodiment includes: creating at least one verification task between the first data table in the source database and the second data table in the target database according to the primary key sharding information; executing various verification tasks in parallel, and slidingly determining the current first verification primary key of the first data table in each primary key area and the current second verification primary key of the second data table according to the primary key area corresponding to each verification task and the primary key sorting method in each primary key area; obtaining multiple first column data to be compared in the row where the current first verification primary key is located from the first data table, and obtaining multiple second column data to be compared in the row where the current second verification primary key is located from the second data table; according to the order of each first column data to be compared and the order of each second column data to be compared, each first column data to be compared is compared with the corresponding second column data to be compared in turn, and obtaining the comparison result of each first column data to be compared with the corresponding second column data to be compared. This method creates multiple verification tasks through primary key sharding information, thereby dividing the global verification task into multiple subtasks for parallel execution, which can improve the efficiency of data verification. During the data verification process, based on the principle that the primary keys of the database are arranged in order, the primary keys can be slid in sequence to quickly and accurately determine the row where the data to be verified is located. Based on the row where the data is located, the data in each column of the row can be read from the data table in real time, so that the data in each column of the data row can be compared with each other to complete data verification. The efficiency of data reading and data verification is effectively improved.

[0130] The following describes the devices, equipment, storage media, etc. used to execute the data consistency verification method provided by this application. The specific implementation process and technical effects are described above and will not be repeated below.

[0131] Fig.12 Schematic diagram of a data consistency verification device provided in an embodiment of the present application, the functions implemented by the data consistency verification device correspond to the steps performed by the above method. Fig.12 As shown, the device may include: a creation module 120, a determination module 121, an acquisition module 122 and a processing module 123; A creation module 120 is used to create at least one verification task between a first data table in a source database and a second data table in a target database according to the primary key sharding information, each verification task being used to compare the consistency of data in a corresponding primary key region in the first data table and the second data table; The determination module 121 is used to execute various verification tasks in parallel, and according to the primary key areas corresponding to the various verification tasks and the primary key sorting methods in each primary key area, slide to determine the current first verification primary key of the first data table and the current second verification primary key of the second data table in each primary key area; The acquisition module 122 is used to acquire multiple first to-be-compared column data of the row where the current first verification primary key is located from the first data table, and acquire multiple second to-be-compared column data of the row where the current second verification primary key is located from the second data table; The processing module 123 is used to compare each first column data to be compared with the corresponding second column data to be compared according to the order of each first column data to be compared and the order of each second column data to be compared, so as to obtain a comparison result between each first column data to be compared and the corresponding second column data to be compared.

[0132] Optionally, a creation module 120 is specifically used to determine the primary key sharding information by adopting the primary key sharding strategy according to the global primary key information; the global primary key information is used to characterize the value range of the primary key of the first data table and the second data table; According to the primary key sharding information, primary key regions are divided for the first data table and the second data table, and according to the primary key region division result, at least one verification task between the first data table and the second data table is created.

[0133] Optionally, the determination module 121 is specifically configured to determine a current first primary key end value and a current second primary key end value according to the global primary key information; Determine at least one primary key sharding point according to the first primary key end value and the second primary key end value; According to each primary key sharding point, primary key sharding information corresponding to the current first primary key end value and the current second primary key end value is determined.

[0134] Optionally, the determination module 121 is specifically configured to round down the average value of the first primary key end value and the second primary key end value to obtain at least one primary key sharding point if the types of the first primary key end value and the second primary key end value are digital types.

[0135] Optionally, the determination module 121 is specifically configured to determine, if the types of the first primary key end value and the second primary key end value are string types, target characters at corresponding positions in the primary key fragmentation point to be generated according to characters at corresponding positions in the first primary key end value and the second primary key end value; At least one primary key fragmentation point is obtained according to the target characters at the corresponding positions in the primary key fragmentation points to be generated.

[0136] Optionally, the determination module 121 is specifically configured to perform primary key synchronous sliding on the first data table and the second data table according to the primary key region and the primary key sorting method in the primary key region; According to the current sliding position, respectively determine the value of the current primary key of the first data table and the value of the current primary key of the second data table; According to the value of the current primary key of the first data table and the value of the current primary key of the second data table, a current first verification primary key of the first data table and a current second verification primary key of the second data table are determined.

[0137] Optionally, the determination module 121 is specifically configured to use the current primary key of the first data table as the current first verification primary key of the first data table, and use the current primary key of the second data table as the current second verification primary key of the second data table if the value of the current primary key of the first data table is the same as the value of the current primary key of the second data table; If the value of the current primary key of the first data table is greater than the value of the current primary key of the second data table, the next primary key of the current primary key of the second data table is used as the new current primary key; and the row where the current primary key of the second data table is located is marked as a difference; Using the current primary key of the first data table as the current first verification primary key of the first data table, and using the new current primary key of the second data table as the current second verification primary key of the second data table; If the value of the current primary key of the first data table is less than the value of the current primary key of the second data table, the next primary key of the current primary key of the first data table is used as the new current primary key; and the row where the current primary key of the first data table is located is marked as a difference; The new current primary key of the first data table is used as the current first verification primary key of the first data table, and the current primary key of the second data table is used as the current second verification primary key of the second data table.

[0138] Optionally, the processing module 123 is specifically configured to determine the current first column data to be compared according to the order of each first column data to be compared; Determine the current second column data to be compared corresponding to the current first column data to be compared according to the order of each second column data to be compared; The current first column data to be compared and the current second column data to be compared are compared to obtain a comparison result between the current first column data to be compared and the current second column data to be compared.

[0139] The above-mentioned device is used to execute the method provided by the aforementioned embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0140] The above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital singnal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0141] The above modules can be connected or communicated with each other via a wired connection or a wireless connection. The wired connection may include a metal cable, an optical cable, a hybrid cable, etc., or any combination thereof. The wireless connection may include a connection in the form of a LAN, a WAN, Bluetooth, a ZigBee, or NFC, or any combination thereof. Two or more modules may be combined into a single module, and any one module may be divided into two or more units. Those skilled in the art will clearly understand that for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application.

[0142] Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, which may be a computing device with data processing capabilities.

[0143] The device includes: a processor 801 and a storage medium 802 .

[0144] The storage medium 802 is used to store programs, and the processor 801 calls the programs stored in the storage medium 802 to execute the above method embodiment. The specific implementation method and technical effect are similar and will not be repeated here.

[0145] Among them, the storage medium 802 stores program code, and when the program code is executed by the processor 801, the processor 801 executes various steps in the data consistency verification method according to various exemplary embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0146] The processor 801 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and may implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0147] The storage medium 802, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The storage medium may include at least one type of storage medium, for example, a flash memory, a hard disk, a multimedia card, a card-type storage medium, a random access storage medium (Random Access Memory, RAM), a static random access storage medium (Static Random Access Memory, SRAM), a programmable read-only storage medium (Programmable Read Only Memory, PROM), a read-only storage medium (Read Only Memory, ROM), an electrically erasable programmable read-only storage medium (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic storage medium, a disk, an optical disk, and the like. The storage medium is any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The storage medium 802 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0148] Optionally, the present application also provides a program product, such as a computer-readable storage medium, including a program, which is used to execute the above method embodiment when executed by a processor.

[0149] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0150] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0151] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0152] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only storage medium (English: Read-Only Memory, referred to as: ROM), random access storage medium (English: Random Access Memory, referred to as: RAM), disk or optical disk and other media that can store program codes.

Claims

1. A data consistency verification method, characterized in that: include: At least one verification task is created between the first data table in the source database and the second data table in the target database according to the primary key sharding information, each verification task is used to compare the consistency of data in the corresponding primary key regions of the first data table and the second data table; Execute various verification tasks in parallel, and according to the primary key areas corresponding to the various verification tasks and the primary key sorting methods in the primary key areas, slide to determine the current first verification primary key of the first data table and the current second verification primary key of the second data table in each primary key area; Acquire a plurality of first to-be-compared column data of the row where the current first verification primary key is located from the first data table, and acquire a plurality of second to-be-compared column data of the row where the current second verification primary key is located from the second data table; According to the order of each of the first column data to be compared and the order of each of the second column data to be compared, each of the first column data to be compared is compared with the corresponding second column data to be compared in turn to obtain a comparison result between each of the first column data to be compared and the corresponding second column data to be compared.

2. The method according to claim 1, characterized in that The step of creating at least one verification task between the first data table in the source database and the second data table in the target database according to the primary key sharding information includes: According to the global primary key information, the primary key sharding strategy is adopted to determine the primary key sharding information; the global primary key information is used to characterize the value range of the primary key of the first data table and the second data table; According to the primary key sharding information, primary key regions are divided for the first data table and the second data table, and according to the primary key region division result, at least one verification task is created between the first data table and the second data table.

3. The method according to claim 2, characterized in that The method of determining the primary key sharding information by adopting the primary key sharding strategy according to the global primary key information includes: Determine the current first primary key end value and the current second primary key end value according to the global primary key information; Determine at least one primary key sharding point according to the first primary key end value and the second primary key end value; According to each primary key sharding point, primary key sharding information corresponding to the current first primary key end value and the current second primary key end value is determined.

4. The method according to claim 3, characterized in that The determining at least one primary key sharding point according to the first primary key end value and the second primary key end value includes: If the types of the first primary key end value and the second primary key end value are digital types, the average value of the first primary key end value and the second primary key end value is rounded down to obtain the at least one primary key fragmentation point.

5. The method according to claim 3, characterized in that: The determining at least one primary key sharding point according to the first primary key end value and the second primary key end value includes: If the types of the first primary key end value and the second primary key end value are string types, then according to the characters at the corresponding positions in the first primary key end value and the second primary key end value, respectively determine the target characters at the corresponding positions in the primary key fragmentation point to be generated; The at least one primary key fragmentation point is obtained according to the target characters at the corresponding positions in the primary key fragmentation points to be generated.

6. The method according to claim 1, characterized in that The step of slidingly determining the current first verification primary key of the first data table and the current second verification primary key of the second data table in each primary key area according to the primary key area corresponding to each verification task and the primary key sorting method in each primary key area includes: According to the primary key area and the primary key sorting method within the primary key area, the primary key synchronous sliding is performed on the first data table and the second data table; Determine the current primary key value of the first data table and the current primary key value of the second data table according to the current sliding position; According to the value of the current primary key of the first data table and the value of the current primary key of the second data table, a current first verification primary key of the first data table and a current second verification primary key of the second data table are determined.

7. The method according to claim 6, characterized in that The determining, according to the value of the current primary key of the first data table and the value of the current primary key of the second data table, the current first verification primary key of the first data table and the current second verification primary key of the second data table comprises: If the value of the current primary key of the first data table is the same as the value of the current primary key of the second data table, the current primary key of the first data table is used as the current first verification primary key of the first data table, and the current primary key of the second data table is used as the current second verification primary key of the second data table; If the value of the current primary key of the first data table is greater than the value of the current primary key of the second data table, the next primary key of the current primary key of the second data table is used as the new current primary key; and the row where the current primary key of the second data table is located is marked as a difference; the current primary key of the first data table is used as the current first verification primary key of the first data table, and the new current primary key of the second data table is used as the current second verification primary key of the second data table; If the value of the current primary key of the first data table is less than the value of the current primary key of the second data table, the next primary key of the current primary key of the first data table is used as the new current primary key; and the row where the current primary key of the first data table is located is marked as a difference; the new current primary key of the first data table is used as the current first verification primary key of the first data table, and the current primary key of the second data table is used as the current second verification primary key of the second data table.

8. The method according to claim 1, characterized in that The step of comparing each of the first column data to be compared with the corresponding second column data to be compared in sequence according to the order of each of the first column data to be compared and the order of each of the second column data to be compared, and obtaining a comparison result between each of the first column data to be compared and the corresponding second column data to be compared, comprises: Determine the current first column data to be compared according to the order of each of the first column data to be compared; Determine, according to the order of each of the second to-be-compared column data, the current second to-be-compared column data corresponding to the current first to-be-compared column data; The current first column data to be compared and the current second column data to be compared are compared to obtain a comparison result between the current first column data to be compared and the current second column data to be compared.

9. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to implement the data consistency verification method as described in any one of claims 1 to 8.

10. A computer program product, characterized in that The computer program product comprises a computer program code. When the computer program code is executed by a computing device, the computing device executes the data consistency verification method according to any one of claims 1 to 8.

Images