Data consistency verification method and system suitable for data synchronization

Abstract

The invention relates to the technical field of data processing and synchronization, in particular to a data consistency verification method and system suitable for data synchronization, which comprises the following steps: setting a configuration file and a verification rule, establishing a source end and destination end database connection according to the configuration file, obtaining a database table structure and index information, and verifying the data consistency. Verifying the consistency of the structures and attributes of the library tables at the two ends, selecting at least one of the following verification modes based on the configured verification mode, and performing data completion or verification termination according to the verification result; the method has the beneficial effects that by optimizing the data processing flow and adopting more advanced algorithms and technologies, the unnecessary data transmission quantity is reduced, and the data synchronization speed is increased. For example, only varying data portions, rather than the entire database or file, are transferred using incremental synchronization techniques, thereby significantly reducing bandwidth usage and synchronization time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing and synchronization, in particular to a data consistency verification method and system suitable for data synchronization. BACKGROUND

[0002] In today's digital era, fast and accurate data transmission is crucial for enterprise operations. With the development of information technology and the expansion of enterprise business scope, the demand for data exchange between different systems is growing. Structured data synchronization, as a key technology to ensure data consistency, plays a vital role in data integration, migration, backup and other scenarios. However, traditional methods often face low efficiency and inaccurate consistency verification when dealing with large-scale data sets.

[0003] Traditionally, structured data synchronization usually relies on simple copy-paste or timestamp-based incremental update strategies. These methods can meet the needs of small-scale data sets, but their limitations gradually emerge when faced with massive data. For example, the simple copy-paste method cannot effectively identify the differences between source data and target data, resulting in the transmission of a large amount of redundant data; while the timestamp-based method is limited by the time precision of the database management system, which may lead to data loss or duplication. In addition, when it comes to cross-platform or heterogeneous system data synchronization, the above problems become more complex due to the differences in data format and storage mechanisms between systems.

[0004] In recent years, in order to cope with these challenges, researchers have proposed a series of solutions, such as using hash algorithms for data block comparison, using distributed computing frameworks to speed up data processing, etc. Although these technologies have improved the efficiency and accuracy of data synchronization to some extent, they still have some shortcomings. For example, hash algorithms have high computational overhead when dealing with large files, and are difficult to handle dynamic changing data; distributed computing frameworks can speed up the processing process, but for small and medium-sized data sets, the configuration and maintenance costs are high.

[0005] Therefore, how to design a new structured data synchronization scheme that can guarantee high efficiency and ensure data consistency has become a technical problem to be solved. SUMMARY

[0006] The purpose of the present application is to provide a data consistency verification method and system suitable for data synchronization to solve the problems raised in the background technology.

[0007] To achieve the above purpose, the present application provides the following technical scheme: a data consistency verification method suitable for data synchronization, comprising the following steps:

[0008] Set configuration file and check rule, define data source, check object, ignore object, log level and check mode parameter;

[0009] According to the configuration file, the source end and the destination database connection are established, the library table structure and index information are obtained, and the consistency of the library table structure and the properties of the two ends is checked.

[0010] Based on the configured check mode, at least one of the following check modes is selected: count mode: count the number of records of the source end and the destination table, and output the tables with consistent and inconsistent record numbers; rows mode: calculate data check code row by row or in blocks, locate data differences by comparing check codes, and generate data completion SQL statements; sample mode: sample source end data according to a proportion, and check the consistency of the check codes of the sampled data and the corresponding data of the destination end;

[0011] According to the check result, data completion or termination of the check is performed, and timed check is supported to realize incremental data synchronization.

[0012] Preferably, the rows mode includes the following steps:

[0013] Row-by-row check: the table fields are spliced in order to generate MD5 check codes and primary keys for each row of data; the check code sets of the source end and the destination end are compared to locate the different data rows; and delete or insert statements are generated according to the differences to perform data completion in the destination end;

[0014] Block check: sample and sort data according to the primary key, and divide the data interval; generate aggregated check codes for the data in each interval, and compare the interval check codes of the source end and the destination end; for the intervals with inconsistent check codes, return to the row-by-row check mode to locate the differences.

[0015] Preferably, the configuration file includes the following parameters: data source configuration: define the source end and the destination database connection string; check object configuration: specify the tables to be checked and the tables to be ignored, supporting wildcard matching; check rule configuration: including check mode (count / rows / sample), sampling rate, block size, whether to check non-indexed tables, and whether to strictly match the table structure; log configuration: define log level and output file name; repair configuration: specify the data completion SQL file path and the maximum allowed number of difference records.

[0016] Preferably, the data check for non-indexed tables includes the following steps: if the number of table records is less than a threshold, read the source end and the destination data to the check server for comparison; if the number of table records is greater than the threshold, select a block field according to the field randomness, and sample to generate a data interval; read the source end and the destination data to the check server according to the interval, and compare the consistency of the data in the interval; the field randomness is determined by the number of field value groups, and the more groups, the higher the randomness.

[0017] Preferably, the data consistency check between homogenous or heterogeneous databases only requires the source and destination databases to support the following functions: a function for generating data check codes; a data sampling and sorting function; a field value grouping statistics function; and a check process implemented by standard SQL statements, independent of specific database types.

[0018] A system for a data consistency check method for data synchronization, comprising:

[0019] A configuration management module for defining and parsing data source connection parameters, check object rules, log configuration and check mode parameters, establishing source and destination database connections, and obtaining library table structures and index information;

[0020] A check rule engine for selecting the following submodules according to the configured check mode (count / rows / sample): a record number check submodule for counting the record numbers of the source and destination tables and outputting consistency results; a row-level check submodule for generating data check codes row by row or in blocks and locating difference data by comparing the check codes; and a sampling check submodule for sampling source data in proportion and checking the consistency of the sampled data and corresponding data in the destination;

[0021] A data completion module for generating difference repair SQL statements according to the check results and executing data insertion or deletion operations in the destination;

[0022] A timing scheduling module for supporting periodic triggering of check tasks and realizing incremental data synchronization;

[0023] A log and monitoring module for recording check process logs, counting difference data amounts, and terminating the check when the threshold is exceeded.

[0024] Preferably, the row-level check submodule comprises:

[0025] A row-by-row check unit for concatenating table fields in order to form strings, generating MD5 check codes and primary keys for each row of data, locating difference data rows by comparing the check code sets of the source and destination, and generating INSERT or DELETE statements for the difference data and transmitting them to the data completion module for execution;

[0026] A block check unit for sampling and sorting data according to primary keys, dividing data intervals, generating aggregated check codes for the data in each interval, comparing the interval check codes of the source and destination, and calling the row-by-row check unit to locate differences for intervals with inconsistent check codes;

[0027] A check code generation unit for supporting MD5 conversion of binary data, NULL value replacement, date and time stringification, and floating point precision control.

[0028] Preferably, the configuration management module comprises:

[0029] Data source configuration unit: store source and destination database connection strings, support MySQL, Oracle relational database;

[0030] Check object configuration unit: define the tables to be checked and the tables to be ignored, support wildcard matching and case sensitivity setting;

[0031] Check rule configuration unit: including check mode (count / rows / sample), sampling rate, block size, whether to check the table without index, whether to strictly match the table structure;

[0032] Log configuration unit: define log level (debug / info / warn / error) and output file path;

[0033] Repair configuration unit: specify the data filling SQL file storage path and the maximum allowed difference record number threshold.

[0034] Preferably, the data check for the table without index includes:

[0035] Full comparison unit: when the number of table records is less than the preset threshold, read the source and destination data to the check server for comparison;

[0036] Block comparison unit: when the number of table records is greater than the preset threshold, execute the following steps: select a block field randomly according to field randomness, sample data interval; read the source and destination data to the check server according to the interval, compare the data consistency in the interval; field randomness is determined by the number of field value groups, the more groups, the higher the randomness; Randomness analysis unit: support field value distribution statistics through SQL statement, auxiliary determination of block field.

[0037] Preferably, the system is suitable for data consistency check between homogeneous or heterogeneous databases, only need to meet the following conditions: the source and destination databases support the function of generating data check code; the source and destination databases support data sampling and sorting function; the source and destination databases support field value grouping statistics function; the check process is realized through standard SQL statement, and does not depend on specific database type or version.

[0038] Compared with the prior art, the beneficial effects of the present application are:

[0039] The method and system for data consistency check suitable for data synchronization provided by the present application optimize the data processing process, use more advanced algorithms and technologies, reduce unnecessary data transmission amount, and improve the speed of data synchronization. For example, using incremental synchronization technology to transmit only the changed data part, not the entire database or file, thereby significantly reducing bandwidth usage and synchronization time.

[0040] The data comparison and verification mechanism is introduced to ensure the accurate matching between the source data and the target data. The mechanism can not only identify and correct the data differences, but also effectively deal with the dynamically changing data environment, and ensure the high consistency of the data between different systems.

[0041] A general data conversion and adaptation interface is designed to enable the application to support various data formats and storage media, and to realize the seamless connection of data between heterogeneous systems. Regardless of the differences between the data source and the target system, smooth data migration and synchronization can be realized through the interface. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 The method flowchart of the application. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution of the application clear, complete and the advantages more clear and obvious, the embodiments of the application are further described in detail below in combination with the drawings. It should be understood that the specific embodiments described herein are part of the embodiments of the application, not all embodiments, and are used to explain the embodiments of the application, and do not limit the embodiments of the application. All other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0044] Embodiment one, please refer to Figure 1 The application provides a technical solution: a method for data consistency verification suitable for data synchronization, comprising the following steps:

[0045] I. Setting configuration file and verification rule

[0046] In this part, the basic operation rules and necessary configuration information are defined, including data source, tables to be verified and tables to be ignored, case matching, operation log, etc.

[0047] The function of this part is to establish the source and destination database connection, and obtain the library table structure of the source and destination databases and whether there is an index, etc. Whether the library table structure of both ends is consistent, and whether the table attributes are consistent.

[0048] The code is as follows:

[0049] [DSN]

[0050] # Define the source and destination database connection method

[0051] srcdsn = mysql: user: password@tcp (ip: port) / dbname? charset = utf8

[0052] dstdsn = mysql : user : password @ tcp ( ip : port ) / dbname? charset = utf8

[0053] [SCHEMA]

[0054] # Define the object to be checked and the naming matching method

[0055] # The object to be checked

[0056] tables = db1. *

[0057] # Ignore the object to be checked

[0058] ignore-tables = db1. abcd

[0059] [LOG]

[0060] # Define the running log level / debug / info / warn / error

[0061] level = info

[0062] log = log file name

[0063] [RULE]

[0064] # Whether to detect tables without indexes

[0065] check_no_index = no

[0066] # Whether to match case table names: yes for matching, no for uppercase matching

[0067] lower_case_Name = yes

[0068] # Whether to strictly judge the structure, yes means that the table structure is completely consistent, no only matches the column name

[0069] strict_struct = yes

[0070] II. Consistency checking method and rule design

[0071] 1. Check the count

[0072] According to the table obtained in the previous step, the record number of the source and destination tables is obtained respectively, the source and destination record data of the table with consistent record number and inconsistent record number are counted, and the result is output to complete the check.

[0073] This method is fast and only used for rough comparison. When the record number is consistent, the data may not be consistent.

[0074] 2, Check the way of rows

[0075] This way to check all the data, the main idea is to avoid the source and destination of the complete data transmission to the check server, reduce the network IO and data risk exposure in the network, speed up the check speed.

[0076] Method 1. Unit, get the row data check code

[0077] Take the test table user in mysql as an example, concatenate all the fields in order, set the delimiter to generate the md5 code and primary key of the row data. Binary data can take its md5 code as value for concatenation. Null value can be converted to a specific value by case when to distinguish value and null. Time and date type can be converted to string, float type number needs to be specified after conversion Character precision.

[0078] Select md5(concat_ws(‘|’,id,nickname,email,md5(picture))),id fromuser

[0079] In this way, the md5 code and primary key of the record row are obtained. The above data is obtained for each table in the source A and destination B, and the difference set of the two data sets is calculated. A-B is the data that the source adds, and B-A is the data that the destination adds.

[0080] Further data completion function can be realized, that is, delete data in the destination (B-A), get complete data from the source (A-B), and convert it to an insert statement to execute in the destination. After the above processing, the data on both ends is completely consistent.

[0081] Method 2: Unit, get the data verification code

[0082] For the case of data consistency and small data difference, a more optimized method can be used. First, sample according to a certain proportion (such as one thousandth), and sort the obtained data according to the primary key. In this way, multiple intervals can be formed according to the primary key, and each interval is approximately one thousand record structure. For example, if the table has 10,000 records, sampling one thousandth will take about 10 records, or 9 or more. Sort by primary key, if there are 9 values, 10 intervals will be generated: less than the minimum value, the interval composed of the middle, and greater than or equal to the maximum value. Generate data intervals for the source database using this method. On the basis of method one, sort the values of an interval, generate a check code for each row, concatenate all the check codes of the rows to generate a check code, which can further compress the amount of data transmitted, an average of one thousand rows of data one check code.

[0083] For example, MySQL, with similar syntax to achieve multi-line data concatenation and generation of check code.

[0084]

[0085] For example, MySQL, the syntax of the sampling function (sampling rate one thousandth) is

[0086] SELECT

[0087] Id, nickname, email

[0088] From user

[0089] Where Rand()<0.001

[0090] order by id

[0091] According to the obtained table, each interval obtains the check code, and compares the interval and check code data taken from the source and destination. If the check is consistent, it means that the interval is completely consistent. For inconsistent intervals, the data is obtained line by line according to mode one for processing. The same way can be used to generate data to fill in the sql statement.

[0092] 3, check mode sample

[0093] This mode is to sample part of the data to evaluate the overall consistency.

[0094] The implementation is to sample the source data at a certain proportion (such as one tenth or one hundredth), obtain its primary key and check code, and obtain its check code from the destination data according to the primary key. Compare the primary key and check code values taken from both ends.

[0095] Because it is sampled, the data is not enough to fill in.

[0096] Configuration file for check rule setting

[0097] [RULE]

[0098] #check rule, count: compare record number rows: compare line by line sample: sample comparison

[0099] mode = rows

[0100] #sampling rate, 100 represents one hundredth, 1000 represents one thousandth, 1 is equivalent to rows

[0101] Sample_rows = 100

[0102] # Sampling row by row or block by block, 1: row by row, other positive integers: approximately the number of records per block

[0103] Block_rows = 1000

[0104] # Repair sql file

[0105] Fix_sql = fix.sql

[0106] # Maximum number of error records, if the number of error records exceeds this number, the current check will be aborted. If the difference is too large, it means that the synchronization is problematic, and further checking is meaningless

[0107] Max_fail_rows = 100000

[0108] Three, function extension

[0109] The above processing is mainly for tables with primary keys or indexes. A unique record can be identified according to the index, and then it can be judged whether the data is consistent.

[0110] For tables without indexes, if the number of records is not large, such as less than 10,000, the source and destination data can be read into the check server for comparison. For more data, try to read the data in blocks to read the data into the check server for comparison; The method of dividing blocks is to view the randomness of each field (grouping by field value statistics, in the case of the same total number of records, the more the grouping, the greater the randomness), select the field with the greatest randomness, sample the data value of the field, and generate data intervals in the same way as the index division; Read the source and destination data into the check server for comparison.

[0111] Take the number of field values in the MySQL lookup table as an example

[0112] SELECT nickname, count(nickname)

[0113] FROM user

[0114] GROUP BY nickname

[0115] The number of records taken above is the number of nickname, and the sum of each count(nickname) is the total number of records.

[0116] The above method is not limited to the mysql database, as long as the database supports similar functions.

[0117] The above method is not limited to the same database, and the source and destination databases support the same check function.

[0118] Embodiment two, on the basis of embodiment one, proposes a system for data consistency check for data synchronization, comprising:

[0119] Configuration management module: used for defining and parsing data source connection parameters, check object rules, log configuration and check mode parameters, establishing source and destination database connection, and obtaining library table structure and index information; the configuration management module comprises:

[0120] Data source configuration unit: stores source and destination database connection strings, and supports MySQL and Oracle relational databases;

[0121] Check object configuration unit: defines the tables to be checked and the tables to be ignored, and supports wildcard matching and case sensitivity setting;

[0122] Check rule configuration unit: including check mode (count / rows / sample), sampling rate, block size, whether to check non-indexed table, and whether to strictly match table structure;

[0123] Log configuration unit: defines log level (debug / info / warn / error) and output file path;

[0124] Repair configuration unit: specifies the storage path of data filling SQL file and the maximum allowed difference record number threshold.

[0125] Check rule engine: according to the configured check mode (count / rows / sample), the following submodules are selected for execution: record number check submodule: statistics the record number of source and destination tables, and outputs consistency result; row-level check submodule: generates data check code by row or block, and compares check code to locate difference data; sampling check submodule: samples source data according to proportion, and checks the consistency of sampled data and corresponding data in destination; the row-level check submodule comprises:

[0126] Row-by-row check unit: concatenates table fields in order to form a string, generates MD5 check code and primary key for each row of data; compares the check code sets of source and destination to locate difference data rows; generates INSERT or DELETE statement for difference data and transmits it to the data filling module for execution;

[0127] Block check unit: samples and sorts data according to primary key, and divides data interval; generates aggregated check code for data in each interval, and compares source and destination interval check codes; for the interval with inconsistent check codes, call the row-by-row check unit to locate the difference;

[0128] Check code generation unit: support binary data MD5 conversion, NULL value replacement, date and time stringification, and floating point precision control.

[0129] Data padding module: generate difference repair SQL statements according to the check results, and perform data insertion or deletion operations at the destination end;

[0130] Timing scheduling module: support periodic triggering of check tasks to realize incremental data synchronization;

[0131] Log and monitoring module: record check process logs, count difference data, and terminate the check when the threshold is exceeded.

[0132] Data check for tables without indexes includes:

[0133] Full comparison unit: when the number of table records is less than the preset threshold, read the source and destination data to the check server for comparison;

[0134] Block comparison unit: when the number of table records is greater than the preset threshold, the following steps are performed: select a block field randomly by field randomness, sample to generate data intervals; read the source and destination data to the check server according to the intervals, and compare the consistency of the data in the intervals; field randomness is determined by the number of grouped values, and the more the groups, the higher the randomness; Randomness analysis unit: support statistical field value distribution through SQL statements to assist in determining the block field.

[0135] The system is suitable for data consistency check between homogeneous or heterogeneous databases, and only needs to meet the following conditions: the source and destination databases support functions to generate data check codes; the source and destination databases support data sampling and sorting functions; the source and destination databases support field value grouping statistics functions; the check process is implemented through standard SQL statements and does not depend on specific database types or versions.

[0136] Although embodiments of the present application have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for data consistency verification suitable for data synchronization, characterized in that: Includes the following steps: Configure the configuration file and validation rules, and define the data source, validation objects, ignore objects, log level and validation mode parameters; Establish database connections between the source and destination ends according to the configuration file, obtain database table structure and index information, and verify the consistency of database table structure and attributes at both ends; Based on the configured verification mode, select at least one of the following verification methods: count mode: count the number of records in the source and destination tables, and output the tables with the same number of records and those with different numbers of records; rows mode: calculate the data check code row by row or block by block, locate the data difference by comparing the check codes, and generate a data completion SQL statement; sample mode: sample the source data proportionally and verify the consistency of the check codes of the sampled data and the corresponding data in the destination. Based on the verification results, data completion or termination of verification is performed, and timed verification is supported to achieve incremental data synchronization.

2. The method for data consistency verification suitable for data synchronization according to claim 1, characterized in that: The rows pattern includes the following steps: Row-by-row verification: Concatenate table fields in order to generate the MD5 checksum and primary key of each row of data; compare the checksum sets of the source and destination to locate the data rows with differences; generate delete or insert statements based on the differences, and perform data completion on the destination. Block verification: Sample and sort the data by primary key to divide the data into intervals; generate an aggregated check code for the data in each interval and compare the check codes of the source and destination intervals; for intervals with inconsistent check codes, fall back to the row-by-row verification method to locate the differences.

3. The method for data consistency verification suitable for data synchronization according to claim 2, characterized in that: The configuration file includes the following parameters: Data source configuration: defines the database connection strings for the source and destination; Validation object configuration: specifies the tables to be validated and the tables to be ignored, supporting wildcard matching; Validation rule configuration: includes validation mode (count / rows / sample), sampling rate, block size, whether to validate tables without indexes, and whether to strictly match the table structure; Log configuration: defines the log level and output file name; Repair configuration: specifies the SQL file path for data completion and the maximum number of allowed difference records.

4. The method for data consistency verification suitable for data synchronization according to claim 3, characterized in that: Data verification for tables without indexes includes the following steps: If the number of records in the table is less than the threshold, all data from the source and destination are read to the verification server for comparison; if the number of records in the table is greater than the threshold, fields are randomly selected for sampling to generate data intervals; data from the source and destination are read to the verification server according to the intervals, and the consistency of data within the intervals is compared; field randomness is determined by the number of groupings of statistical field values, and the more groupings, the higher the randomness.

5. A method for data consistency verification suitable for data synchronization according to claim 4, characterized in that: This function is suitable for data consistency verification between homogeneous or heterogeneous databases. It only requires the source and destination databases to support the following functions: functions for generating data check codes; data sampling and sorting functions; field value grouping and statistical functions; the verification process is implemented through standard SQL statements and does not depend on any specific database type.

6. A system for using the data consistency verification method for data synchronization according to claim 5, characterized in that: include: Configuration Management Module: Used to define and parse data source connection parameters, validation object rules, log configuration and validation mode parameters, establish database connections between the source and destination ends, and obtain database table structure and index information; Validation rule engine: Based on the configured validation mode (count / rows / sample), select and execute the following sub-modules: Record count validation sub-module: Counts the number of records in the source and destination tables and outputs a consistency result; Row-level verification submodule: Generates data verification codes row by row or block by block, and compares the verification codes to locate the data with differences; Sampling verification submodule: Samples source data proportionally and verifies the consistency between the sampled data and the corresponding data at the destination. Data completion module: Generates SQL statements to correct discrepancies based on the verification results, and performs data insertion or deletion operations on the destination end; The timed scheduling module supports periodically triggering verification tasks to achieve incremental data synchronization. Log and monitoring module: Records the verification process log, counts the amount of differential data, and terminates the verification when the threshold is exceeded.

7. The system according to claim 6, characterized in that: The row-level validation submodule includes: Row-by-row verification unit: Concatenates table fields into strings in sequence to generate the MD5 checksum and primary key of each row of data; compares the checksum sets of the source and destination ends to locate the rows of data that differ; generates INSERT or DELETE statements for the data that differs and transmits them to the data completion module for execution. Block verification unit: Samples and sorts data by primary key, divides data into intervals; generates aggregated check codes for data in each interval, compares the check codes of the source and destination intervals; for intervals with inconsistent check codes, calls the row-by-row verification unit to locate the differences; Checksum generation unit: Supports binary data MD5 conversion, NULL value replacement, date and time stringification, and floating-point precision control.

8. The system according to claim 7, characterized in that: The configuration management module includes: Data source configuration unit: Stores the database connection strings for the source and destination, supporting MySQL and Oracle relational databases; Validation Object Configuration Unit: Defines the tables to be validated and the tables to be ignored, supporting wildcard matching and case sensitivity settings; The verification rule configuration unit includes the verification mode (count / rows / sample), sampling rate, block size, whether to verify tables without indexes, and whether to strictly match the table structure. Log configuration unit: Defines the log level (debug / info / warn / error) and the output file path; Repair configuration unit: Specifies the storage path of the SQL file for data completion and the maximum allowed number of differential records threshold.

9. A system according to claim 8, characterized in that: Data validation for tables without indexes includes: Full comparison unit: When the number of records in the table is less than the preset threshold, the full data from the source and destination ends is read to the verification server for comparison; Block Comparison Unit: When the number of table records exceeds a preset threshold, the following steps are performed: Select block fields according to field randomness and sample to generate data intervals; Read source and destination data to the verification server according to the intervals and compare the data consistency within the intervals; Field randomness is determined by the number of groupings of statistical field values, and the more groupings, the higher the randomness; Randomness Analysis Unit: Supports statistical distribution of field values ​​through SQL statements to assist in determining block fields.

10. A system according to claim 9, characterized in that: The system is suitable for data consistency verification between homogeneous or heterogeneous databases, provided that the following conditions are met: both the source and destination databases support functions for generating data check codes; both the source and destination databases support data sampling and sorting functions; and both the source and destination databases support field value grouping and statistical functions. The verification process is implemented using standard SQL statements and does not depend on a specific database type or version.

Images