Cross-data-platform database migration verification method and system applied to autonomous controllable transformation

Abstract

The invention relates to the technical field of database migration verification, in particular to a cross-data-platform database migration verification method and system applied to autonomous controllable transformation. The method comprises the following steps: configuring a source database and a target database in a data source management module; newly establishing a comparison task in the task management module; a comparison database is connected through a dynamic data source assembly, different databases are connected through JDBC, and comparison verification is executed after database connection parameters are configured; database comparison is carried out in the task management module, wherein the database comparison comprises data volume verification and data consistency verification; a comparison report is generated and stored in a built-in mode. The problem of data integrity in the cross-platform data migration process is solved, the system fully considers the difference between the MySQL database and the local database in character set, function and technical characteristics, data verification and comparison work after migration of the MySQL database source and various local target databases is supported, consistency verification before and after data migration is achieved, and the verification efficiency is improved. And the risk of data loss caused by data migration is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of database migration verification, and in particular to a database migration verification method and system for cross-data platforms applied to independent and controllable transformation. Background Art

[0002] Currently, the conventional ways of mainstream cross-platform heterogeneous data migration cover the following several: one is to use the export tool attached to the source database and the import tool of the target database, and necessary intermediate processing is required during this period; the second is manual migration, which usually involves relatively cumbersome operations; the third is to use existing data migration software on the market; the fourth is to independently develop migration software; the fifth is the exclusive migration solution provided by a specific database. These methods have their own advantages and disadvantages.

[0003] In view of the significant differences between large databases and domestic database systems (such as KingbaseES, DM, Highgo, etc.) in terms of resource optimization configuration, table structure design principles, and overall database structure, the following challenges are often encountered during the database migration process: First, the mismatch of character sets between the old and new databases may lead to garbled data after migration, especially the correct display of Chinese information is at risk; second, the unique functions and technologies in the old version of the database may not be recognized in the new database, and may even conflict with some core features of the new database, affecting the data integrity and function implementation after migration.

[0004] Therefore, in order to ensure the integrity of the migrated data, the verification work after the data migration is particularly important. However, the existing database migration verification systems on the market have the following disadvantages: (1) Most of the migration verification systems are developed by database manufacturers to implement data verification and comparison (many-to-one) after migrating multiple non-domestic databases to the domestic database. In the actual process of independent and controllable transformation of the system, the POC test link is usually involved, and multiple database products need to be selected for adaptation testing (one-to-many). After sufficient test verification, the optimal solution is determined for subsequent transformation work; (2) Second, the interaction interface is not friendly enough, with few functions and poor usability.

[0005] Therefore, there is an urgent need to propose a database migration verification system for cross-data platforms applied to independent and controllable transformation, which can support the data verification after migrating the Mysql database source and multiple target databases, realize the consistency verification before and after data migration, detect data loss, and can generate a data verification report. Summary of the Invention

[0006] In order to solve the above-mentioned problems, the present invention provides a database migration verification method and system for cross-data platforms applied to independent and controllable transformation.

[0007] In a first aspect, a database migration verification method for a cross-data platform applied to autonomous and controllable transformation provided by the present invention adopts the following technical solutions: A database migration verification method for a cross-data platform applied to autonomous and controllable transformation includes: Configure the source database and the target database; Use configuration checks and recommendations for data verification; Connect the source database and the target database through a dynamic data source component, use JDBC to connect to different databases, and perform comparison verification after configuring the database connection parameters. Among them, select the verification mode according to the verification scenario, and perform data volume verification and data consistency verification in sequence based on the verification mode; Perform secondary verification of the differential data after performing the comparison verification; Repair the abnormal data after verification; Generate a verification report and store it internally.

[0008] Further, the configuration of the source database and the target database includes configuring the source database and the target database in the data source management module, including IP address, port number, database type, database version, driver, and URL information, and setting the comparison target and scope, including database version, table structure, data type, and data volume.

[0009] Further, the use of configuration checks and recommendations for data verification includes respectively performing service JVM memory configuration verification, core thread number verification, table-level progress verification, memory parameter verification, large object verification, time type millisecond value verification, task creation verification according to the mode, and large table sharding verification.

[0010] Further, the selection of the verification mode according to the verification scenario includes respectively selecting the concise verification mode, the detailed verification mode, the sampling verification mode, and the sharding verification mode according to the verification scenario. Among them, the concise verification mode includes data consistency verification of the number of data entries at the source end and the target end; the detailed verification mode includes data volume verification and data consistency verification of the full volume of data in the table; the sampling verification mode includes data consistency verification of the data that meets the conditions in the table.

[0011] Further, the data volume verification includes using data volume verification to verify whether the data volume of the tables or specific columns in the database before and after migration is consistent, by executing the command SELECT COUNT(*) FROM table_name, and the summation operation for specific columns, and comparing the number of data entries or the total amount of specific data before and after migration.

[0012] Further, the data consistency check includes performing a hash operation HASH(data) on the data to generate a hash value of a fixed length. Among them, the hash value of the data before migration is H(d_b), and the hash value of the data after migration is H(d_a). By comparing the hash values of the data before and after migration, the consistency of the data is verified. If: H(d_b) = H(d_a) then the data is consistent; if: H(d_b) ≠ H(d_a) then there are differences in the data.

[0013] Further, after performing the comparison check, secondary verification of the differential data is carried out, including detailed data verification during the real-time synchronization of incremental data. Based on the verification result differences caused by synchronization delays, after the data is stored at the target end, secondary verification is performed on the differential tables or differential data, and the data of the current source end and target end is queried according to the primary key for verification to achieve the consistency confirmation of the differential data.

[0014] Further, after the verification, repair of abnormal data is carried out, including analyzing whether there are abnormal situations such as system service exceptions, memory overflows, and data synchronization exceptions when data inconsistencies occur during the data verification process, and selecting a processing solution according to the cause of the exception. When selecting the repair method, it includes automatic data repair, automatic data repair based on the accumulated verification results, and manual data repair.

[0015] Further, generating and storing the verification report built-in includes calculating the difference quantity according to the data volume verification and data consistency verification. The difference quantity is used to calculate the data volume difference of the tables or specific columns in the database before and after migration, and the granularity of the difference quantity is accurate to the table level: Δ = |V d_b - V d_a | where Δ is the difference quantity, V d_b is the data volume after migration, and V d_a is the data volume before migration.

[0016] In the second aspect, a database migration verification system for a cross-data platform applied to autonomous and controllable transformation includes: A data source management module for configuring the information of the source database and the target database; A task management module for scheduling and executing data verification tasks; A timed task module for scheduling and executing data verification tasks; A task log module for recording the execution process of data verification tasks and abnormal situations of the results; A user management module for managing user information; Menu management module, used to configure the system menu.

[0017] In summary, the present invention has the following beneficial technical effects: 1. It solves the data integrity problem during cross-platform data migration. The system fully considers the differences in character sets, functions, and technical characteristics between the MySQL database and local databases, supports data verification and comparison work after migrating from a MySQL database source to multiple local target databases, realizes consistency verification before and after data migration, and avoids the risk of data loss caused by data migration.

[0018] 2. By using a dynamic data source component to connect the source database and the target database, using JDBC to connect different databases, and executing the comparison and verification technical solution after configuring the database connection parameters, that is, through methods such as dynamic data source components and JDBC connections, it can connect from a single MySQL source database to multiple different local target databases, and perform comparison and verification work on data migration after connection, thus realizing the adaptation test from a single source database to multiple target databases. Different from the migration verification systems mainly developed for database manufacturers on the market, this system supports data verification after migrating from a MySQL database source to multiple local database targets, ensures the compatibility of data between different database systems, and realizes true cross-data platform verification.

[0019] 3. It meets the requirements for the autonomous and controllable transformation of software systems: This system can support the use in the autonomous and controllable POC test link, transformation link, and dual-track operation link. In the early stage, it can perform adaptation tests on various database products to help users determine the optimal database transformation plan; in the transformation link, it can verify the consistency before and after data migration, thereby reducing the transformation risk and improving the transformation efficiency; in the dual-track operation link, it can set up scheduled tasks to regularly compare the data inconsistencies between the new and old databases.

[0020] 4. It provides a friendly interaction interface and improves usability: The system interaction interface is simple, enhancing the user experience and reducing the operation difficulty.

[0021] 5. It automatically generates a data verification report for subsequent analysis and decision-making: After completing the data migration verification, this system can automatically generate a detailed data verification report, including key information such as data consistency verification results and data loss detection. It provides clear data migration verification results for users to ensure the smooth completion of the database migration work. Description of the Drawings

[0022] Figure 1 It is an architecture diagram of a cross-data platform database migration verification system applied to autonomous and controllable transformation in Embodiment 1 of the present invention; Figure 2It is another schematic diagram of the architecture of a database migration verification system for cross-data platforms applied to autonomous and controllable transformation in Embodiment 1 of the present invention. Detailed implementation manners

[0023] The present invention will be further described in detail below with reference to the accompanying drawings.

[0024] Embodiment 1 Referring to Figure 1 , a database migration verification method for cross-data platforms applied to autonomous and controllable transformation in this embodiment includes: Step 1: Configure the source database and the target database in the data source management module; Step 2: Create a comparison task in the task management module; Step 3: Connect to the comparison database through the dynamic data source component, use JDBC to connect to different databases, and execute comparison verification after configuring the database connection parameters; Step 4: Perform database comparison in the task management module, including data volume verification and data consistency verification; Step 5: Generate a comparison report and store it internally.

[0025] Specifically, it includes the following steps: Step 1: Configure the source database and the target database in the data source management module; Among them, the purpose of data verification is to verify the data consistency between the source and the target end, and to find and make up for possible anomalies in the synchronization. Therefore, when selecting the business tables to be verified, the following conditions need to be met: Tables that need to perform data synchronization; and tables that are frequently changed and are more important in the business; At the same time, considering the limitations of the data verification function itself, the following constraints are imposed on the tables that need to perform data verification: the data volume is below ten million records; there is a primary key or a unique index identifier (when performing full-volume verification, if there is no primary key reference, the performance will be relatively low).

[0026] The working mechanism of the data verification function is: (1) Sort the table data to be verified from the source end and the target end respectively; (2) Query the sorted data; (3) Compare the queried data row by row and column by column; Therefore, the main resources used in the data verification process are: (1) The cpu resources, network bandwidth resources, and IO read operation resources of the source database node at the source end; (2) The cpu resources, network bandwidth resources, and IO read operation resources of the target database node at the target end; (3)Verify the CPU resources of the deployment node.

[0027] Configure the source database and the target database in the data source management module, including IP address, port number, database type, database version, driver, and URL information, and set the comparison targets and scopes, including database version, table structure, data type, and data volume.

[0028] Among them, the prerequisite for using the system data verification and repair function is to install and deploy the system management platform and the system data synchronization program first. The structure of the data verification and repair module is as Figure 1 shown: The system management platform is responsible for connecting to the synchronization program node to collect synchronization service information (such as database connection information, filter configuration information, etc.), creating verification tasks, scheduling tasks, and repair tasks.

[0029] When performing data verification, query data by connecting to the source and target databases through JDBC, verify the data at both ends, and record the verification results in the compare metadata library. When performing data repair, obtain the difference results from the compare metadata library and perform data repair (INSERT / UPDATE / DELETE operations) at the target end.

[0030] Step 2: Use configuration check and recommendation for data verification Before creating a data verification task, it is necessary to select the most suitable data verification service parameter configuration according to the actual business situation of the customer.

[0031] (1)Verify the JVM memory configuration of the service (key, global parameter) Modification method: compare / conf / wrapper.conf Modify the parameter wrapper.java.maxmemory, with a default of 4G and the unit being MB. Modify it according to the actual situation, for example, modify it to 40960.

[0032] The modification takes effect after restarting the verification service (compare / bin / fscompare restart).

[0033] (2)Verify the number of core threads (key, global parameter) The parameter means the maximum number of tables that support simultaneous verification (when a large table is divided into N pieces for verification, it is considered N tables in the same verification task).

[0034] Modification method: Data verification - Verification configuration - Number of core threads for verification. The default number of threads is 5, and it takes effect after saving the modification.

[0035] The setting of parameters needs to refer to the number of CPU cores and the maximum number of connections to the database.

[0036] It is recommended to set it according to 70% of the number of CPU cores. If there are other services / databases on the console machine, it needs to be set according to the remaining CPU. Verify that the number of core threads < the maximum number of connections to the database.

[0037] (3)Table-level verification progress (turn off as needed, global parameter) The control panel interface displays the verification progress of each table. Enabling it will reduce the verification performance. Determine whether to turn it off according to needs.

[0038] Modification method: compare / conf / appliation.properties Modify the parameter tableProgressBar=on, which is enabled by default. Set it to off to turn it off.

[0039] Restart the verification service (compare / bin / fscompare restart) for the modification to take effect.

[0040] (4)Memory verification parameters (turn on as needed in the case of encoding differences, global parameter) Inconsistent sorting in heterogeneous databases can lead to inconsistent verification results, which is generally turned on when encoding in GBK in Oracle.

[0041] Modification method: compare / conf / appliation.properties Modify the parameter useCacheCompre=off, which is disabled by default. Set it to on to turn it on.

[0042] Restart the verification service (compare / bin / fscompare restart) for the modification to take effect.

[0043] (5)Large object verification (turn on as needed, global parameter) If you need to verify large object types (TEXT / BLOB / CLOB / XML / BYTEA / NCLOB), you need to turn on large object verification and recreate the verification task. Enabling large object verification and having large object types in the table will reduce the data verification / data repair performance.

[0044] Modification method: Data verification - Verification configuration - Whether to verify large objects, which is disabled by default. Click Save after modification to take effect.

[0045] Large object verification threshold: When the length of large object data judged by the length method exceeds this value, the content of this column is not verified for consistency.

[0046] Synchronously skip data exceeding the threshold: If the data in the large object column exceeds the threshold, when verifying and repairing data, the data in this column is not synchronized (this column is not included in the insert / update statement).

[0047] (6) Verify the millisecond value of the time type (enabled as needed, global parameter) Modification method: Data verification -> Verification configuration -> Verify the millisecond value of the time type, which is disabled by default and does not verify the millisecond value. Save the changes to take effect.

[0048] (7) Create verification tasks according to the mode Create verification tasks in the way of one mode for one task. Do not perform a full database verification. The number of parallel tasks can be set under multiple tasks.

[0049] Modification method: Data verification -> Verification configuration -> Number of task parallelisms, default value is 3. Save the changes to take effect.

[0050] If there are many tables and large table data under the mode, split the tables and create multiple verification tasks for verification.

[0051] (8) Sharding of large tables (set at the verification task level, set each time a task is created) Improve the verification performance by splitting large tables into multiple small tables.

[0052] There are two methods: 1. All tables are sharded in the same way When creating a verification task, click on Advanced Parameters, where there is a setting for the amount of data per shard. It is recommended to fill in 200,000.

[0053] 2. Set sharding for a single table Setting method: Data verification - New verification task - Select verification object - Sharding, click the sharding button, select the sharding column, and set the number of shards. It is recommended to select the primary key or an indexed column as the sharding column, and integer / time type sharding is supported.

[0054] Example: Number of shards = Total number of table data / 100,000, for example, 30 million records are divided into 300 shards.

[0055] Step 3: Connect to the comparison database through the dynamic data source component, use JDBC to connect to different databases, and execute comparison verification after configuring the database connection parameters; Among them, it includes the following aspects: (1) When it comes to the scenario of roughly judging whether there are differences in data for large data volume tables, adopt the streamlined verification mode, and only need to verify whether the number of data entries in the source and target ends is consistent; Among them, connect to the source and target databases through JDBC, and execute "select count(*)" to verify whether the number of data entries is consistent.

[0056] Only verify whether the number of data entries is consistent, and do not verify whether the actual data is consistent. For dynamic scenarios, that is, when business data is continuously inserted into the source database and incremental data is also being synchronized, due to the existence of a short synchronization delay, there may be differences in the verification results in the streamlined mode. (2)Detailed verification mode, verify all the data in the table, and confirm whether the detailed data is consistent.

[0057] In the detailed verification mode, obtain all the data of the source and target ends of the verification table, and verify the consistency of the data at both ends row by row and column by column; Among them, connect to the source and target databases through JDBC, query the data and sort it according to the primary key, verify the data at both ends in sequence through the sorted primary key, and register the results in the compare metadata library. The verification results are divided into consistent, more in the source end, more in the target end, and differences between the source and target ends. For tables without a primary key, the verification operation will be performed with all non-large object columns as the primary key columns.

[0058] (3)Sampling verification mode, only verify the data that meets the conditions in the table, filter by time, and verify whether the detailed data of the filtered results is consistent; Among them, set the filtering conditions when creating the verification task. Obtain partial data of the source and target ends of the verification table according to the filtering conditions, and verify the consistency of the data at both ends row by row and column by column.

[0059] Connect to the source and target databases through JDBC, query the data according to the filtering conditions and sort it according to the primary key, verify the data at both ends in sequence through the sorted primary key, and register the results in the compare metadata library. The verification results are divided into consistent, more in the source end, more in the target end, and differences between the source and target ends. For tables without a primary key, the verification operation will be performed with all non-large object columns as the primary key columns.

[0060] (4)Sharding verification mode Since querying a large amount of data at one time in the detailed verification of a large table may cause verification exceptions. Through sharding verification, the large table is split into multiple pieces, and the verification of the large table is completed by verifying the data of each piece, which is used to improve the efficiency of data verification.

[0061] When creating a verification task, set the shard size or the number of shards, and split the large table into multiple pieces. During the verification execution, the data of each shard will be verified, thus covering all the data in the table. It is recommended to use the primary key column with good data selectivity for sharding, and only integer and time type columns are supported for sharding. Based on the shard size (i.e., the number of shards), obtain the maximum and minimum values at the source end, calculate the shard intervals, and set the filtering conditions according to the shard intervals (i.e., set the where intervals). Each interval corresponds to a shard. When performing shard verification, each shard is regarded as a small table and executed separately, and data is filtered and verified according to the filtering conditions.

[0062] Step 4: Secondary verification of differential data Among them, during the real-time synchronization of incremental data, detailed data verification is performed. Due to synchronization latency, there are differences in the verification results. After the data is imported into the target end, it is possible to quickly verify the differential tables or differential data, improving the user experience of the data verification function.

[0063] Quickly add differential data (tables) for secondary verification based on the verification results, query the current source-end and target-end data according to the primary key for verification, and more quickly and conveniently achieve the consistency confirmation of differential data.

[0064] Automatically trigger the original data verification task based on the differential data according to the verification results. After the verification task is triggered, it will query the current source-end and target-end data according to the primary key for verification.

[0065] Step 5: Verification and repair 1. During the data verification process, if data inconsistencies are found, data repair processing needs to be carried out in a timely manner, and at the same time, the reasons for the data inconsistencies need to be investigated, and measures need to be formulated to ensure that data inconsistencies will not occur again. This includes (1) System service exception: When it is found that Kingbase FlySync is not running properly, it is necessary to deeply analyze the reasons for the process exception and perform recovery work; (2) Memory overflow: Adjust the memory size, set automatic recovery or analyze the reasons why automatic recovery is not effective; If it has not been running properly for a short period of time, only the system needs to be started to make it continue to work, and the system will automatically perform breakpoint resumption and continue to synchronize data; If it has not been running properly for a long time, it is possible that the archive logs of the source-end database have been cleared, resulting in the system being unable to perform breakpoint resumption. At this time, it is necessary to re-execute the migration of the stock data or use a data verification tool to verify and repair the data and then reset the system to make it continue to work.

[0066] (3) Data synchronization exception When the data at both ends is inconsistent, it is necessary to analyze the reasons for the data inconsistency and formulate strategies to avoid the recurrence of inconsistencies; if the data inconsistency is caused by other manual write operations at the target end, resulting in data conflicts, it is necessary to contact the customer's technical personnel in a timely manner for confirmation, and adjust the synchronization strategy if necessary; if the data is lost due to the synchronization tool itself, it is necessary to further check the system log, locate the link where the data is lost, and formulate a targeted treatment plan; when there is data latency, it is necessary to analyze the reasons for the latency and formulate improvement measures; determine the link where the data latency occurs and make adjustments according to the methods given in the functional and performance tests.

[0067] 2. Data repair methods After discovering data inconsistency, the operation method for performing data repair is as follows: (1) It can be configured to automatically perform data repair after a certain delay after the data verification task ends; (2) It can be configured to automatically perform data repair again after the total number of different data entries exceeds a certain threshold according to the cumulative results of data verification (3) Manually perform data repair as needed For the first method, data repair can be completed immediately after discovering data differences, but the frequency of data repair will be relatively higher; for the second method, it can be selected when data differences in the business process can be tolerated, which can reduce the frequency of data repair. You can choose among the above three data repair methods according to the actual business situation.

[0068] Specifically, the data repair function supports manual and automatic repair of data at the target end. The manual method supports data repair at the data level, single-table level, multi-table level, and operation type; the automatic method supports repairing data at the target end according to the verification task level, and can be set to trigger automatic data repair according to the amount of different data and time.

[0069] Among them, when performing data repair, obtain the difference result from the compare meta-library, connect to the target end through jdbc, and perform data repair (INSERT / UPDATE / DELETE operations) at the target end.

[0070] As a further implementation method, When performing data repair, by default, data repair stops when an error occurs. When data repair skips conflicts and continues to repair, it will automatically skip errors and continue to repair.

[0071] Modification method: compare / conf / appliation.properties, modify the parameter compare.setting.replicateBatchSize=1 and compare.setting.replicateFailurePolicy=WARN; Among them, N in the parameter compare.setting.replicateBatchSize=N represents N pieces of data per batch during data repair. Setting it to 1 will reduce the repair performance; setting it to 50, if an error occurs, 50 pieces of data in this batch will be skipped and can be adjusted as needed.

[0072] Embodiment 2 This embodiment provides a database migration verification system for cross-data platforms applied to autonomous and controllable transformation, as Figure 2 shown, including: A data source management module for configuring source database and target database information; A task management module for scheduling and executing data verification tasks; A scheduled task module for scheduling and executing data verification tasks; A task log module for recording the execution process of data verification tasks and abnormal situations of results; A user management module for managing user information; A menu management module for configuring system menus.

[0073] The above are all preferred embodiments of the present invention. The protection scope of the present invention is not limited by this. Therefore, all equivalent changes made according to the structure, shape, and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A database migration verification method for cross - data platforms applied to autonomous and controllable transformation, characterized in that, Including: Configure the source database and the target database; Use configuration checks and recommendations for data verification; Connect the source database and the target database through the dynamic data source component, use JDBC to connect to different databases, execute comparison verification after configuring the database connection parameters, and select the verification mode according to the verification scenario, and perform data volume verification and data consistency verification in sequence based on the verification mode; Perform secondary verification on the differential data after performing the comparison verification; Repair the abnormal data after verification; Generate a verification report and store it internally.

2. The database migration verification method for a cross-data platform applied to autonomous and controllable transformation according to claim 1, wherein, The configuration of the source database and the target database includes configuring the source database and the target database in the data source management module, including IP address, port number, database type, database version, driver, and URL information, and setting the comparison target and scope, including database version, table structure, data type, and data volume.

3. The database migration verification method for a cross-data platform applied to autonomous and controllable transformation according to claim 2, wherein The data verification using configuration checks and recommendations includes performing service JVM memory configuration verification, core thread number verification, table-level progress verification, memory parameter verification, large object verification, time type millisecond value verification, task creation verification according to the mode, and large table sharding verification respectively.

4. The database migration verification method for a cross-data platform applied to autonomous and controllable transformation according to claim 3, wherein The selection of the verification mode according to the verification scenario includes respectively selecting the concise verification mode, the detailed verification mode, the sampling verification mode, and the sharding verification mode according to the verification scenario. Among them, the concise verification mode includes data consistency verification of the number of data entries at the source end and the target end; the detailed verification mode includes data volume verification and data consistency verification of the full amount of data in the table; the sampling verification mode includes data consistency verification of the data that meets the conditions in the table.

5. A database migration verification method for a cross-data platform applied to autonomous and controllable transformation according to claim 4, characterized in that The data volume verification includes using data volume verification to verify whether the data volume of the table or specific columns in the database before and after migration is consistent, by executing the command SELECT COUNT(*) FROM table_name, and the summation operation for specific columns, and comparing the number of data entries or the total amount of specific data before and after migration.

6. The database migration verification method for a cross-data platform applied to autonomous and controllable transformation according to claim 5, characterized in that, The data consistency verification includes performing a hash operation HASH(data) on the data to generate a hash value of a fixed length. Among them, the data hash value before migration is H(d_b), and the data hash value after migration is H(d_a). By comparing the hash values of the data before and after migration, the consistency of the data is verified. If: H(d_b) = H(d_a) Then the data is consistent; if: H(d_b) ≠ H(d_a) Then there are differences in the data.

7. A database migration verification method for a cross-data platform applied to autonomous and controllable transformation according to claim 6, characterized in that The secondary verification of the differential data after performing the comparison verification includes performing detailed data verification during the real-time synchronization of incremental data. Based on the verification result differences caused by the synchronization delay, after the target end is warehoused, perform secondary verification on the differential table or differential data, and query the current source end and target end data according to the primary key for verification to achieve the consistency confirmation of the differential data.

8. A database migration verification method for a cross-data platform applied to autonomous and controllable transformation according to claim 7, characterized in that After the verification, abnormal data repair is performed, including analyzing whether there are abnormal situations such as system service exceptions, memory overflows, and data synchronization exceptions when data inconsistencies occur during the data verification process, and selecting a processing solution based on the cause of the exception. When selecting a repair method, it includes automatic data repair, automatic data repair based on the accumulated verification results, and manual data repair by users.

9. A database migration verification method for a cross-data platform applied to autonomous and controllable transformation according to claim 8, characterized in that The generation and storage of the verification report are built-in, including calculating the difference amount based on the data volume verification and data consistency verification. The difference amount is used to calculate the difference in the data volume of tables or specific columns in the database before and after migration, and the granularity of the difference amount is accurate to the table level: Δ = |V d_b - V d_a | Among them, Δ is the difference amount, V d_b is the data amount after migration, and V d_a is the data amount before migration.

10. A database migration verification system across data platforms applied to autonomous and controllable transformation, characterized in that, It includes: The data source management module is used to configure the source database and target database information; The task management module is used to schedule and execute data verification tasks; The scheduled task module is used to schedule and execute data verification tasks; The task log module is used to record the execution process of data verification tasks and abnormal situations of the results; The user management module is used to manage user information; The menu management module is used to configure the system menu.

Images