Data migration method and system, storage medium, and electronic device

Abstract

The invention provides a data migration method, comprising the steps of data migration preparation, data reflux, data patch generation and data migration completion judgment. The invention also relates to a data migration system, a storage medium and an electronic device. The invention overcomes the shortcomings of strong dependence of the import and export tools, adopts the data synchronous return tool of the data warehouse, overcomes the shortcomings of checking efficiency or feasibility, and the data consistency can not be strictly ensured by successful checking: the invention adopts line-level field-by-field comparison, and the data consistency can be ensured by successful checking. Overcomes the shortcomings of not being able to check out the discrepancy data and abnormal reasons, andavoids the repeated consumption of process time. The present invention overcomes the shortcomings of DBA manual operation, low efficiency and possible human error, and the migration flow can be executed serially through the task tree of the data warehouse. The present invention realizes high efficiency and automatic data migration through the system.

Description

technical field [0001] The present invention relates to the field of data migration, in particular to a data migration method and system, a storage medium, and electronic equipment. Background technique [0002] Database middleware is a middleware system that organizes, stores and manages data according to the data structure; middleware users can manage database permissions, data structures, and add, delete, and query operations on the database through the database language. With the rapid growth of the business volume supported by the application system, the capacity and performance of database middleware encounter bottlenecks, which cannot support business data storage and business service read and write operations; the rapid development of database technology, new technologies provide better features, such as distribution Compared with the traditional centralized database, the centralized database has the ability of unlimited horizontal expansion; in many scenarios of bus...

AI Technical Summary

Problems solved by technology

[0010] 1. The strong dependence of database import and export tools makes it not suitable for all types of databases, or cannot support large data migration

When the old database does not have an export tool, or the new database does not have an import tool, the migration process cannot be completed; when the import and export tool has a limit on the size of the data file, the database may need to be migrated in stages, or even the data migration cannot be completed

[0011] 2. The consistency check of new and old data is carried out by means of statistics and sampling. There are problems with the check efficiency or feasibility, and the success of the check cannot strictly guarantee the data consistency

When the amount of table data is huge, it takes a long time to make statistics on the table and is inefficient, and may even be impossible to execute; when the check is successful, there may be cases where negative and negative may be positive, resulting in misjudgment of data consistency, and successful check can only guarantee a high probability of consistency

[0012] 3. When checking the inconsistency, it is impossible to check out the difference data and the cause of the exception. It may be that the export process or the import process is abnormal, so it is necessary to re-export the data file and import, which needs to consume process time repeatedly

[0013] 4. DBA manual operation is inefficient, and there may be human errors

When the amount of table data to be migrated is huge, the operation of each step is very time-consuming, and serial execution between steps consumes a lot of time for the DBA, which is inefficient and prone to human errors

Images