A Data Validation Method Oriented to Database Data Migration

Abstract

The present invention relates to a data verification method oriented to database data migration. Compared with the prior art, the database data migration method and system of this application regards each data table as a migration task, and uses database import and export tools or custom programs as different tasks. The main means to read and write data in the database, run these tasks in a distributed cluster, and add a custom database sub-algorithm to the data processing, divide the read data into multiple files, and then call the database import in the task Export tools or custom programs import data into multiple new databases. All data migrations are carried out at the same time without mutual influence, which greatly speeds up the speed of data migration. At the same time, the data verification function is provided to ensure the stability and data correctness during the migration process. During the data migration process, distributed verification is performed on the data to ensure the security and completeness of the data verification. At the same time, users are allowed to customize the data verification method, and finally the verification result data is returned to the corresponding user.

Description

【Technical field】 [0001] The invention is dedicated to data security access, and ensures that the data is free from missing and complete during the migration process by performing distributed verification on the migration data during the data migration process of the database. 【Background technique】 [0002] In early Internet applications, the underlying data generally adopts a storage solution for storing all data in a single database. With the development of Internet applications, the number of users has increased sharply, and the amount of data storage has increased exponentially. The underlying data single-database storage scheme restricts or even limits the further expansion of Internet applications. Therefore, in order to solve the bottleneck problem of single database storage, it is necessary to gradually adopt another solution that supports horizontal storage. To solve the bottleneck of data storage, the existing storage structure must be switched. However, due to t...

AI Technical Summary

Problems solved by technology

To solve the bottleneck of data storage, the existing storage structure must be switched. However, due to the huge amount of underlying data at present, how to migrate the original massive data to a new storage structure that supports horizontal expansion while switching the storage structure It is also a big bottleneck point, especially this kind of expansion cannot guarantee that the correct and complete data results will be obtained in the end

Moreover, the common expansion method is not only costly and inefficient, but also the security and reliability cannot be guaranteed

Images