Data migration method and system, storage medium, and electronic device

A technology for migrating systems and data, applied in the direction of electrical digital data processing, special data processing applications, instruments, etc., can solve problems such as inapplicability to all types of databases, problems with checking efficiency or feasibility, and inability to strictly guarantee data consistency, etc., to achieve Overcoming the inability to check out the difference data and abnormal causes, avoiding repeated consumption of process time, and overcoming the effect of checking efficiency or feasibility

Active Publication Date: 2018-12-21
ADVANCED NEW TECH CO LTD
View PDF11 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0010] 1. The strong dependence of database import and export tools makes it not suitable for all types of databases, or cannot support large data migration
When the old database does not have an export tool, or the new database does not have an import tool, the migration process cannot be completed; when the import and export tool has a limit on the size of the data file, the database may need to be migrated in stages, or even the data migration cannot be completed
[0011] 2. The consistency check of new and old data is carried out by means of statistics and sampling. There are problems with the check efficiency or feasibility, and the success of the check cannot strictly guarantee the data consistency
When the amount of table data is huge, it takes a long time to make statistics on the table and is inefficient, and may even be impossible to execute; when the check is successful, there may be cases where negative and negative may be positive, resulting in misjudgment of data consistency, and successful check can only guarantee a high probability of consistency
[0012] 3. When checking the inconsistency, it is impossible to check out the difference data and the cause of the exception. It may be that the export process or the import process is abnormal, so it is necessary to re-export the data file and import, which needs to consume process time repeatedly
[0013] 4. DBA manual operation is inefficient, and there may be human errors
When the amount of table data to be migrated is huge, the operation of each step is very time-consuming, and serial execution between steps consumes a lot of time for the DBA, which is inefficient and prone to human errors

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data migration method and system, storage medium, and electronic device
  • Data migration method and system, storage medium, and electronic device
  • Data migration method and system, storage medium, and electronic device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0059] Example 1, such as Figure 4 As shown, a data migration method includes the following steps:

[0060] S11. Synchronize the old database data to the data warehouse; use the data warehouse synchronization tool to synchronize the old database table data to the data warehouse, assuming that this version is the data warehouse data version A.

[0061] S21, new database table data cleaning; clean up the new database table data, create a new database by deleting and creating a new table, or directly emptying the table data; for example, after using drop, truncate, delete or PHP script to delete the data table in MySQL Then use create or PHP script to create a data table.

[0062] S3. Backflow data or data patches to the new database through the data warehouse synchronization tool; synchronously return the data warehouse data version A (data patch if there is a data patch) to the new database through the data warehouse synchronization tool. Among them, the core component ETL i...

Embodiment 2

[0073] Example 2, such as Figure 5 As shown, a data migration method includes the following steps:

[0074] S12. New database table data cleanup; clean up new database table data, create a new database by deleting and creating a new table, or directly emptying table data; for example, after MySQL uses drop, truncate, delete or PHP script to delete the data table Then use create or PHP script to create a data table.

[0075] S22. Synchronize the old database data to the data warehouse; use the data warehouse synchronization tool to synchronize the old database table data to the data warehouse to generate the first data version of the data warehouse. It should be understood that the cleanup of new database table data and the synchronization of old database data to the data warehouse are relatively isolated, and there is no time or program sequence restriction relationship.

[0076]S23, the first reflow judgment, judging whether there is data or not in the second data version,...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data migration method, comprising the steps of data migration preparation, data reflux, data patch generation and data migration completion judgment. The invention also relates to a data migration system, a storage medium and an electronic device. The invention overcomes the shortcomings of strong dependence of the import and export tools, adopts the data synchronous return tool of the data warehouse, overcomes the shortcomings of checking efficiency or feasibility, and the data consistency can not be strictly ensured by successful checking: the invention adopts line-level field-by-field comparison, and the data consistency can be ensured by successful checking. Overcomes the shortcomings of not being able to check out the discrepancy data and abnormal reasons, andavoids the repeated consumption of process time. The present invention overcomes the shortcomings of DBA manual operation, low efficiency and possible human error, and the migration flow can be executed serially through the task tree of the data warehouse. The present invention realizes high efficiency and automatic data migration through the system.

Description

technical field [0001] The present invention relates to the field of data migration, in particular to a data migration method and system, a storage medium, and electronic equipment. Background technique [0002] Database middleware is a middleware system that organizes, stores and manages data according to the data structure; middleware users can manage database permissions, data structures, and add, delete, and query operations on the database through the database language. With the rapid growth of the business volume supported by the application system, the capacity and performance of database middleware encounter bottlenecks, which cannot support business data storage and business service read and write operations; the rapid development of database technology, new technologies provide better features, such as distribution Compared with the traditional centralized database, the centralized database has the ability of unlimited horizontal expansion; in many scenarios of bus...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/219G06F16/27
Inventor 许哲
Owner ADVANCED NEW TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap