Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A data migration method and system, storage medium, and electronic device

A technology for migrating systems and data, which is applied in database design/maintenance, structured data retrieval, database distribution/replication, etc. It can solve the problems of checking efficiency or feasibility, the inability to strictly guarantee data consistency, long-term efficiency, etc., to achieve Avoid repeated consumption of process time, overcome checking efficiency or feasibility, and ensure the effect of data

Active Publication Date: 2021-05-25
ADVANCED NEW TECH CO LTD
View PDF11 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0010] 1. The strong dependence of database import and export tools makes it not suitable for all types of databases, or cannot support large data migration
When the old database does not have an export tool, or the new database does not have an import tool, the migration process cannot be completed; when the import and export tool has a limit on the size of the data file, the database may need to be migrated in stages, or even the data migration cannot be completed
[0011] 2. The consistency check of new and old data is carried out by means of statistics and sampling. There are problems with the check efficiency or feasibility, and the success of the check cannot strictly guarantee the data consistency
When the amount of table data is huge, it takes a long time to make statistics on the table and is inefficient, and may even be impossible to execute; when the check is successful, there may be cases where negative and negative may be positive, resulting in misjudgment of data consistency, and successful check can only guarantee a high probability of consistency
[0012] 3. When checking the inconsistency, it is impossible to check out the difference data and the cause of the exception. It may be that the export process or the import process is abnormal, so it is necessary to re-export the data file and import, which needs to consume process time repeatedly
[0013] 4. DBA manual operation is inefficient, and there may be human errors
When the amount of table data to be migrated is huge, the operation of each step is very time-consuming, and serial execution between steps consumes a lot of time for the DBA, which is inefficient and prone to human errors

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A data migration method and system, storage medium, and electronic device
  • A data migration method and system, storage medium, and electronic device
  • A data migration method and system, storage medium, and electronic device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0059] Example 1, such as Figure 4 As shown, a data migration method includes the following steps:

[0060] S11. Synchronize the old database data to the data warehouse; use the data warehouse synchronization tool to synchronize the old database table data to the data warehouse, assuming that this version is the data warehouse data version A.

[0061] S21, new database table data cleaning; clean up the new database table data, create a new database by deleting and creating a new table, or directly emptying the table data; for example, after MySQL uses drop, truncate, delete or PHP script to delete the data table Then use create or PHP script to create a data table.

[0062] S3. Backflow data or data patches to the new database through the data warehouse synchronization tool; synchronously return the data warehouse data version A (data patch if there is a data patch) to the new database through the data warehouse synchronization tool. Among them, the core component ETL in th...

Embodiment 2

[0073] Example 2, such as Figure 5 As shown, a data migration method includes the following steps:

[0074] S12, new database table data cleanup; clean up new database table data, create a new database by deleting and creating a new table, or directly empty the table data; for example, in MySQL, use drop, truncate, delete or use PHP script to delete the data table Then use create or PHP script to create a data table.

[0075] S22. Synchronize the old database data to the data warehouse; use the data warehouse synchronization tool to synchronize the old database table data to the data warehouse to generate the first data version of the data warehouse. It should be understood that the cleanup of new database table data and the synchronization of old database data to the data warehouse are relatively isolated, and there is no time or program sequence restriction relationship.

[0076]S23, the first reflow judgment, judging whether there is data or not in the second data versio...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention provides a data migration method, which includes the steps of data migration preparation, data reflow, data patch generation, and data migration completion judgment. The invention also relates to a data migration system, storage medium and electronic equipment. The present invention overcomes the shortcomings of strong dependence on import and export tools, adopts the data synchronous reflow tool of the data warehouse, overcomes the shortcomings of checking efficiency or feasibility, and cannot strictly guarantee data consistency after successful checking: the present invention adopts row-level field-by-field comparison, and the checking is successful The data can be guaranteed to be completely consistent. Overcome the inability to check out the difference data and abnormal causes, and avoid the shortcomings of repeatedly consuming process time. The present invention overcomes the shortcomings of DBA manual operation, low efficiency, and possible human error. The migration process can be serially executed through the task tree of the data warehouse. The present invention realizes efficient and fully automatic data migration through the system.

Description

technical field [0001] The present invention relates to the field of data migration, in particular to a data migration method and system, a storage medium, and electronic equipment. Background technique [0002] Database middleware is a middleware system that organizes, stores, and manages data according to the data structure; middleware users can manage database permissions, data structures, and add, delete, and modify data through the database language. With the rapid growth of the business volume supported by the application system, the capacity and performance of database middleware encounter bottlenecks, which cannot support business data storage and business service read and write operations; the rapid development of database technology, new technologies provide better features, such as distribution Compared with the traditional centralized database, the centralized database has the ability of unlimited horizontal expansion; in many scenarios of business requirements a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/27G06F16/21
CPCG06F16/219G06F16/27
Inventor 许哲
Owner ADVANCED NEW TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com