Method, system and equipment for batch migration of big data tables between databases and medium

A database and data table technology, applied in the computer field, can solve problems such as data importing into the database and data migration failure, etc., and achieve the effect of flexibly migrating file size control methods, reducing workload, and realizing batchization.

Pending Publication Date: 2022-03-08
中国农业银行股份有限公司福建省分行
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, different database vendors on the market basically only provide data import or export operations for their own database products, and have never considered how to import their own data into other vendors' databases.
Moreover, in the process of data import and export, if the number of records in the data table is particularly large, reaching billions or tens of billions, data migration will often fail due to various problems

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method, system and equipment for batch migration of big data tables between databases and medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0037] This embodiment provides a method for batch migration of large data tables between databases, including:

[0038] Step 1. Generate a corresponding configuration file according to the data table to be migrated, the configuration file includes at least one row, and each row corresponds to 10 configuration columns, and each configuration row is: source database, source database table Name, source encoding method, target database, target database data table name, target database encoding method, migration method, migration basis field, start value, and end value;

[0039] Step 2. Read the configuration file cyclically and perform batch migration of data tables; after each migration process is completed, according to the configuration file's source database table name, target database data table name, migration method, migration basis field, start value and end value , Generate the query SQL script for the number of records in the data table, respectively obtain the number o...

Embodiment 2

[0044] In this embodiment, a system for batch migration of large data tables between databases is provided, including:

[0045] The setting module generates a corresponding configuration file according to the data tables to be migrated, the configuration file includes at least one row, and each row corresponds to 10 configuration columns, and each configuration row is: source database, source database table Name, source encoding method, target database, target database data table name, target database encoding method, migration method, migration basis field, start value, and end value;

[0046] The migration module reads the configuration file cyclically and performs batch migration of data tables; after each migration process is completed, according to the configuration file's source database table name, target database data table name, migration method, migration basis field, start value and end value , Generate the query SQL script for the number of records in the data tabl...

Embodiment 3

[0052] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and operable on the processor. When the processor executes the computer program, any implementation manner in Embodiment 1 can be implemented.

[0053] Since the electronic device introduced in this embodiment is the device used to implement the method in Embodiment 1 of this application, based on the method described in Embodiment 1 of this application, those skilled in the art can understand the electronic device of this embodiment. Specific implementation methods and various variations thereof, so how the electronic device implements the method in the embodiment of the present application will not be described in detail here. As long as a person skilled in the art implements the equipment used by the method in the embodiment of the present application, it all belongs to the protection scope of the present application.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a method, a system, equipment and a medium for batch migration of big data tables among databases, and the method comprises the following steps: generating a corresponding configuration file according to a data table needing to be migrated, the configuration file comprises at least one row, each row corresponds to 10 configuration columns, and each row corresponds to 10 configuration columns; each configuration column comprises a source database, a source database table name, a source coding mode, a target database, a target database data table name, a target database coding mode, a migration mode, a migration basis field, a start numerical value and an end numerical value; and the configuration file is circularly read, and batch migration of the data table is performed, so that the data migration difficulty and migration workload of developers are reduced.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a method, system, device and medium for batch migration of large data tables between databases. Background technique [0002] In recent years, with the explosive growth of data, the continuous enrichment of database products on the market, the sudden emergence of domestic database products, the continuous intensification of industry competition, the speed of updating database products has also increased. Many internal management systems have been developed within the bank over the years. With the upgrading of database products, the selection of database products between internal systems has gradually diversified, and data migration between different database products has emerged. However, different database vendors on the market basically only provide data import or export operations for their own database products, and have never considered how their own data can be imported i...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/21G06F16/22
CPCG06F16/214G06F16/2282
Inventor 曾钊饶龙翔钱美旋
Owner 中国农业银行股份有限公司福建省分行
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap