Distributed cluster data migration method and device

A distributed cluster and data technology, applied in digital data protection, computer security devices, structured data retrieval, etc., can solve the problems of high task pressure on management nodes, occupation of storage resources, and low file transfer rate.

Pending Publication Date: 2021-09-14
INDUSTRIAL AND COMMERCIAL BANK OF CHINA
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] First, data migration involves the interaction between production and test environments, requiring a change application, resulting in a long process
[0004] Second, the dump method is adopted in the traditional ORACLE database, and the client tool programs EXP and IMP are used
When dumping files, someone needs to pay attention to the dump process, find out and solve errors in time, and consume manpower; the exported files are stored in the scheduling server, which will occupy storage resources and affect the operation of other programs; the file transfer rate is low
[0005] Third, when performing data migration in a traditional distributed database, it is necessary to send import / export tasks from the data management node to the data node and receive the processing results. The task pressure on the management node is huge

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed cluster data migration method and device
  • Distributed cluster data migration method and device
  • Distributed cluster data migration method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0035] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of this application.

[0036] Considering the problems existing in the existing technology: first, data migration involves the interaction between production and test environments, and a change application needs to be submitted, resulting in a long process; second, the dump method is used in the traditional ORACLE database, and the client tool program is used EXP and IMP. Based on ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention provides a distributed cluster data migration method and device, relates to the field of big data processing and can also be used for the financial field, and the method comprises the following steps: performing data deformation on to-be-migrated data in a source database according to a preset data deformation rule, extracting the to-be-migrated data subjected to data deformation to a target table of the source database, wherein the target table is mapped to a corresponding transfer machine through a data transmission port and an IP address defined in a preset appearance; extracting data in the target table to the appearance, and generating a transfer machine data file, so that a scheduling server migrates the transfer machine data file to a corresponding target database according to directory parameters in the transfer machine; the data migration efficiency can be effectively improved, and system and human resources are saved.

Description

technical field [0001] This application relates to the field of distributed technology, and can also be used in the field of big data processing or finance, and specifically relates to a distributed cluster data migration method and device. Background technique [0002] Based on the background of big data distributed cluster data migration, the data migration scenario of large-scale data import / export currently has the following difficulties: [0003] First, data migration involves the interaction between the production and test environments, requiring a change application, resulting in a longer process. [0004] Second, the dump method is adopted in the traditional ORACLE database, and the client tool programs EXP and IMP are used. Based on production control requirements, the data needs to be deformed before exporting. The general process is "data deformation" → "dump export file" → "transfer" → "dump import file". When dumping files, someone needs to pay attention to t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/21G06F16/27G06F21/62
CPCG06F16/214G06F16/27G06F21/6227G06F2221/2107
Inventor 金童张世瑛赵吉昆梁晔华
Owner INDUSTRIAL AND COMMERCIAL BANK OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap