Unlock instant, AI-driven research and patent intelligence for your innovation.

Cross-database data migration method and cross-database data access method

A data access and database technology, applied in the database field, can solve problems such as increasing the access pressure of business systems and online business impact, and achieve the effect of reducing pressure and ensuring smooth operation.

Inactive Publication Date: 2020-05-05
UNICLOUD (NANJING) DIGITAL TECH CO LTD
View PDF3 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This method can do incremental data synchronization very well, but if you use sqoop to import and export the full amount of business data, because the database is huge, directly accessing the online environment database will increase the access pressure of the business system, and will affect the online business impact

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Cross-database data migration method and cross-database data access method
  • Cross-database data migration method and cross-database data access method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0024] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. It should be understood, however, that the invention may be embodied in various forms, and some exemplary and non-limiting embodiments are shown in the drawings and described below, and are not intended to limit the invention to the specific embodiments described. .

[0025] figure 1 An implementation manner of a data access method according to the present invention is shown, including the following steps:

[0026] (1) When storing data in the oracle / mysql database, export the stored data when the system is idle;

[0027] (2) Generate a business data file in the format specified by Hbase according to the exported data;

[0028] (3) Create an Hbase data table, and design the rowKey format according to business requirements;

[0029] (4) Write the business data file into the Hbase database through the importtsv command.

[0030] In one or more...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a cross-database data migration method and a cross-database data access method, and the data migration method comprises the steps: exporting the stored data when a system is idle when an oracle / mysql database stores the data; generating a service data file in a specified format of the Hbase according to the exported data; establishing an Hbase data table, and designing a rowKey format according to service requirements; and writing the service data file into an Hbase database through an importttsv command. According to the data access method, data migration is realized based on the data migration method, then a query expression in a data access request is converted into rowKey query through a phoenix query engine and a thritf query engine, and required data is queriedfrom an Hbase database and returned to a client. According to the invention, data migration can be carried out on the premise of not influencing the stability of the online system, and the access pressure of the online system is reduced.

Description

technical field [0001] The invention relates to the technical field of databases, in particular to a cross-database data migration method and a data access method. Background technique [0002] With the construction and operation of the system, the system data is increasing. Smart cities are connecting more and more third-party business data. The efficiency of traditional relational databases such as mysql and oracle is not satisfactory in the writing process of large amounts of data, the application when the fields are not fixed, and the processing of simple queries that need to return results quickly. It is necessary to solve the database bottleneck in the concurrent operation of large amounts of data. [0003] Traditional performance tuning starts with SQL optimization and database performance, but the bottleneck of the row-based database mysql / oracle is that when the amount of data exceeds tens of millions, the row-based database will expose performance weaknesses. Aft...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/21G06F16/27
CPCG06F16/214G06F16/27
Inventor 胡立鑫
Owner UNICLOUD (NANJING) DIGITAL TECH CO LTD