HBase-based heterogeneous database online exchange and sharing system in big data environment

Abstract

The invention discloses an HBase-based heterogeneous database online exchange and sharing system in a big data environment. The system comprises a heterogeneous database unified access component, a data collection module, a data mapping module, a data conversion module, a data fusion module, a data distribution module and a data delivery module. The heterogeneous database unified access component provides a unified database access interface; the data collection module collects data in a source database; the data mapping module establishes a mapping relation between all heterogeneous databases and a unified semantic sharing model; the data conversion module realizes data conversion; the data fusion module fuses the unified semantic sharing model with a unified semantic sharing model of historical data and stores a fused model into an HBase intermediate database; the data distribution module sends to-be-exchanged field data to the data conversion module according to a timestamp; the data delivery module pushes the data to a corresponding target database. Through the system, automatic online exchange and sharing of mass data between the heterogeneous databases are realized, and exchange and sharing efficiency is high.

Description

technical field [0001] The invention relates to an HBase-based online exchange and sharing system for heterogeneous databases in a big data environment, and belongs to the field of data exchange. Background technique [0002] At present, heterogeneous databases are widely used in enterprise application systems. These heterogeneous databases often have large differences, such as different data storage structures, different data types, and different data representation methods. Very inconvenient. The rapid development of informatization construction has led to a huge amount of information that needs to be exchanged and shared, and information exchange has the characteristics of multi-source, heterogeneous, and massive. How to realize efficient and automatic online exchange and sharing of big data between heterogeneous databases has become a problem that needs to be solved urgently. [0003] The data import and export tool provided by SQL Server realizes the data migration of...

AI Technical Summary

Problems solved by technology

The unified exchange model adopted by this method is in the form of XML file, which does not support high concurrent reading and writing, the efficiency of exchange and sharing is low, and the amount of data that can be processed is small

Images