Method for achieving automatic synchronization of multi-source heterogeneous data resources

Abstract

The invention relates to a method for achieving automatic synchronization of multi-source heterogeneous data resources. A metadata service node, a service node and a production node are related in the method, the metadata service node is responsible for storing a data dictionary in a distributed heterogeneous database system, the service node is responsible for analyzing user requests in the distributed heterogeneous database system and carrying out synchronization with a database on the production node periodically, and the production node is responsible for requests such as inquiring and updating in a corresponding production application system, wherein the service node is formed by a configuration recognizer, a blog parser and a data synchronizer, the configuration recognizer is used for analyzing the configuration of a source database on the production node and selecting a feasible synchronizing mode, the blog parser is used for analyzing blogs of different dataset systems and is a component for first-phase preparing work of an increment synchronization mode, and the data synchronizer carries out data synchronization according to the synchronizing mode of the configuration recognizer. The method for achieving the automatic synchronization of the multi-source heterogeneous data resources enables the automatic synchronization of the multi-source heterogeneous data resources to be achieved, and can be used in data resource processing.

Description

technical field [0001] The invention relates to the field of computer application technology, in particular to a method for realizing automatic synchronization of multi-source heterogeneous data resources. Background technique [0002] In the era of big data, storing and organizing data is the basis of upper-level application services. Through the distributed heterogeneous database system, different data sources can be integrated into a logically unified data source to provide data access interfaces for upper-layer application services. The distributed heterogeneous database system cannot affect the production node, so the production system can only be used as the data source of the heterogeneous database system, that is, the data on the production node is synchronized to the front-end computer as the data operation in the heterogeneous database system object. The data on the front-end nodes and production nodes need to be updated regularly to provide an effective data sou...

AI Technical Summary

Problems solved by technology

However, there are some problems with this mechanism: (1) The production node needs to enable the log function of the database and configure relevant parameters correctly, otherwise incremental synchronization cannot be performed; (2) The database system generally only provides an interface for parsing log files, but does not Provide a complete incremental synchronization function; (3) Different database systems have different log recording methods, and parsing logs requires users to have certain relevant knowledge. These factors limit the scope of use of incremental synchronization

This synchronization method consumes a lot of system resources, and some databases do not provide complete data migration tools, which cannot realize synchronization automation

Images