Data synchronous loading method based on database log

Abstract

The invention discloses a data synchronous loading method based on a database log. The method includes the steps that a data synchronous loading system is provided, wherein the system comprises a log streaming data analysis module, a data sending module, a data receiving and caching module, a data adaptation and recovery module and a data loading module, the log streaming data analysis module and the data sending module are arranged in a source-side database, the data receiving and caching module, the data adaptation and recovery module and the data loading module are arranged in a target-side database, the log streaming data analysis module is connected with the data sending module, the data sending module is connected with the data receiving and caching module, and the data receiving and caching module is connected with the data adaptation and recovery module and the data loading module; a log file generated in the database operation process is analyzed to obtain data operation motion, then data is sent to the target side in a network sending mode, and a corresponding analysis mode is selected to load the data according to the type of the target-side database. Thus, the purpose of different-database backup is achieved, the influence on a service system in the database backup process is effectively reduced, the flexibility of data backup is improved, and the reliability and consistency of the loaded data are improved.

Description

technical field [0001] The invention belongs to the technical field of data exchange, synchronization and data backup, and in particular relates to a data synchronous loading method based on a database log. Background technique [0002] With the continuous integration and in-depth analysis of various business systems, there is an urgent need for a new method for data synchronization between different database systems. Taking the power business system as an example, there is a parent-child relationship, total-score relationship data tables and businesses among the headquarters, provinces (cities), and local business systems; but for different business system construction, the early planning is not perfect, resulting in The database versions of each business system are inconsistent, which causes the technical bottleneck of synchronizing the local data table to the provincial (city) business system data table. If the SQL copy of the data operation is simply performed, random da...

AI Technical Summary

Problems solved by technology

Taking the power business system as an example, there is a parent-child relationship, total-score relationship data tables and businesses among the headquarters, provinces (cities), and local business systems; but for different business system construction, the early planning is not perfect, resulting in The database versions of each business system are inconsistent, which causes the technical bottleneck of synchronizing the local data table to the provincial (city) business system data table. If the SQL copy of the data operation is simply performed, random data and time data will be inconsistent. If the data changes are obtained through triggers, the response time of the business system will be greatly reduced

The traditional database master-backup replication process is based on the database stream replication mode, and only the same type of database replication can be achieved. During the replication process, the processing performance of the database will be occupied and the response speed of the business system will be reduced.

Images