Log synchronization method and device, storage medium, electronic device

Abstract

The disclosure relates to a log synchronization method and device, and belongs to the technical field of data processing. The method includes: reading a transaction log from a source database and storing the transaction log in a plurality of read ring queues; storing the transaction log in each of the read ring queues into a consumption ring queue, and storing the transaction log in a plurality of read ring queues. The transaction log in the consumer ring queue is sent to the thread that establishes a connection with the target database; the thread writes the transaction log into the target database and records the log code corresponding to the successfully written transaction log. The method improves the speed of transaction log storage and greatly reduces memory consumption by adopting the circular queue to store the transaction log.

Description

technical field [0001] The present disclosure relates to the technical field of data processing, and in particular, to a log synchronization method, a log synchronization device, a computer-readable storage medium, and electronic equipment. Background technique [0002] With the development of Internet technology, more and more users use the Internet. In order to ensure the stability of the system, the deployment structure of the system has also begun to change from the initial deployment of a single computer room to the deployment of multiple computer rooms in the same city and remote multi-computer rooms. . When the system is deployed in multiple computer rooms, the generated data will also be distributed in multiple computer rooms, which also brings great challenges to data synchronization among multiple computer rooms. [0003] Currently, there are two ways to keep data synchronized: one is to rely on the synchronization mechanism provided by the data source software it...

AI Technical Summary

Problems solved by technology

[0004] However, the above two methods have corresponding shortcomings: first, for MySql, relying on the synchronization mechanism provided by MySql itself, when the load pressure on the source database is high, it will cause data synchronization delay; for Redis, use Redis data During synchronization, when the primary node is replaced or the log buffer range of the primary node is exceeded, full data synchronization will occur, resulting in extended data synchronization time, and Redis cannot provide external services during data synchronization; secondly, apply double write to ensure data consistency The logic invades the application program, resulting in relatively complex program logic; when writing to multiple data sources at the same time, when some of the data sources fail to write, it is difficult to achieve data consistency in all data sources

Images