Log reading method based on log analysis synchronization and data synchronization system

A log and database technology, applied in the field of log reading method and data synchronization system based on log parsing and synchronization, can solve problems such as data synchronization errors

Active Publication Date: 2020-10-30
WUHAN DAMENG DATABASE
View PDF9 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

After testing, this phenomenon has a high probability of occurring in the running environment of the virtual machine. Once it occurs, it will cause data synchronization errors. What is more serious is that if the database fails to restart at this time, the database will not be able to use this damaged REDO log for recovery

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Log reading method based on log analysis synchronization and data synchronization system
  • Log reading method based on log analysis synchronization and data synchronization system
  • Log reading method based on log analysis synchronization and data synchronization system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0048] At present, although the data lock technology can only allow one process to access a certain file, the data lock is applicable to the same type of database or the database of the same manufacturer. The write operations on one side and the read operations on the destination database side originate from databases of two different manufacturers, so the above data lock cannot be used, and the method in this embodiment is applicable to databases of different sources or types.

[0049] combine image 3 , in this embodiment, a synchronization system is deployed on the source database and the destination database, the source database synchronization system reads logs from the source database, and the target database synchronization system is responsible for sending the synchronization operation application from the source to the destination database.

[0050] Among them, when the source-side data synchronization system starts, it needs to initialize an LSN information linked l...

Embodiment 2

[0085] Based on the aforementioned embodiment 1, here, taking ORACLE as an example, the implementation process of the aforementioned embodiment 1 is illustrated:

[0086] Build a data synchronization system on database A and database B, and set the minimum delay of data synchronization to Y seconds, set the time interval for polling the database log sequence number to Z seconds, and the source data synchronization system is in the database An auxiliary table T (C INT) is created on A to advance the log.

[0087] The database interaction process of the source log synchronization system is as follows:

[0088] (1) The database interaction thread obtains the largest log LSN and current time of the current database from the source database every Z seconds, and the obtained SQL is select dbms_flashback.get_system_change_number,sysdate from dual; get the value of the log serial number LSN and time for LSN1 and TIME1.

[0089] Step (2) Save LSN1 and TIME1 in the LSN information cha...

Embodiment 3

[0096] see Figure 4 , Figure 4 It is a schematic structural diagram of a data synchronization system provided by an embodiment of the present invention. The data synchronization system in this embodiment includes one or more processors 41 and a memory 42 . in, Figure 4 A processor 41 is taken as an example.

[0097] Processor 41 and memory 42 can be connected by bus or other means, Figure 4 Take connection via bus as an example.

[0098] The memory 42, as a non-volatile computer-readable storage medium based on the log reading method, can be used to store non-volatile software programs, non-volatile computer-executable programs and modules, the methods of the above-mentioned embodiments and the corresponding program instructions. The processor 41 executes various functional applications and data processing by running non-volatile software programs, instructions and modules stored in the memory 42 to implement the methods of the foregoing embodiments.

[0099] Wherei...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a log reading method based on log analysis synchronization and a data synchronization system, and the log reading method comprises the steps of obtaining the current maximum log serial number and current time of a source end database in a polling interval; establishing association between the current maximum log serial number and the current time, and adding the associationto an LSN information chain table; reading a to-be-synchronized log record from a log file, obtaining a log serial number of the to-be-synchronized log record, and obtaining writing time to which theto-be-synchronized log record belongs based on the LSN information chain table; performing strategy log reading and synchronization according to the difference value between the writing time to whichthe log record to be synchronized belongs and the current system time. According to the invention, the log file access positions of the data synchronization process and the database process are staggered through the time difference, and therefore read-write conflicts can be effectively prevented.

Description

technical field [0001] The invention belongs to the technical field of data synchronization, and more specifically relates to a log reading method and a data synchronization system based on log parsing and synchronization. Background technique [0002] In the solution based on log parsing and synchronization, a database log capture process will be deployed on the source database to continuously scan and read the online logs of the database, so as to capture the running logs of the database for data synchronization at the first time. This architecture will cause the log file of the database to be opened by two or more processes at the same time. Among them, the database opens the online log file and writes the log, and the data synchronization process opens the online log file to read the log. If the database write and data Synchronous read operations concurrently on the same file offset of the same online log file may cause read-write conflicts, causing the new data to be wr...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/17G06F16/178G06F16/23G06F16/27
CPCG06F16/1734G06F16/178G06F16/23G06F16/27Y02D10/00
Inventor 孙峰付铨彭青松刘启春
Owner WUHAN DAMENG DATABASE
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap