Data synchronization method of database system, storage medium and device

By parsing synchronized transactions from the transaction log of the source database into the source memory in the database system, and resuming parsing from the transaction with the reference sequence number in case of anomalies, the problem of data synchronization interruption is solved, the continuity and reliability of data synchronization are achieved, and storage pressure is reduced.

CN116701528BActive Publication Date: 2026-04-10CETC JINCANG (BEIJING) TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
CETC JINCANG (BEIJING) TECH CO LTD
Filing Date
2023-03-06
Publication Date
2026-04-10

AI Technical Summary

Technical Problem

During the data synchronization process of a database system, abnormal situations such as machine crashes, disk failures, or operating system failures can cause errors in the data written to physical files, leading to interruption of data synchronization and affecting user experience.

Method used

The synchronization transaction is parsed from the transaction log of the source database, written to the source memory, and then read from the physical file to the target memory. During the reading process, the synchronization transaction is checked for errors. If an error occurs, the sequence number of the previous transaction of the erroneous transaction is obtained as the reference sequence number. The synchronization transaction is re-parsed starting from the transaction corresponding to the reference sequence number to ensure the continuity and reliability of data synchronization.

Benefits of technology

Even in abnormal situations, the database system will not interrupt data synchronization. By resuming parsing from the transaction before the one that caused the error, the continuity and reliability of data synchronization are guaranteed, reducing unnecessary physical file storage pressure.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116701528B_ABST
    Figure CN116701528B_ABST
Patent Text Reader

Abstract

The application provides a data synchronization method of a database system, a storage medium and equipment. The data synchronization method of the database system comprises the following steps: parsing a synchronization transaction from a transaction log of a source end database into a source end memory; writing the synchronization transaction from the source end memory into a physical file; reading the synchronization transaction from the physical file into a target end memory; detecting whether the synchronization transaction is wrong in the reading process; if the synchronization transaction is wrong, obtaining a sequence number of a previous transaction of the wrong synchronization transaction, and recording the sequence number as a reference sequence number; and starting from a transaction corresponding to the reference sequence number, parsing the synchronization transaction from the transaction log of the source end database again. Even if an abnormal situation occurs in the process of data synchronization, resulting in a wrong synchronization transaction when writing from the source end memory into the physical file, and the synchronization transaction is wrong when reading from the physical file into the target end memory, the database system will not interrupt the data synchronization, and the continuity of the data synchronization is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database, in particular to a data synchronization method of database system, storage medium and equipment. BACKGROUND

[0002] In the use of database system, data synchronization work between different databases is needed. In the process of data synchronization work, the increase, deletion and modification changes of data are analyzed from the log of source database, and the changes belonging to the same transaction are combined together to synchronize to the target database in transaction unit. In order to ensure reliability, the synchronization data is usually written into physical file, that is, hard disk and other hardware storage, and then the synchronization data is written from the physical file to the target database.

[0003] In the synchronization process, if abnormal conditions such as machine abnormal downtime, disk failure, operating system failure and the like occur, and the synchronization program is writing data into the physical file, the data written into the physical file will be wrong. In turn, it will cause errors when reading data from the physical file subsequently, thereby causing the data synchronization to be interrupted and affecting user use. SUMMARY

[0004] An object of the present application is to provide a data synchronization method of database system, storage medium and equipment which can solve any of the above problems.

[0005] A further object of the present application is to reduce the storage pressure of physical file.

[0006] Another further object of the present application is to further ensure the continuity and reliability of data synchronization.

[0007] In particular, the present application provides a data synchronization method of database system, comprising:

[0008] parsing synchronization transactions from the transaction log of source database to source memory;

[0009] writing synchronization transactions from source memory to physical file;

[0010] reading synchronization transactions from physical file to target memory;

[0011] detecting whether the synchronization transactions are wrong in the reading process;

[0012] if the synchronization transactions are wrong, obtaining the sequence number of the last transaction of the wrong synchronization transactions, which is recorded as reference sequence number;

[0013] starting from the transaction corresponding to the reference sequence number, re-parsing synchronization transactions from the transaction log of source database.

[0014] Optionally, after the reference sequence number is acquired, the data synchronization method comprises:

[0015] acquiring the location of the transaction corresponding to the reference sequence number in the physical file;

[0016] emptying all physical files after the location of the transaction corresponding to the reference sequence number.

[0017] Optionally, the step of parsing the synchronization transaction from the transaction log of the source end database to the source end memory comprises:

[0018] calculating the check value of the synchronization transaction according to a preset calculation mode, denoted as a first check value;

[0019] Optionally, the step of writing the synchronization transaction from the source end memory to the physical file comprises:

[0020] calculating the check value of the synchronization transaction according to a preset calculation mode, denoted as a second check value;

[0021] judging whether the first check value and the second check value are the same;

[0022] if the first check value and the second check value are not the same, parsing the synchronization transaction from the transaction log of the source end database again.

[0023] Optionally, the step of calculating the check value of the synchronization transaction according to a preset calculation mode comprises:

[0024] composing a string from all characters of the synchronization transaction;

[0025] calculating the MD5 value of the string.

[0026] Optionally, the step of parsing the synchronization transaction from the transaction log of the source end database to the source end memory comprises:

[0027] parsing the synchronization transaction from the transaction log of the source end database;

[0028] storing the synchronization transaction in a preset source end memory queue.

[0029] Optionally, the step of reading the synchronization transaction from the physical file to the target end memory comprises:

[0030] reading the synchronization transaction from the physical file;

[0031] storing the synchronization transaction in a preset target end memory queue.

[0032] Optionally, the step of detecting whether the synchronization transaction is erroneous during the reading process comprises:

[0033] if the synchronization transaction is not erroneous, writing the synchronization transaction from the target end memory to the target end database.

[0034] Optionally, after the reference sequence number is acquired, the data synchronization method comprises:

[0035] putting the reference sequence number into a preset shared queue.

[0036] According to another aspect of the present application, there is also provided a machine readable storage medium having stored thereon a machine executable program which, when executed by a processor, implements the data synchronization method of the database system according to any one of the above.

[0037] According to still another aspect of the present application, there is also provided a computer device comprising a memory, a processor and a machine executable program stored on the memory and running on the processor, and the processor implements the data synchronization method of the database system according to any one of the above when executing the machine executable program.

[0038] The data synchronization method of the database system of the present application can parse the synchronization transaction from the transaction log of the source database to the source memory, write the synchronization transaction from the source memory to the physical file, read the synchronization transaction from the physical file to the target memory, and detect whether the synchronization transaction is erroneous during the reading process. If the synchronization transaction is erroneous, the sequence number of the last transaction of the erroneous synchronization transaction is acquired as a reference sequence number, and then the synchronization transaction is parsed from the transaction log of the source database starting from the transaction corresponding to the reference sequence number. That is, even if abnormal situations such as machine abnormal downtime, disk failure, operating system failure, etc. occur during the data synchronization, resulting in erroneous synchronization transaction when writing from the source memory to the physical file, so that the synchronization transaction is erroneous when reading from the physical file to the target memory, the database system will not interrupt the data synchronization. Instead, the reference sequence number is acquired, and the data synchronization is restarted from the transaction corresponding to the reference sequence number, which helps to ensure the continuity of the data synchronization. On the other hand, since the transaction is parsed from the last transaction of the erroneous transaction, no transaction is skipped, which helps to ensure the reliability of the data synchronization.

[0039] Further, the data synchronization method of the database system of the present application can acquire the sequence number of the last transaction of the erroneous synchronization transaction as a reference sequence number when the synchronization transaction is erroneous when reading from the physical file to the target memory, and then clear all data after the transaction corresponding to the reference sequence number in the physical file, so as to reduce unnecessary physical files and reduce storage pressure.

[0040] Further, the data synchronization method of the database system of the present application calculates the check value of the synchronization transaction after resolving the synchronization transaction from the transaction log of the source database into the source memory, and calculates the check value of the synchronization transaction again in the same way after writing the synchronization transaction from the source memory into the physical file. By comparing the two check values, if the two check values are not the same, it means that the synchronization transaction is written into the physical file from the source memory, so that the error transaction can be parsed from the source database after the error transaction is written into the physical file, and the continuity and reliability of data synchronization are further ensured.

[0041] The above and other objects, advantages and features of the present application will become more apparent from the following detailed description of some embodiments thereof, when taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0042] Hereinafter, some embodiments of the present application will be described in detail with reference to the accompanying drawings. The same reference numbers in different drawings denote the same or similar components or parts. It will be understood by those skilled in the art that the drawings are not necessarily drawn to scale. In the drawings:

[0043] Figure 1 is a schematic flowchart of a data synchronization method of a database system according to an embodiment of the present application;

[0044] Figure 2 is a schematic flowchart of a data synchronization method of a database system according to another embodiment of the present application;

[0045] Figure 3 is a schematic flowchart of a data synchronization method of a database system according to yet another embodiment of the present application;

[0046] Figure 4 is a schematic diagram of a machine-readable storage medium according to an embodiment of the present application;

[0047] Figure 5 is a schematic diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] Those skilled in the art should understand that the embodiments described below are only a part of the embodiments of the present application, and are not intended to limit the protection scope of the present application. Based on the embodiments provided by the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor should fall within the protection scope of the present application.

[0049] It is to be appreciated that the logic and / or steps represented in the flow diagrams and / or described herein, for example, can be considered as a list of executable instructions for implementing logic functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or a combination thereof.

[0050] As shown in FIG. 1, in one embodiment, the data synchronization method of the database system generally includes: Figure 1

[0051] Step S101, parsing the synchronization transaction from the transaction log of the source database into the source memory. Specifically, the source data synchronization software parses the transaction operation from the transaction log of the source database, forms a complete transaction by grouping the transaction operations belonging to the same transaction, that is, the synchronization transaction for data synchronization, and stores the synchronization transaction into the source memory, that is, into the memory of the source host (such as a computer).

[0052] In one embodiment, this step includes parsing the synchronization transaction from the transaction log of the source database and storing the synchronization transaction into a preset source memory queue. Specifically, the queue is a first-in first-out data structure, and by placing the synchronization transaction in the memory queue, the subsequent processing process of the transaction in order is facilitated.

[0053] Step S102, writing the synchronization transaction from the source memory to the physical file. Specifically, that is, writing the synchronization transaction into the hardware storage such as the disk of the source host to solidify the synchronization transaction and form a physical file, thereby avoiding data loss.

[0054] Step S103, reading the synchronization transaction from the physical file to the target memory. Specifically, the target data synchronization software reads the synchronization transaction written into the physical file into the target memory, that is, reads the synchronization transaction existing in the physical file into the memory of the target host.

[0055] In one embodiment, this step includes reading the synchronization transaction from the physical file and storing the synchronization transaction into a preset target memory queue. Specifically, the queue is a first-in first-out data structure, and by placing the synchronization transaction in the target memory queue, the synchronization of the transaction in the target database in order is facilitated.

[0056] ​Step S104, detecting whether the synchronization transaction is failed during reading, if yes, executing step S105, if no, executing step S107. Specifically, during reading the synchronization transaction from the physical file to the target end memory, the synchronization transaction needs to be parsed, so as to transform the synchronization transaction into a specific message format of the target end data synchronization software. If the synchronization transaction written in the physical file is failed due to abnormal conditions, an error will be reported during the transformation.

[0057] Step S105, obtaining the sequence number of the last transaction of the failed synchronization transaction, denoted as the reference sequence number. Specifically, the sequence number is a number identifying the execution order of the transaction. If the synchronization transaction is failed, the sequence number of the last transaction of the failed synchronization transaction is obtained, that is, the sequence number of the last transaction read into the target end memory is obtained. Exemplarily, assuming that the transaction with the sequence number 100 is failed, the reference sequence number is 99.

[0058] Step S106, starting from the transaction corresponding to the reference sequence number, the synchronization transaction is parsed from the transaction log of the source end database again. Specifically, the work of parsing the transaction log of the source end database is stopped, the position of the first transaction operation of the transaction corresponding to the reference sequence number in the transaction log of the source end database is found, and the transaction corresponding to the reference sequence number is parsed from the first transaction operation of the transaction corresponding to the reference sequence number, that is, the data synchronization process is restarted from the transaction corresponding to the reference sequence number. Because the sequence number of the failed transaction is after the reference sequence number, the first transaction operation of the failed transaction is after the first transaction operation of the last transaction. Starting from the first transaction operation of the last transaction of the failed transaction, the failed transaction can be parsed completely. The transaction operation of the transaction with the sequence number before the reference sequence number parsed in the process of restarting the parsing can be discarded.

[0059] Step S107, writing the synchronization transaction from the target end memory to the target end database. If the reading of the synchronization transaction is not failed, the synchronization transaction is written to the target end database, so as to complete the synchronization of the synchronization transaction between the source end database and the target end database.

[0060] Further, in other words, the four work threads of parsing the synchronization transaction from the transaction log of the source end database to the source end memory, writing the synchronization transaction from the source end memory to the physical file, reading the synchronization transaction from the physical file to the target end memory, and writing the synchronization transaction from the target end memory to the target end database, constitute a pipeline type data synchronization mechanism. When the reading of the synchronization transaction from the physical file to the target end memory is failed, the current pipeline process is stopped, and the pipeline process is restarted from the transaction corresponding to the reference sequence number.

[0061] Further, after the reference sequence number is acquired, the reference sequence number is put into a preset shared queue. Specifically, because there are four threads for data synchronization, by pre-configuring the shared queue, when an error occurs in a thread, the error content (i.e., error type and reference sequence number) and thread ID (Identity document, identification number) can be put into the shared queue for reading, so that other threads can obtain the error of the error thread, and communication between threads is realized.

[0062] In the scheme of the embodiment, by parsing the synchronization transaction from the transaction log of the source-end database into the source-end memory, writing the synchronization transaction from the source-end memory into a physical file, and reading the synchronization transaction from the physical file into the target-end memory, whether the synchronization transaction is erroneous is detected in the reading process. If the synchronization transaction is erroneous, the sequence number of the last transaction of the erroneous synchronization transaction is acquired, which is recorded as a reference sequence number. Then, the synchronization transaction is parsed from the transaction log of the source-end database starting from the transaction corresponding to the reference sequence number.

[0063] That is, even if abnormal situations such as machine abnormal downtime, disk failure, operating system failure, etc. occur in the process of data synchronization, resulting in erroneous synchronization transactions written from the source-end memory into the physical file, so that the synchronization transaction is erroneous when read from the physical file into the target-end memory, the database system will not interrupt the data synchronization. Instead, the reference sequence number is acquired, and the data synchronization work is restarted from the transaction corresponding to the reference sequence number. On the one hand, this helps to ensure the continuity of data synchronization. On the other hand, because the transaction is parsed from the previous transaction of the erroneous transaction, no transaction is skipped, which is beneficial to ensure the reliability of data synchronization.

[0064] As shown in FIG. 1, Figure 2 In one embodiment, the data synchronization method of the database system generally includes:

[0065] Step S201: parsing the synchronization transaction from the transaction log of the source-end database into the source-end memory.

[0066] Step S202: calculating the check value of the synchronization transaction according to a preset calculation method, which is recorded as a first check value. Specifically, this step includes forming a string from all characters of the synchronization transaction, and calculating the MD5 (Message-Digest Algorithm, information digest algorithm) value of the string. That is, a long string of characters is formed from all characters of the synchronization transaction, and the MD5 value is calculated.

[0067] Step S203: writing the synchronization transaction from the source-end memory into a physical file.

[0068] Step S204, the check value of the synchronization transaction is calculated according to a pre-designed calculation mode, and is recorded as a second check value. Specifically, the check value of the synchronization transaction is calculated again according to the same mode as that for calculating the first check value, and is recorded as the second check value.

[0069] Step S205, it is judged whether the first check value and the second check value are the same. If yes, step S207 is executed; if no, step S206 is executed.

[0070] Step S206, the synchronization transaction is parsed from the transaction log of the source end database again. Specifically, if the first check value and the second check value are not the same, it indicates that the synchronization transaction has an error in the process of being written from the source end memory to the physical file. Therefore, the transaction does not need to be further synchronized, and the transaction needs to be parsed from the transaction log of the source end database again.

[0071] Step S207, the synchronization transaction is read from the physical file to the target end memory. Specifically, if the first check value and the second check value are the same, it indicates that the synchronization transaction parsed from the source end database is the same as the content of the transaction after being written from the source end memory to the physical file. Therefore, the synchronization of the transaction is continued, and the target end data synchronization software reads the synchronization transaction written to the physical file into the target end memory, that is, reads the synchronization transaction existing in the physical file into the memory of the target end host.

[0072] Step S208, it is detected whether the synchronization transaction has an error in the reading process. If yes, step S209 is executed; if no, step S211 is executed.

[0073] Step S209, the sequence number of the last transaction of the error synchronization transaction is obtained, and is recorded as a reference sequence number.

[0074] Step S210, the synchronization transaction is parsed from the transaction log of the source end database again from the transaction corresponding to the reference sequence number.

[0075] Step S211, the synchronization transaction is written from the target end memory to the target end database.

[0076] In the scheme of the embodiment, the check value of the synchronization transaction is calculated after the synchronization transaction is parsed from the transaction log of the source end database to the source end memory, and the check value of the synchronization transaction is calculated again in the same calculation mode after the synchronization transaction is written from the source end memory to the physical file. By comparing the check values calculated twice, if the check values calculated twice are not the same, it indicates that the synchronization transaction has an error when being written from the source end memory to the physical file. Therefore, the error transaction can be parsed from the source end database again after the error transaction is written in the physical file, and the continuity and reliability of the data synchronization are further ensured.

[0077] As Figure 3As shown, in one embodiment, the data synchronization method of the database system generally comprises:

[0078] Step S301, parsing the synchronization transactions from the transaction log of the source database into the source memory.

[0079] Step S302, writing the synchronization transactions from the source memory into the physical file.

[0080] Step S303, reading the synchronization transactions from the physical file into the target memory.

[0081] Step S304, detecting whether the synchronization transactions are erroneous during the reading process, if yes, executing step S305, if no, executing step S308.

[0082] Step S305, obtaining the sequence number of the last transaction of the erroneous synchronization transaction, denoted as the reference sequence number.

[0083] Step S306, obtaining the position of the transaction corresponding to the reference sequence number in the physical file, and emptying all the physical files after the position of the transaction corresponding to the reference sequence number. Specifically, all the transactions corresponding to the reference sequence number and the sequence numbers after the reference sequence number which have been written into the physical file are cleared.

[0084] Step S307, re-parsing the synchronization transactions from the transaction log of the source database starting from the transaction corresponding to the reference sequence number. It should be noted that step S306 and step S307 have no sequence.

[0085] Step S308, writing the synchronization transactions from the target memory into the target database.

[0086] In the scheme of the embodiment, in the case that the synchronization transactions are erroneous when being read from the physical file into the target memory, the sequence number of the last transaction of the erroneous synchronization transaction is obtained, denoted as the reference sequence number, and then all the data after the transaction corresponding to the reference sequence number in the physical file is emptied, so as to reduce unnecessary physical files and reduce the storage pressure.

[0087] The embodiment further provides a machine readable storage medium and a computer device. Figure 4 Fig. 1 is a schematic diagram of a machine readable storage medium 10 according to an embodiment of the present application. Figure 5 Fig. 2 is a schematic diagram of a computer device 20 according to an embodiment of the present application.

[0088] The machine readable storage medium 10 has a machine executable program 11 stored thereon, and the machine executable program 11 is executed by a processor to implement the data synchronization method of the database system of any of the above embodiments.

[0089] The computer device 20 can include a memory 210, a processor 220, and a machine executable program 11 stored on the memory 210 and running on the processor 220, and the processor 220 implements the data synchronization method of the database system of any of the above embodiments when executing the machine executable program 11.

[0090] It should be noted that the logic and / or steps represented in the flowcharts and / or described herein, for example, can be considered as a list of executable instructions for implementing logic functions, and can be embodied in any machine readable storage medium for use by or in connection with an instruction execution system, apparatus or device, such as a computer-based system, a system including a processor or other system that can fetch the instructions from the instruction execution system, apparatus or device and execute the instructions, or in conjunction with such an instruction execution system, apparatus or device.

[0091] For the description of the present embodiment, the machine readable storage medium 10 can be any device that can contain, store, communicate, propagate or transport the program for use by or in connection with the instruction execution system, apparatus or device, or in conjunction with such an instruction execution system, apparatus or device. More specific examples (non-exhaustive list) of computer readable medium include the following: electrical connections having one or more wires (electronic devices), portable computer diskette (magnetic devices), random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), fiber optic devices, and portable compact disk read only memory (CDROM). In addition, the machine readable storage medium 10 can even be paper or other suitable medium on which the program can be printed, as the program can be electronically obtained, for example, by optical scanning of the paper or other medium, followed by electronic conversion of the optical scanning into an electronically available form, and then storing the program in computer memory.

[0092] It should be understood that parts of the present application can be implemented in hardware, software, firmware or a combination thereof. In the above described embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.

[0093] The computer device 20 can be, for example, a server, a desktop computer, a notebook computer, a tablet computer, or a smart phone. In some examples, the computer device 20 can be a cloud computing node. The computer device 20 can be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules can include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types. The computer device 20 can be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules can be located in both local and remote computer system storage media including memory storage devices.

[0094] The computer device 20 can include a processor 220 adapted to execute instructions stored in memory 210, which in operation provide temporary storage of instructions during execution by the processor 220. The processor 220 can be a single core processor, multi-core processor, computing cluster, or any number of other configurations. The memory 210 can include random access memory (RAM), read only memory (ROM), flash memory, or any other suitable memory systems.

[0095] The processor 220 can be connected through a system interconnect (e.g., a PCI, PCI-Express, or other interconnect) to an I / O interface adapted to connect the computer device 20 to one or more I / O devices (input / output devices). The I / O devices can include, for example, a keyboard and a pointing device, where the pointing device can include a touchpad or touchscreen, among others. The I / O devices can be built-in components of the computer device 20, or can be devices externally connected to the computer device.

[0096] The processor 220 can also be linked through the system interconnect to a display interface adapted to connect the computer device 20 to a display device. The display device can include a display screen that is a built-in component of the computer device 20. The display device can also include a computer monitor, television, or projector, among others, that is externally connected to the computer device 20. Further, a network interface controller (NIC) can be adapted to connect the computer device 20 to a network through the system interconnect. In some embodiments, the NIC can use any suitable interface or protocol, such as an Internet Small Computer Systems Interface, among others, to transfer data. The network can be a cellular network, a radio network, a wide area network (WAN), a local area network (LAN), or the Internet, among others. Remote devices can be connected to the computer device through the network.

[0097] The flowcharts provided by the embodiments are not intended to indicate that the operations of the methods will be executed in any particular order, or that all of the operations of the methods are included in every case. In addition, the methods can include additional operations. Additional changes can be made to the above-described methods within the scope of the technical ideas provided by the embodiments.

[0098] Thus, those skilled in the art will recognize that the present application has been well- described and clearly demonstrated by way of example, and that many changes and modifications can be made thereto without departing from the spirit and scope of the application. It is therefore intended to cover in the appended claims all such changes and modifications that are within the scope of this application.

Claims

1. A data synchronization method of a database system, comprising: parsing a synchronization transaction from a transaction log of a source database to a source memory; writing the synchronization transaction from the source memory to a physical file; reading the synchronization transaction from the physical file to a target memory; detecting whether the synchronization transaction is erroneous during the reading process; if the synchronization transaction is erroneous, obtaining a sequence number of a previous transaction of the erroneous synchronization transaction, denoted as a reference sequence number; re-parsing a synchronization transaction from the transaction log of the source database starting from a transaction corresponding to the reference sequence number.

2. The data synchronization method of a database system according to claim 1, wherein, After the reference sequence number is obtained, the data synchronization method comprises: obtaining a position of the transaction corresponding to the reference sequence number in the physical file; emptying all physical files after the position of the transaction corresponding to the reference sequence number.

3. The data synchronization method of a database system according to claim 1, wherein, After the step of parsing a synchronization transaction from a transaction log of a source database to a source memory, the method comprises: calculating a check value of the synchronization transaction according to a preset calculation mode, denoted as a first check value; After the step of writing the synchronization transaction from the source memory to a physical file, the method comprises: calculating a check value of the synchronization transaction according to the preset calculation mode, denoted as a second check value; judging whether the first check value and the second check value are the same; if the first check value and the second check value are not the same, re-parsing the synchronization transaction from the transaction log of the source database.

4. The data synchronization method of a database system according to claim 3, wherein, The step of calculating a check value of the synchronization transaction according to a preset calculation mode comprises: composing a string from all characters of the synchronization transaction; calculating an MD5 value of the string.

5. The data synchronization method of a database system according to claim 1, wherein, The step of parsing a synchronization transaction from a transaction log of a source database to a source memory comprises: parsing a synchronization transaction from a transaction log of a source database; storing the synchronization transaction in a preset source memory queue.

6. The data synchronization method of a database system according to claim 1, wherein, The step of reading the synchronization transaction from the physical file to a target memory comprises: reading the synchronization transaction from the physical file; storing the synchronization transaction in a preset target memory queue.

7. The data synchronization method of a database system according to claim 1, wherein, After the step of detecting whether the synchronization transaction is erroneous during the reading process, the method comprises: if the synchronization transaction is not erroneous, writing the synchronization transaction from the target memory to a target database.

8. The data synchronization method of a database system according to claim 1, wherein, After the reference sequence number is obtained, the data synchronization method comprises: putting the reference sequence number into a preset shared queue. 9.A machine readable storage medium having stored thereon a machine executable program which, when executed by a processor, implements the data synchronization method of a database system according to any one of claims 1 to 8. 10.A computer device comprising a memory, a processor, and a machine executable program stored on the memory and running on the processor, and the processor implements the data synchronization method of a database system according to any one of claims 1 to 8 when executing the machine executable program.

Citation Information

Patent Citations

  • Database data synchronization system, method and device

    CN109947801A

  • Heterogeneous database synchronous data consistency verification method and device

    CN110347746A