Dual-center database synchronization method, device, electronic device and storage medium

Abstract

The present invention discloses a dual-center database synchronization method, including: a data loading module of a target-end database loads the DML transaction data of a source-end database itself, and performs data operations on a preset identification table of the target-end database, and a log file module of the target-end database stores the acquired DML transaction data of the source-end database itself; a log analysis module of the target-end database parses the DML transaction data of the target-end database; when the log analysis module of the target-end database obtains a DML statement of the target-end database identification after parsing, the DML transaction data of the target-end database corresponding to the DML statement of the target-end database identification is deleted, and the undeleted DML transaction data of the target-end database is sent to the data loading module of the source-end database. The present invention avoids repeated storage of data and the formation of a data synchronization dead loop. The present invention also discloses a device, an electronic device, and a computer-readable storage medium for implementing the above method.

Description

Technical Field

[0001] The present invention relates to the field of database technology, and in particular to a dual-center database synchronization method, device, electronic equipment and storage medium. Background Art

[0002] The dual-center structure database system is a solution for high availability and data consistency, in which the two databases of the dual-center structure database maintain data consistency through a real-time synchronization mechanism.

[0003] In the prior art, for the dual-center structure database of the Caché database, database A and database B are synchronized in real time. After the DB-A journal obtains the data entered by the user, it sends the data to the DB-A log analysis program. The DB-A log analysis program records the log and synchronizes it to database B according to the policy, so the data is sent to the DB-B data loading program. After the DB-B data loading program saves the data, it sends the data to the DB-B log analysis program. However, the data does not have a mark indicating that the data source is database A, so the DB-B log analysis program will synchronize it to database A according to the policy. Database A will store the data again, resulting in duplicate storage of the data, and more seriously, it may cause the data to form an infinite loop between DB-A and DB-B. Since the data cannot be marked when it is synchronized, each data synchronization will trigger a new round of synchronization operation, and the data will circulate continuously between the two databases, which not only wastes storage resources, but also may cause system performance to decline, and even cause the system to crash. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a dual-center database synchronization method, device, electronic device and storage medium. The technical problem to be solved by the present invention is achieved through the following technical solutions: A first aspect of an embodiment of the present invention provides a dual-center database synchronization method, comprising the following steps: The data loading module of the target database loads the received DML transaction data of the source database itself, and performs data operations on the preset identification table of the target database to generate a target database identification DML statement, and adds the target database identification DML statement to the corresponding source database DML transaction data itself; wherein the preset identification table of the target database is pre-created in the target database; The log file module of the target database stores the acquired DML transaction data of the source database itself as the DML transaction data of the target database, and then sends the DML transaction data of the target database to the log analysis module of the target database; The log analysis module of the target database analyzes the DML transaction data of the target database; When the log analysis module of the target database obtains the target database identification DML statement after parsing, the target database DML transaction data corresponding to the target database identification DML statement is deleted, and the undeleted target database DML transaction data is sent to the data loading module of the source database.

[0005] In one embodiment of the present invention, the method further comprises: The source database responds to executing at least one DML statement of the source database itself, and the log file module obtains corresponding DML transaction data of the source database itself; The log file module of the source database sends the DML transaction data of the source database itself to the log analysis module of the source database; The log analysis module of the source database sends the DML transaction data of the source database itself to the data loading module of the target database.

[0006] In one embodiment of the present invention, the performing of data operation on the preset identification table of the target database includes: adding a piece of data to the preset identification table of the target database.

[0007] In one embodiment of the present invention, the table name of the preset identification table of the target-end database is unique in the target-end database.

[0008] A second aspect of an embodiment of the present invention provides a dual-center database synchronization device, including: The data loading module of the target database is used to load the received DML transaction data of the source database itself, and perform data operations on the preset identification table of the target database to generate a target database identification DML statement, and add the target database identification DML statement to the corresponding source database DML transaction data; wherein the preset identification table of the target database is pre-created in the target database; The log file module of the target database is used to store the acquired DML transaction data of the source database itself as the DML transaction data of the target database, and then send the DML transaction data of the target database to the log analysis module of the target database; A log analysis module of the target database, used to parse the DML transaction data of the target database; The log analysis module of the target database is also used to delete the target database DML transaction data corresponding to the target database identification DML statement when the log analysis module of the target database obtains the target database identification DML statement after parsing, and send the undeleted target database DML transaction data to the data loading module of the source database.

[0009] In one embodiment of the present invention, it also includes: The source database response module is used for obtaining the corresponding source database DML transaction data in the log file module in response to executing at least one source database DML statement. The log file module of the source database is used to send the DML transaction data of the source database itself to the log analysis module of the source database; The log analysis module of the source database is used to send the DML transaction data of the source database itself to the data loading module of the target database.

[0010] In one embodiment of the present invention, the performing of data operation on the preset identification table of the target database includes: adding a piece of data to the preset identification table of the target database.

[0011] In one embodiment of the present invention, the table name of the preset identification table of the target-end database is unique in the target-end database.

[0012] A third aspect of an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a dual-center database synchronization method provided by the first aspect of an embodiment of the present invention is implemented.

[0013] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, a dual-center database synchronization method provided by the first aspect of an embodiment of the present invention is implemented.

[0014] Beneficial effects of the present invention: The present invention performs data operations on a preset identification table through a target-side database to generate a DML statement for the table, and the DML transaction data of the received source-side database belong to the same transaction. The preset identification table is globally unique, and an identification DML statement is generated in the DML transaction data of the source-side database. Therefore, the DML transaction data of the source-side database can be identified in the target-side database through the identification DML statement later, and filtered, thereby avoiding synchronizing the data of the source-side database already in the target-side database to the source-side database again when the target-side database synchronizes data with the source-side database, avoiding repeated storage of data and forming a data synchronization dead loop, which not only saves storage resources but also improves system performance.

[0015] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0016] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 A schematic flow chart of a dual-center database synchronization method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0019] like Figure 1 As shown, a first aspect of an embodiment of the present invention provides a dual-center database synchronization method, comprising the following steps: Step 11, the data loading module of the target database loads the received DML transaction data of the source database itself, and performs data operations on the preset identification table of the target database to generate a target database identification DML statement, and adds the target database identification DML statement to the corresponding source database DML transaction data itself.

[0020] The preset identification table of the target-side database is created in advance in the target-side database.

[0021] Step 12: The log file module of the target database stores the acquired DML transaction data of the source database itself as the DML transaction data of the target database, and then sends the DML transaction data of the target database to the log analysis module of the target database.

[0022] Step 13: The log analysis module of the target database analyzes the DML transaction data of the target database.

[0023] Step 14, when the log analysis module of the target database obtains the target database identification DML statement after parsing, the target database DML transaction data corresponding to the target database identification DML statement is deleted, and the undeleted target database DML transaction data is sent to the data loading module of the source database.

[0024] In this embodiment, the log file module of the target database stores the acquired DML transaction data of the source database itself as the DML transaction data of the target database, so as to synchronize the data of the source database with the target database, and at the same time, the synchronized data is identified by generating a target database identification DML statement. Therefore, at this time, the DML transaction data of the target database includes the DML transaction data of the target database itself (generated by the user operating the target database) and the identified DML transaction data of the source database itself. After deleting the DML transaction data of the target database corresponding to the DML statement of the target database identification, the undeleted DML transaction data of the target database is also the DML transaction data of the target database itself, and the undeleted DML transaction data of the target database is sent to the source database to synchronize the data of the target database with the source database, and complete the data synchronization of both parties. At the same time, after the DML transaction data of the target database itself is sent to the source database, the source database also uses the same method to identify the DML transaction data of the target database itself. When the data of the source end is to be synchronized to the target database next time, the DML transaction data of the target database itself will not be synchronized to the target database again.

[0025] The present invention performs data operations on a preset identification table through a target-side database to generate a DML statement for the table, and the DML transaction data of the received source-side database belong to the same transaction. The preset identification table is globally unique, and an identification DML statement is generated in the DML transaction data of the source-side database. Therefore, the DML transaction data of the source-side database can be identified in the target-side database through the identification DML statement later, and filtered, thereby avoiding synchronizing the data of the source-side database already in the target-side database to the source-side database again when the target-side database synchronizes data with the source-side database, avoiding repeated storage of data and forming a data synchronization dead loop, which not only saves storage resources but also improves system performance.

[0026] Based on the first aspect of the embodiment of the present invention, the second aspect of the embodiment of the present invention further describes in detail a dual-center database synchronization method. The second aspect of the embodiment of the present invention provides a dual-center database synchronization method, which is applied to a dual-center structure database of a Caché database, when the first database is a source database, the second database is a target database, and when the first database is a target database, the second database is a source database, including the following steps: Step 21: In response to the first database executing at least one DML statement of the first database itself, the log file module obtains corresponding DML transaction data of the first database itself.

[0027] In this step, the first database responds to one or more first database DML statements executed by the user, and these statements are written into the first database log file module. In the log file module, transactions are used as units, including multiple groups of DML transaction data, one or more DML statements belong to the same group of DML transaction data, and each group of transaction data has its own transaction identifier.

[0028] Step 22: The log file module of the first database sends the DML transaction data of the first database itself to the log analysis module of the first database.

[0029] Step 23: The log analysis module of the first database sends the DML transaction data of the first database itself to the data loading module of the second database.

[0030] In this step, the log analysis module stores the DML transaction data of the first database itself as the DML transaction data of the first database, and sends the DML transaction data of the first database itself to the data loading module of the second database.

[0031] Step 24, the data loading module of the second database loads the received DML transaction data of the first database itself, and performs data operations on the preset identification table of the second database to generate a second database identification DML statement, and adds the second database identification DML statement to the corresponding DML transaction data of the first database itself.

[0032] The second database preset identification table is pre-created in the second database.

[0033] In this step, when the data loading module of the second database identifies that the executed set of DML transaction data is the DML transaction data of the first database itself (identified by the transaction identifier), a piece of data is added to the preset identification table of the second database at the same time, that is, an operation is performed on the preset identification table of the second database. At this time, a DML statement corresponding to the operation (the second database identification DML statement) can be generated, and the statement is added to the set of DML transaction data. For example, the executed set of DML transaction data is an operation of adding a piece of data to a database table and modifying a piece of data to another database table, and it is identified as the DML transaction data of the first database itself. After the execution is completed or during the execution, the data loading module performs an operation on the preset identification table of the second database, adds a piece of data to the preset identification table of the second database, and uses the DML statement corresponding to the operation (the second database identification DML statement) as a DML statement of the group, so that when the second database identification DML statement is subsequently identified in the group, it can be determined that the set of DML data is the DML transaction data of the first database itself.

[0034] The preset identification table of the second database is a table that is created in advance and has a globally unique identification that is different from the identifications of all tables in the second database.

[0035] Step 25: The log file module of the second database stores the acquired DML transaction data of the first database itself as the DML transaction data of the second database, and then sends the DML transaction data of the second database to the log analysis module of the second database.

[0036] In this step, the log file module of the second database obtains the DML transaction data of the first database itself with the DML statement of the second database identifier added, and stores the DML transaction of the first database itself with the added statement. The stored data are all DML transaction data of the second database, with the transaction identifier of the second database but not the transaction identifier of the first database, and the grouping of each group of transaction data will not change.

[0037] After the data is stored, the DML transaction data of the first database itself has been synchronized to the second database, and the data of the second database needs to be synchronized to the first database later.

[0038] Step 26: The log analysis module of the second database analyzes the DML transaction data of the second database.

[0039] Step 27, when the log analysis module of the second database obtains the second database identification DML statement after parsing, the second database DML transaction data corresponding to the second database identification DML statement is deleted, and the undeleted second database DML transaction data is sent to the data loading module of the first database.

[0040] In this step, the log analysis module of the second database identifies that a certain group of DML transaction data in the DML transaction data of the second database has a second database identification DML statement, that is, it identifies that there is a DML statement for operating a preset identification table of the second database in the group of DML transaction data, then the group of DML transaction data comes from the first database and is the DML transaction data of the first database itself. At this time, the group of DML transaction data where the second database identification statement is located is deleted, that is, the DML transaction data of the first database itself is deleted, and the remaining undeleted data is the data without identification. These undeleted data are the data of the second database itself, that is, the data that needs to be synchronized to the first database. These undeleted data are sent to the data loading module of the first database, thereby completing the data synchronization of the second database to the first database, and the DML transaction data of the first database itself will not be synchronized to the first database again, thereby ensuring that the data of the first database will not be synchronized to the first database.

[0041] Here, the undeleted DML transaction data of the second database is also the DML transaction data of the second database itself. The source of the DML transaction data of the second database itself in the log analysis module of the second database is: the second database responds to the execution of at least one DML statement of the second database itself, and the log file module obtains the corresponding DML transaction data of the second database itself; the log file module of the second database sends the DML transaction data of the second database itself to the log analysis module of the second database. The log analysis module of the second database sends the DML transaction data of the second database itself to the data loading module of the first database, that is, sends the undeleted DML transaction data of the second database to the data loading module of the first database, thereby completing the data synchronization of the second database to the first database.

[0042] Step 28, when the first database needs to synchronize data with the second database again, the data loading module of the first database loads the received DML transaction data of the second database itself, and performs data operations on the preset identification table of the first database to generate a first database identification DML statement, and adds the first database identification DML statement to the corresponding second database's own DML transaction data; wherein the preset identification table of the first database is pre-created in the second database.

[0043] Step 29: The log file module of the first database stores the acquired DML transaction data of the second database itself as the DML transaction data of the first database, and then sends the DML transaction data of the first database to the log analysis module of the first database.

[0044] Step 30: The log analysis module of the first database analyzes the DML transaction data of the first database.

[0045] Step 31, when the log analysis module of the first database obtains the first database identification DML statement after parsing, the first database DML transaction data corresponding to the first database identification DML statement is deleted, and the undeleted first database DML transaction data is sent to the data loading module of the second database. Here, the second database's own DML transaction data is deleted, and the undeleted data is the first database's own DML transaction data that was not synchronized last time (different from the first database's own DML transaction data in step 21), and the undeleted data is synchronized to the second database, thereby ensuring that the second database's own DML transaction data will not be synchronized to the second database, that is, the data of the second database will not be synchronized to the second database.

[0046] In this embodiment, during the process of continuous synchronization between two databases, after the data to be synchronized is synchronized to the other party, the other party will not synchronize the data back, thereby ensuring that the data will not be synchronized repeatedly.

[0047] A third aspect of an embodiment of the present invention provides a dual-center database synchronization device, including: The data loading module of the target database is used to load the received DML transaction data of the source database itself, and perform data operations on the preset identification table of the target database to generate the target database identification DML statement, and add the target database identification DML statement to the corresponding source database DML transaction data; wherein the preset identification table of the target database is pre-created in the target database; The log file module of the target database is used to store the acquired DML transaction data of the source database itself as the DML transaction data of the target database, and then send the DML transaction data of the target database to the log analysis module of the target database; The log analysis module of the target database is used to analyze the DML transaction data of the target database; The log analysis module of the target database is also used to delete the target database DML transaction data corresponding to the target database identification DML statement when the log analysis module of the target database obtains the target database identification DML statement after parsing, and send the undeleted target database DML transaction data to the data loading module of the source database.

[0048] In one embodiment of the present invention, it also includes: The source database response module is used for obtaining the corresponding source database DML transaction data in the log file module in response to executing at least one source database DML statement. The log file module of the source database is used to send the DML transaction data of the source database itself to the log analysis module of the source database; The log analysis module of the source database is used to send the DML transaction data of the source database itself to the data loading module of the target database.

[0049] In one embodiment of the present invention, performing data operation on the preset identification table of the target database includes: adding a piece of data to the preset identification table of the target database.

[0050] In one embodiment of the present invention, the table name of the preset identification table of the target-end database is unique in the target-end database.

[0051] A fourth aspect of an embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a dual-center database synchronization method provided in the above-mentioned embodiment of the present invention is implemented.

[0052] A fifth aspect of an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of a dual-center database synchronization method provided by the above-mentioned embodiment of the present invention are implemented.

[0053] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0054] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware systems.

[0055] The method provided in the embodiment of the present invention can be applied to electronic devices. Specifically, the electronic device can be: a desktop computer, a portable computer, an intelligent mobile terminal, a server, etc. This is not limited here, and any electronic device that can implement the present invention belongs to the protection scope of the present invention.

[0056] As for the device / electronic device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0057] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0058] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0059] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0060] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A dual-center database synchronization method, characterized in that: The following steps are involved: The data loading module of the target database loads the received DML transaction data of the source database itself, and performs data operations on the preset identification table of the target database to generate a target database identification DML statement, and adds the target database identification DML statement to the corresponding source database DML transaction data itself; wherein the preset identification table of the target database is pre-created in the target database; The log file module of the target database stores the acquired DML transaction data of the source database itself as the DML transaction data of the target database, and then sends the DML transaction data of the target database to the log analysis module of the target database; The log analysis module of the target database analyzes the DML transaction data of the target database; When the log analysis module of the target database obtains the target database identification DML statement after parsing, the target database DML transaction data corresponding to the target database identification DML statement is deleted, and the undeleted target database DML transaction data is sent to the data loading module of the source database.

2. The method according to claim 1, characterized in that The method further comprises: The source database responds to executing at least one DML statement of the source database itself, and the log file module obtains corresponding DML transaction data of the source database itself; The log file module of the source database sends the DML transaction data of the source database itself to the log analysis module of the source database; The log analysis module of the source database sends the DML transaction data of the source database itself to the data loading module of the target database.

3. The method according to claim 2, characterized in that The data operation on the preset identification table of the target end database includes: adding a piece of data to the preset identification table of the target end database.

4. The method according to claim 2, characterized in that The table name of the preset identification table of the target-end database is unique in the target-end database.

5. A dual-center database synchronization device, characterized in that: include: The data loading module of the target database is used to load the received DML transaction data of the source database itself, and perform data operations on the preset identification table of the target database to generate a target database identification DML statement, and add the target database identification DML statement to the corresponding source database DML transaction data; wherein the preset identification table of the target database is pre-created in the target database; The log file module of the target database is used to store the acquired DML transaction data of the source database itself as the DML transaction data of the target database, and then send the DML transaction data of the target database to the log analysis module of the target database; A log analysis module of the target database, used to parse the DML transaction data of the target database; The log analysis module of the target database is also used to delete the target database DML transaction data corresponding to the target database identification DML statement when the log analysis module of the target database obtains the target database identification DML statement after parsing, and send the undeleted target database DML transaction data to the data loading module of the source database.

6. The device according to claim 5, characterized in that Also includes: The source database response module is used for obtaining the corresponding source database DML transaction data in the log file module in response to executing at least one source database DML statement. The log file module of the source database is used to send the DML transaction data of the source database itself to the log analysis module of the source database; The log analysis module of the source database is used to send the DML transaction data of the source database itself to the data loading module of the target database.

7. The device according to claim 6, characterized in that The data operation on the preset identification table of the target end database includes: adding a piece of data to the preset identification table of the target end database.

8. The device according to claim 5, characterized in that The table name of the preset identification table of the target-end database is unique in the target-end database.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the dual-center database synchronization method as described in any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the dual-center database synchronization method according to any one of claims 1 to 4 is implemented.

Images