Database synchronization method, database synchronization device, and readable storage medium

Abstract

The application relates to the technical field of information synchronization, in particular to a database synchronization method, a database synchronization device and a readable storage medium. The method is applied to a database synchronization system, the database synchronization system comprises a log analysis module, a synchronization judgment module and a synchronization execution module, and the method comprises the following steps: when the log analysis module monitors that a change record is updated, change information is acquired according to the change record, the change information is encapsulated into a change information table, and then the change information table is sent to the synchronization judgment module; after the synchronization judgment module receives the change information table, the change source of the change record is determined according to the change information table; when the change source is a business change operation, the synchronization execution module generates a synchronization execution statement carrying a synchronization identifier according to the change information table, and sends the synchronization execution statement to a target database for execution. The bidirectional synchronization operation between databases is realized, and the occurrence of a synchronization dead loop can be prevented.

Description

Technical Field

[0001] This invention relates to the field of information synchronization technology, and in particular to a database synchronization method, a database synchronization device, and a readable storage medium. Background Technology

[0002] A common method for synchronizing data between multiple databases is by adding a special table. This involves creating a dedicated synchronization table in each database to record change information. Each time a data change operation (such as adding, modifying, or deleting) occurs, a corresponding record is inserted into the synchronization table, recording relevant information such as table name, operation type, data content, and timestamp. Each database periodically reads the change records from the synchronization table. Based on these records, the changed data is synchronized to the target database according to synchronization rules and mapping relationships.

[0003] Because database synchronization is achieved by adding special tables, it's impossible to accurately determine whether data changes in the database were caused by business operations or synchronization operations. Therefore, during bidirectional database synchronization, the same data may be repeatedly synchronized between databases, leading to a synchronization deadlock. Thus, commonly used bidirectional database synchronization methods inevitably suffer from the problem of entering a synchronization deadlock.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this invention is to provide a database synchronization method that addresses the problem that commonly used bidirectional database synchronization methods cannot prevent the database from entering a synchronization dead loop.

[0006] To achieve the above objectives, this invention provides a database synchronization method applied to a database synchronization system. The database synchronization system includes a log parsing module, a synchronization judgment module, and a synchronization execution module. The database synchronization method includes:

[0007] When the log parsing module detects an update of a change record, it obtains the change information based on the change record, encapsulates the change information into a change information table, and sends it to the synchronization judgment module.

[0008] After the synchronization judgment module receives the change information table, it determines the source of the change record based on the change information table.

[0009] When the source of the change is a business change operation, the synchronization execution module generates a synchronization execution statement carrying a synchronization identifier based on the change information table, and sends the synchronization execution statement to the target database for execution to achieve database synchronization operation.

[0010] Optionally, after the step of determining the source of the change record based on the change information table after the synchronization judgment module receives the change information table, the method further includes:

[0011] When the source of the change is a database synchronization operation, the synchronization execution module performs a change record deletion operation.

[0012] Optionally, the step of determining the source of the change record based on the change information table after the synchronization judgment module receives the change information table includes:

[0013] The change statement is retrieved from the change information table and analyzed to determine whether the change statement carries a synchronization identifier.

[0014] If the change statement carries the synchronization identifier, the source of the change is determined to be the database synchronization operation;

[0015] If the change statement does not carry the synchronization identifier, the source of the change is determined to be the business change operation.

[0016] Optionally, when the source of the change is a business change operation, the step of the synchronization execution module generating a synchronization execution statement carrying a synchronization identifier based on the change information table includes:

[0017] Retrieve the change information from the change information table;

[0018] Based on the change information, determine the data to be synchronized, and determine the mapping relationship between the source database and the target database;

[0019] Based on the data to be synchronized and the mapping relationship, the change information of the source database is converted into the change information of the target database;

[0020] Add a synchronization identifier to the change information of the target database, and generate the synchronization execution statement carrying the synchronization identifier.

[0021] Optionally, after the step of generating a synchronization execution statement carrying a synchronization identifier based on the change information table and sending the synchronization execution statement to the target database for execution to achieve database synchronization when the change source is a business change operation, the method further includes:

[0022] When the target database receives the synchronization execution statement carrying the synchronization identifier, it executes the synchronization execution statement and updates the target database with the change information.

[0023] The change information of the target database is applied to the target database to achieve database synchronization.

[0024] Optionally, the change records include table change records and field change records. The step of obtaining change information based on the change records when the log parsing module detects an update, encapsulating the change information into a change information table, and then sending it to the synchronization judgment module includes:

[0025] Based on the table change records, generate a table structure change information table;

[0026] Based on the field change records, generate a field change information table.

[0027] Optionally, after the step of determining the source of the change record based on the change information table after the synchronization judgment module receives the change information table, the method further includes:

[0028] Based on the source of the change, determine whether to perform the database synchronization operation;

[0029] If the source of changes to both the table change record and the field change record is a database synchronization operation, then the database synchronization operation will not be performed, and the table change record and the field change record will be deleted.

[0030] If the source of the change in the table record and / or the field record is a business change operation, then the database synchronization operation will be executed.

[0031] Optionally, when the source of the change is a business change operation, the step of the synchronization execution module generating a synchronization execution statement carrying a synchronization identifier based on the change information table includes:

[0032] Based on the table structure change information table and the field change information table, determine the table structure mapping relationship and field mapping relationship between the source database and the target database;

[0033] Based on the table structure mapping relationship and the table structure change information table, generate a table synchronization execution statement carrying a synchronization identifier;

[0034] Based on the field mapping relationship and the field change information table, generate a field synchronization execution statement carrying a synchronization identifier.

[0035] In addition, to achieve the above objectives, the present invention also provides a database synchronization device, which includes a memory, a processor, and a database synchronization program stored in the memory and executable on the processor. When the database synchronization program is executed by the processor, it implements the steps of the database synchronization method as described above.

[0036] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a database synchronization program, which, when executed by a processor, implements the steps of the database synchronization method as described above.

[0037] This invention provides a database synchronization method, a database synchronization device, and a readable storage medium. When the log parsing module detects an update to a change record, it obtains the change information based on the change record, encapsulates the change information into a change information table, and sends it to the synchronization judgment module. Upon receiving the change information table, the synchronization judgment module determines the source of the change record based on the change information table. When the change source is a business change operation, the synchronization execution module generates a synchronization execution statement carrying a synchronization identifier based on the change information table and sends the synchronization execution statement to the target database for execution. This allows for bidirectional synchronization between databases. By attaching a synchronization identifier to the synchronization execution statement, the log parsing module monitors and judges the database again after the statement is executed. If the target database records the synchronization operation in its log again, the synchronization judgment module can determine that the change record has already undergone a synchronization operation due to the presence of the synchronization identifier, thus avoiding further database synchronization operations and preventing synchronization loops. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the hardware operating environment of the database synchronization device involved in the embodiments of the present invention;

[0039] Figure 2 This is a flowchart illustrating the first embodiment of the database synchronization method of the present invention;

[0040] Figure 3 This is a flowchart illustrating a second embodiment of the database synchronization method of the present invention;

[0041] Figure 4 This is a flowchart illustrating the third embodiment of the database synchronization method of the present invention;

[0042] Figure 5 This is a flowchart illustrating the fourth embodiment of the database synchronization method of the present invention;

[0043] Figure 6 This is a flowchart illustrating the fifth embodiment of the database synchronization method of the present invention;

[0044] Figure 7 This is a schematic diagram of the system architecture of the sixth embodiment of the database synchronization method of the present invention.

[0045] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0046] This application's database synchronization method, when the log parsing module detects an update to a change record, retrieves the change information based on the change record, encapsulates this information into a change information table, and sends it to the synchronization judgment module. Upon receiving the change information table, the synchronization judgment module determines the source of the change record based on the change information table. When the change source is a business change operation, the synchronization execution module generates a synchronization execution statement carrying a synchronization identifier based on the change information table and sends the execution statement to the target database for execution. This achieves bidirectional synchronization between databases while preventing synchronization loops.

[0047] To better understand the above technical solutions, exemplary embodiments of this disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of this disclosure to those skilled in the art.

[0048] As one implementation scheme, Figure 1 This is a schematic diagram of the hardware operating environment of the database synchronization device involved in the embodiments of the present invention.

[0049] like Figure 1 As shown, the database synchronization device may include: a processor 101, such as a central processing unit (CPU), a memory 102, and a communication bus 103. The memory 102 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. Optionally, the memory 102 may also be a storage device independent of the aforementioned processor 101. The communication bus 103 is used to enable communication between these components.

[0050] Those skilled in the art will understand that Figure 1The structure shown does not constitute a limitation on the database synchronization device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0051] like Figure 1 As shown, the memory 102, which is a computer-readable storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, and a database synchronization program.

[0052] exist Figure 1 In the database synchronization device shown, processor 101 and memory 102 can be installed in the database synchronization device. The database synchronization device calls the database synchronization program stored in memory 102 through processor 101 and performs the following operations:

[0053] When the log parsing module detects an update of a change record, it obtains the change information based on the change record, encapsulates the change information into a change information table, and sends it to the synchronization judgment module.

[0054] After the synchronization judgment module receives the change information table, it determines the source of the change record based on the change information table.

[0055] When the source of the change is a business change operation, the synchronization execution module generates a synchronization execution statement carrying a synchronization identifier based on the change information table, and sends the synchronization execution statement to the target database for execution to achieve database synchronization operation.

[0056] In one embodiment, the processor 101 can be used to invoke a database synchronization program stored in the memory 102 and perform the following operations:

[0057] When the source of the change is a database synchronization operation, the synchronization execution module performs a change record deletion operation.

[0058] In one embodiment, the processor 101 can be used to invoke a database synchronization program stored in the memory 102 and perform the following operations:

[0059] The change statement is retrieved from the change information table and analyzed to determine whether the change statement carries a synchronization identifier.

[0060] If the change statement carries the synchronization identifier, the source of the change is determined to be the database synchronization operation;

[0061] If the change statement does not carry the synchronization identifier, the source of the change is determined to be the business change operation.

[0062] In one embodiment, the processor 101 can be used to invoke a database synchronization program stored in the memory 102 and perform the following operations:

[0063] Retrieve the change information from the change information table;

[0064] Based on the change information, determine the data to be synchronized, and determine the mapping relationship between the source database and the target database;

[0065] Based on the data to be synchronized and the mapping relationship, the change information of the source database is converted into the change information of the target database;

[0066] Add a synchronization identifier to the change information of the target database, and generate the synchronization execution statement carrying the synchronization identifier.

[0067] In one embodiment, the processor 101 can be used to invoke a database synchronization program stored in the memory 102 and perform the following operations:

[0068] When the target database receives the synchronization execution statement carrying the synchronization identifier, it executes the synchronization execution statement and updates the target database with the change information.

[0069] The change information of the target database is applied to the target database to achieve database synchronization.

[0070] In one embodiment, the processor 101 can be used to invoke a database synchronization program stored in the memory 102 and perform the following operations:

[0071] Based on the table change records, generate a table structure change information table;

[0072] Based on the field change records, generate a field change information table.

[0073] In one embodiment, the processor 101 can be used to invoke a database synchronization program stored in the memory 102 and perform the following operations:

[0074] Based on the source of the change, determine whether to perform the database synchronization operation;

[0075] If the source of changes to both the table change record and the field change record is a database synchronization operation, then the database synchronization operation will not be performed, and the table change record and the field change record will be deleted.

[0076] If the source of the change in the table record and / or the field record is a business change operation, then the database synchronization operation will be executed.

[0077] In one embodiment, the processor 101 can be used to invoke a database synchronization program stored in the memory 102 and perform the following operations:

[0078] Based on the table structure change information table and the field change information table, determine the table structure mapping relationship and field mapping relationship between the source database and the target database;

[0079] Based on the table structure mapping relationship and the table structure change information table, generate a table synchronization execution statement carrying a synchronization identifier;

[0080] Based on the field mapping relationship and the field change information table, generate a field synchronization execution statement carrying a synchronization identifier.

[0081] Based on the hardware architecture of the access control system described above, an embodiment of the authentication method of the present invention is proposed.

[0082] Reference Figure 2 In the first embodiment, the database synchronization method includes the following steps:

[0083] Step S100: When the log parsing module detects an update of a change record, it obtains the change information based on the change record, encapsulates the change information into a change information table, and sends it to the synchronization judgment module.

[0084] In this embodiment, the log parsing module of the database synchronization system listens to the database logs to capture change records generated by change operations. It can be understood that listening to the database logs means that the log parsing module reads and parses the database logs in real time.

[0085] Optionally, for databases that automatically record all data change operations and provide API or query interfaces to query and retrieve change records, the log parsing module can retrieve the database change records through the API or query interface.

[0086] It should be noted that database modification operations include, but are not limited to, adding new data to a database table, deleting specified data from a database table, modifying the values ​​of existing data in a database table, adding, deleting, and modifying table structures, and changing the names of database objects.

[0087] In this embodiment, the log parsing module parses the captured change records to obtain database change information.

[0088] Change information includes, but is not limited to, change statements, change table names, change operations, change transaction IDs, data before the change, and data after the change.

[0089] Specifically, the log parsing module analyzes the database log format to determine the log structure; based on the log structure, it determines the location of change records in the log; once the change record is located, it listens for the change record; when a change record is updated, it reads the change record to the database synchronization system, parses the change record, and extracts the change information from the change record.

[0090] According to the change information table format, the change information is encapsulated into a change information table, and the change information table is transmitted to the synchronization judgment module of the database synchronization system.

[0091] Step S200: After the synchronization judgment module receives the change information table, it determines the source of the change record based on the change information table.

[0092] In this embodiment, the synchronization judgment module of the database synchronization system parses the received change information table to determine the source of the change record, that is, to determine whether the change record was generated by the database synchronization operation or by the business change operation.

[0093] Optionally, a synchronization identifier can be used to determine the source of the change record. Specifically, each time the database synchronization system performs a database synchronization operation on the target database, a synchronization identifier is added to the change record of the target database to indicate that the change record was generated by the database synchronization operation. This allows the synchronization judgment module to determine the source of the change record, i.e., whether it was generated by the database synchronization operation, by verifying the synchronization identifier.

[0094] In one specific implementation, the source of the change record can be determined by using a synchronization operation log. Specifically, after each database synchronization operation is performed on the target database by the database synchronization system, synchronization operation information is generated and saved to the synchronization operation log. The synchronization judgment module can then query the synchronization operation log to determine whether it contains synchronization operation information corresponding to the change information in the change information table, thereby determining the source of the change record, i.e., whether it was generated by the database synchronization operation.

[0095] In another specific implementation, the source of the change record can be determined by using a synchronization operation timestamp. Specifically, each time the database synchronization system performs a database synchronization operation on the target database, a timestamp is added to the change record of the target database to indicate that the change record was generated within the time range of the database synchronization operation. This allows the synchronization judgment module to determine the source of the change record, i.e., whether it was generated by the database synchronization operation, by comparing the timestamp of the change record with the time range of the database synchronization operation.

[0096] Step S300: When the source of the change is a business change operation, the synchronization execution module generates a synchronization execution statement carrying a synchronization identifier according to the change information table, and sends the synchronization execution statement to the target database for execution to realize the database synchronization operation.

[0097] In this embodiment, if the database synchronization system determines through the synchronization judgment module that the source of the change record is a business change operation, the synchronization execution module determines the mapping relationship between the source database and the target database based on the change information table generated by the log parsing module; based on this mapping relationship, a synchronization execution statement carrying a synchronization identifier is generated.

[0098] It should be noted that each database can be both a source database and a target database. However, when a database is a source database, it cannot also be a target database; conversely, when a database is a target database, it cannot also be a source database.

[0099] Optionally, the synchronization execution module of the database synchronization system can generate synchronization execution statements carrying synchronization identifiers by concatenating SQL (Structured Query Language) statements based on the mapping relationship. Specifically, it can generate INSERT, DELETE, and / or UPDATE statements based on the change information in the change information table, add synchronization identifiers to the generated statements, and thus generate synchronization execution statements carrying synchronization identifiers.

[0100] For example, suppose there are two databases, source database A and target database B, and a database synchronization operation needs to be performed between them.

[0101] The database synchronization system monitors the logs of database A through a log parsing module, capturing change records generated by change operations in the source database A, and parsing these change records to obtain the change information. Assume one change record is captured:

[0102] Change statement: INSERT INTO customers(name, age, email) VALUES('John', 25, 'john@example.com')

[0103] Change table name: customers

[0104] Change operation: INSERT

[0105] Change transaction ID: 12345

[0106] Data before the change: None

[0107] The revised data is: ('John', 25, 'john@example.com')

[0108] Based on the change information table format, the change information is encapsulated into a change information table and sent to the synchronization judgment module.

[0109] After receiving the change information table, the synchronization judgment module of the database synchronization system analyzes the change statements in the change information table and determines that the change statement does not carry a synchronization identifier, and the source of the change record is a business change operation.

[0110] The database synchronization system's synchronization execution module generates synchronization execution statements carrying synchronization identifiers based on the change information table:

[0111] INSERT INTO customers(name,age,email)VALUES('John',25,'john@example.com')--Sync

[0112] The synchronous execution statement carries the synchronization flag "--Sync", indicating that the change record was generated by a database synchronization operation.

[0113] The statement is sent to target database B for execution. Target database B executes the synchronous statement and adds a new record to the customers table.

[0114] Through the above steps, the data changes in the source database A are synchronized to the target database B.

[0115] In the technical solution provided in this embodiment, when the log parsing module detects an update of a change record, it obtains the change information based on the change record, encapsulates the change information into a change information table, and sends it to the synchronization judgment module. Upon receiving the change information table, the synchronization judgment module determines the source of the change record based on the change information table. When the source of the change is a business change operation, the synchronization execution module generates a synchronization execution statement carrying a synchronization identifier based on the change information table and sends the synchronization execution statement to the target database for execution, thereby achieving database synchronization. Because the synchronization identifier is appended to the synchronization execution statement when implementing bidirectional synchronization between databases, the change record in the target database carries the synchronization identifier after the target database executes the statement. This allows the synchronization judgment module to determine that the change record has already undergone a synchronization operation, thus preventing the database synchronization system from performing another database synchronization operation on the change operation and preventing synchronization deadlocks.

[0116] Furthermore, based on the above embodiments, in one embodiment, after the step of determining the source of change of the change record according to the change information table after the synchronization judgment module receives the change information table, it further includes:

[0117] Step S400: When the source of the change is a database synchronization operation, the synchronization execution module performs a change record deletion operation.

[0118] Optionally, the synchronization execution module of the database synchronization system can generate a record deletion statement and send it to the source database for execution to remove the changed record from the source database's log. For example, in SQL, the DELETE statement can be used to delete a changed record.

[0119] It should be noted that during bidirectional synchronization, if a deletion operation is not performed, a synchronization loop may occur between databases, causing the same operation to be executed repeatedly in different databases, resulting in data inconsistency and conflicts.

[0120] In the technical solution provided in this embodiment, when the source of the change is a database synchronization operation, a change record deletion operation is performed to delete the change record originating from the database synchronization operation. This ensures that each change record is executed only once, thereby preventing the same database synchronization operation from being repeatedly executed during bidirectional synchronization between databases.

[0121] Furthermore, based on the above embodiments, in one embodiment, the step of determining the source of change of the change record according to the change information table after the synchronization judgment module receives the change information table includes:

[0122] Step S230: Retrieve the change statement from the change information table and analyze the change statement to determine whether the change statement carries a synchronization identifier;

[0123] Step S240: If the change statement carries the synchronization identifier, determine that the change source is the database synchronization operation;

[0124] Step S250: If the change statement does not carry the synchronization identifier, determine that the change source is the business change operation.

[0125] In this embodiment, since the synchronization execution statement generated by the synchronization execution module of the database synchronization system carries a synchronization identifier, when the target database executes the synchronization execution statement and achieves database synchronization, the change statement information contained in the resulting change record will carry this synchronization identifier. Therefore, the synchronization judgment module can determine whether the change statement carries a synchronization identifier by analyzing the change statement in the change information table, and thus determine whether the change source of the change record is a database synchronization operation.

[0126] Optionally, the database synchronization system first presets a synchronization identifier. When determining the source of a change, it can compare the information to determine whether the change statement contains a string that matches the synchronization identifier. If so, the change statement contains the synchronization identifier, and the source of the change is determined to be a database synchronization operation. If not, the change statement does not contain the synchronization identifier, and the source of the change is determined to be a business change operation.

[0127] It should be noted that the preset synchronization flag should be distinguished from the SQL statements in the database to avoid execution conflicts when the database executes SQL statements.

[0128] In the technical solution provided in this embodiment, since the source of the change is determined to be a database synchronization operation or a business change operation by whether the change statement carries a synchronization identifier, it can be ensured that the synchronization judgment module can accurately determine the source of the change record, thereby improving the accuracy and reliability of the synchronization judgment and further avoiding erroneous synchronization or deletion operations.

[0129] Reference Figure 3 Based on the above embodiments, in the second embodiment, when the source of the change is a business change operation, the step of the synchronous execution module generating a synchronous execution statement carrying a synchronization identifier according to the change information table includes:

[0130] Step S310: Obtain the change information from the change information table;

[0131] Step S320: Based on the change information, determine the data to be synchronized, and determine the mapping relationship between the source database and the target database;

[0132] Step S330: Based on the data to be synchronized and the mapping relationship, convert the change information of the source database into the change information of the target database;

[0133] Step S340: Add a synchronization identifier to the change information of the target database and generate the synchronization execution statement carrying the synchronization identifier.

[0134] In this embodiment, the change information table records the data that has been changed in the source database, including the table name, operation type, and the changed data. Therefore, the data to be synchronized can be determined based on the table name and operation type in the change information table.

[0135] Optionally, the mapping relationship between the source database and the target database can be determined by comparing the data before and after the change in the change information table.

[0136] Based on the change information in the change information table, the mapping relationship between the source database and the target database is determined. The mapping relationship includes table name mapping relationship, field mapping relationship, and operation type mapping relationship.

[0137] In this embodiment, the table name, field information, and / or operation type data to be synchronized in the change information table are converted according to the mapping relationship. If the field information is data to be synchronized, field mapping is performed; if the table name is data to be synchronized, table name mapping is performed.

[0138] Since the source database and the target database may have different data structures and formats, it is necessary to generate synchronization execution statements based on the mapping relationship and change information so that the synchronization operation can be performed correctly in the target database.

[0139] In the technical solution provided in this embodiment, by determining the data to be synchronized and the mapping relationship between the source database and the target database based on the change information, the scope of database synchronization is clarified, further ensuring that the database synchronization system only synchronizes the data that needs to be synchronized. After determining the data to be synchronized and the mapping relationship, the change information of the source database is converted into the change information of the target database. Through this conversion, the change information can be correctly converted and synchronized during the database synchronization process, further ensuring the accuracy of database synchronization. By adding a synchronization identifier to the change information of the target database to indicate that this change information was generated by the database synchronization system, the change records of the target database carry the synchronization identifier after the target database performs the synchronization operation, thereby preventing the database synchronization system from repeatedly performing synchronization operations on this change.

[0140] Furthermore, based on the above embodiments, in one embodiment, the change information table may include a table structure change information table and a field change information table. Optionally, table synchronization execution statements and / or field synchronization execution statements carrying synchronization identifiers can be generated based on the table structure change information table and / or the field change information table.

[0141] In this embodiment, the table structure change information table and / or field change information table are first parsed by the synchronization execution module of the database synchronization system; the table structure change information table before the table structure change and the table structure change information table after the table structure change are obtained, and / or the field change information table before the field change and the field change information table after the field change are obtained.

[0142] Then, by comparing the data before and after the table structure change, the table structure mapping relationship is determined, and / or by comparing the data before and after the field change, the field mapping relationship is determined. The table structure mapping relationship refers to the mapping between the source and target databases; the field mapping relationship refers to the mapping between the fields in the source and target databases.

[0143] Finally, based on the table structure mapping relationship and the table structure change information table, a table synchronization execution statement carrying a synchronization identifier can be generated; and / or based on the field mapping relationship and the field change information table, a field synchronization execution statement carrying a synchronization identifier can be generated.

[0144] Optionally, since the table and field names in the database may not be exactly the same before and after the change operation, or there may be differences in naming conventions, the mapping relationship can be determined based on the correspondence between table and field names. Since the data types and lengths of the database fields may not be exactly the same before and after the change operation, the mapping relationship can be determined based on data type compatibility and length compatibility. For example, if a VARCHAR(100) type field in the database is changed to a VARCHAR(255) type field before and after the change operation, a mapping relationship needs to be established from the VARCHAR(100) type field to the VARCHAR(255) type field.

[0145] Optionally, methods for generating synchronous execution statements include, but are not limited to, using command-line tools, libraries and frameworks specific to programming languages. For example, using command-line tools to generate synchronous execution statements; in MySQL, using the `mysqldump` command to export the database table structure and fields and save it as an SQL file. Alternatively, using libraries and frameworks specific to programming languages ​​to generate synchronous execution statements; for example, using the Hibernate framework in Java and using `SchemaExport` to generate synchronous execution statements for database tables.

[0146] In the technical solution provided in this embodiment, since the synchronization execution module determines the mapping relationship between table structure and fields, and generates table synchronization execution statements and field synchronization execution statements carrying synchronization identifiers based on these mapping relationships and change information tables, the database synchronization operation can correctly synchronize the table structure and fields of the source database to the target database. Therefore, the synchronization of table structure and fields can be further improved when performing synchronization operations between databases.

[0147] Reference Figure 4 Based on the above embodiments, in the third embodiment, after the step of generating a synchronization execution statement carrying a synchronization identifier according to the change information table and sending the synchronization execution statement to the target database for execution to realize the database synchronization operation when the change source is a business change operation, the method further includes:

[0148] Step S500: When the target database receives the synchronization execution statement carrying the synchronization identifier, it executes the synchronization execution statement to obtain the change information of the target database;

[0149] Step S600: Apply the change information of the target database to the target database to realize the database synchronization operation.

[0150] In this embodiment, after the target database receives a synchronous execution statement carrying a synchronization identifier, it parses the synchronous execution statement to obtain change information such as table name, field information, and operation type; based on the table name and field information, it verifies whether the corresponding table and field exist in the target database; if not, it performs corresponding table structure creation and / or modification operations to ensure that the change information is correctly applied; based on the operation type, it performs the corresponding database operation to apply the change information to the target database.

[0151] The target database executes corresponding insert, update, and delete operations based on the synchronous execution statements, applying the changes to the target database.

[0152] After the target database completes the database synchronization operation, update a change record carrying a synchronization identifier to prevent the database synchronization from entering an infinite loop.

[0153] In the technical solution provided in this embodiment, after the target database receives the synchronization execution statement carrying the synchronization identifier, it obtains the change information of the target database by executing the synchronization execution statement, and applies the change information to synchronize the change operation in the source database to the target database, thereby realizing database synchronization.

[0154] Reference Figure 5Based on the above embodiments, in the fourth embodiment, the change record includes table change records and field change records. The step of obtaining change information based on the change record when the log parsing module detects an update, encapsulating the change information into a change information table, and then sending it to the synchronization judgment module includes:

[0155] Step S710: Generate a table structure change information table based on the table change records;

[0156] Step S720: Generate a field change information table based on the field change records.

[0157] In this embodiment, when the log parsing module detects an update of a table change record, it obtains the table change information in the table change record, which includes, but is not limited to, table name, operation type, etc.; and encapsulates the table change information such as table name and operation type into a table structure change information table according to the table change information table format.

[0158] When the log parsing module detects an update of a field change record, it retrieves the field change information from the field change record, including field value, field type, operation type, etc.; and encapsulates the field change information such as field value, field type, and operation type into a field structure change information table according to the table change information table format.

[0159] In the technical solution provided in this embodiment, the change records of tables and fields are converted into corresponding table change information and field change information, and then encapsulated into corresponding table structure change information tables and field change information tables. This allows the change information tables to contain all the change information of tables and fields, which can then be called by the synchronization judgment module and the synchronization execution module.

[0160] Reference Figure 6 Based on the above embodiments, in the fifth embodiment, after the step of determining the source of change of the change record according to the change information table after the synchronization judgment module receives the change information table, it further includes:

[0161] Step S730: Determine whether to perform the database synchronization operation based on the source of the change;

[0162] Step S740: If the source of change for both the table change record and the field change record is a database synchronization operation, determine that the database synchronization operation will not be performed, and delete the table change record and the field change record;

[0163] Step S750: If the source of the change in the table change record and / or the field change record is a business change operation, determine to execute the database synchronization operation.

[0164] In this embodiment, the source of the table change record and field change record is determined based on the information in the table structure change information table and the field change information table, so as to determine whether to perform a database synchronization operation.

[0165] When both the table change record and the field change record originate from a database synchronization operation, it means that the table change record and the field change record were generated by a previous database synchronization operation, and therefore it is not necessary to perform a database synchronization operation again.

[0166] After determining that database synchronization will not be performed, table change records and field change records can be deleted from the change information table to prevent the database synchronization operation from entering an infinite loop.

[0167] In this embodiment, the database synchronization operation includes table structure synchronization and field synchronization. When the source of the table change record is a business change operation, and the source of the field change record is a database synchronization operation, only the table structure synchronization operation is performed; when the source of the field change record is a business change operation, and the source of the table change record is a database synchronization operation, only the field synchronization operation is performed; when the source of both the table change record and the field change record is a business change operation, both the table structure synchronization and field synchronization operations are performed simultaneously.

[0168] In the technical solution provided in this embodiment, the source of change of the table change record and the field change record is used to determine whether to perform a database synchronization operation. After determining that a database synchronization operation should be performed, the table structure synchronization operation and / or field synchronization operation should be further determined, thereby improving the accuracy of database synchronization.

[0169] Figure 7 This is a schematic diagram of the system architecture of the sixth embodiment of the database synchronization method of the present invention.

[0170] In this embodiment, the log parsing module 100 is used to monitor the logs of the source database 400 in real time, and parse the change information of the source database 400, such as change statements, changed table names, change operations, change transaction IDs, data before the change, and data after the change, based on the captured change records. Then, this change information is encapsulated into a change information table and sent to the synchronization judgment module 200 of the database synchronization system.

[0171] The synchronization judgment module 200 is used to parse the received change information table and determine whether the change record was generated by the database synchronization operation or by the business change operation.

[0172] The synchronization execution module 300 is used to determine the mapping relationship between the source database 400 and the target database 500 based on the change information table generated by the log parsing module 100; generate a synchronization execution statement carrying a synchronization identifier based on the change information table and the mapping relationship; and send the synchronization execution statement to the target database 500 for execution to realize the database synchronization operation.

[0173] Furthermore, those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the database synchronization device to implement the process steps of the embodiments of the above methods.

[0174] Therefore, the present invention also provides a computer-readable storage medium storing a database synchronization program, which, when executed by a processor, implements the various steps of the database synchronization method described in the above embodiments.

[0175] The computer-readable storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0176] It should be noted that, since the storage medium provided in the embodiments of this application is the storage medium used to implement the methods of the embodiments of this application, those skilled in the art can understand the specific structure and variations of the storage medium based on the methods described in the embodiments of this application, and therefore will not be repeated here. All storage media used in the methods of the embodiments of this application fall within the scope of protection of this application.

[0177] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0178] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0179] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0180] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0181] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, third, etc., does not indicate any order. These words can be interpreted as names.

[0182] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0183] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A database synchronization method, characterized in that, The database synchronization method is applied to a database synchronization system, which includes a log parsing module, a synchronization judgment module, and a synchronization execution module. When the log parsing module detects an update of a change record, it obtains the change information based on the change record, encapsulates the change information into a change information table, and sends it to the synchronization judgment module. The change record includes table change records and field change records, including: generating a table structure change information table based on the table change record; and generating a field change information table based on the field change record. After the synchronization judgment module receives the change information table, it determines the source of the change record based on the change information table, including: calling the change statement in the change information table and analyzing the change statement to determine whether the change statement carries a synchronization identifier; if the change statement carries the synchronization identifier, the source of the change is determined to be a database synchronization operation; if the change statement does not carry the synchronization identifier, the source of the change is determined to be a business change operation. When the source of the change is a database synchronization operation, the synchronization execution module performs a change record deletion operation; When the change originates from a business change operation, the synchronization execution module generates a synchronization execution statement carrying a synchronization identifier based on the change information table, and sends the synchronization execution statement to the target database for execution to achieve database synchronization. This includes: determining the table structure mapping relationship and field mapping relationship between the source database and the target database based on the table structure change information table and the field change information table; generating a table synchronization execution statement carrying a synchronization identifier based on the table structure mapping relationship and the table structure change information table; and generating a field synchronization execution statement carrying a synchronization identifier based on the field mapping relationship and the field change information table. After the step of determining the source of change of the change record based on the change information table after the synchronization judgment module receives the change information table, the method further includes: Based on the source of the change, determine whether to perform the database synchronization operation; If the source of changes to both the table change record and the field change record is a database synchronization operation, then the database synchronization operation will not be performed, and the table change record and the field change record will be deleted. If the source of the change in the table record and / or the field record is a business change operation, then the database synchronization operation will be executed.

2. The database synchronization method as described in claim 1, characterized in that, When the source of the change is a business change operation, the step of the synchronization execution module generating a synchronization execution statement carrying a synchronization identifier based on the change information table includes: Retrieve the change information from the change information table; Based on the change information, determine the data to be synchronized, and determine the mapping relationship between the source database and the target database; Based on the data to be synchronized and the mapping relationship, the change information of the source database is converted into the change information of the target database; Add a synchronization identifier to the change information of the target database, and generate the synchronization execution statement carrying the synchronization identifier.

3. The database synchronization method as described in claim 1, characterized in that, When the source of the change is a business change operation, the synchronization execution module generates a synchronization execution statement carrying a synchronization identifier based on the change information table, and sends the synchronization execution statement to the target database for execution to achieve database synchronization operation. Following this step, the method further includes: When the target database receives the synchronization execution statement carrying the synchronization identifier, it executes the synchronization execution statement to obtain the change information of the target database; The change information of the target database is applied to the target database to achieve database synchronization.

4. A database synchronization device, characterized in that, The database synchronization device includes: a memory, a processor, and a database synchronization program stored in the memory and executable on the processor, the database synchronization program being configured to implement the steps of the database synchronization method as described in any one of claims 1 to 3.

5. A readable storage medium, characterized in that, The readable storage medium stores a database synchronization program, which, when executed by a processor, implements the steps of the database synchronization method as described in any one of claims 1 to 3.

Images