A method, device, equipment and storage medium for data synchronization

Abstract

The present application provides a method, apparatus, device and storage medium for data synchronization. The method includes: determining the active group in the group to which each registered slave library in the master library belongs; judging whether a synchronization confirmation message for the transaction to be synchronized is received from at least one slave library in each of the active groups, and if so, submitting the transaction to be synchronized. The present application utilizes at least one slave library in each active group to realize data synchronization of the transaction to be synchronized, without having to complete data synchronization of the transaction to be synchronized in each slave library, thereby ensuring the transaction processing performance of data synchronization. Moreover, each active group has at least one slave library that completes data synchronization of the transaction to be synchronized, so that when the entire active group where the master library is located fails, the slave libraries that have completed data synchronization in other active groups can still be used to realize master-slave library switching, thereby improving the disaster recovery range of data synchronization.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and specifically to a method, apparatus, device and storage medium for data synchronization. Background Art

[0002] In order to ensure the security of business data, a master-slave architecture is usually used to build a corresponding database cluster when storing data, so as to achieve master-slave synchronization of various business data, thereby improving the high availability and scalability of database storage. At present, asynchronous replication or full synchronous replication is usually used for data synchronization.

[0003] However, in asynchronous replication, the master database does not pay attention to the data synchronization of the current transaction by the slave database, but directly returns the transaction processing result to the client. When the master database fails, the slave database may not have completed data synchronization, resulting in incomplete data after the slave database switches to the master database, which greatly reduces the disaster recovery effect of data synchronization. In full synchronous replication, the master database will wait until all slave databases have completed data synchronization of the current transaction before returning the transaction processing result to the client, which greatly affects the transaction processing performance. Therefore, in order to solve the above problems, it is urgent to design a more efficient disaster recovery data synchronization solution. Summary of the invention

[0004] The present application provides a data synchronization method, apparatus, device and storage medium, which utilizes a group synchronization method to improve the disaster recovery range of data synchronization while ensuring transaction processing performance.

[0005] In a first aspect, an embodiment of the present application provides a method for data synchronization, the method comprising:

[0006] Determine the active group in the group to which each slave library registered in the master library belongs;

[0007] Determine whether a synchronization confirmation message for the transaction to be synchronized is received from at least one slave database in each of the active groups, and if so, commit the transaction to be synchronized.

[0008] In a second aspect, an embodiment of the present application provides a data synchronization device, the device comprising:

[0009] A group determination module is used to determine the active group in the group to which each slave database registered in the master database belongs;

[0010] The data synchronization module is used to determine whether a synchronization confirmation message for a transaction to be synchronized is received from at least one slave database in each of the active groups, and if so, submit the transaction to be synchronized.

[0011] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0012] A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the data synchronization method provided in the first aspect of the present application.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program enables a computer to execute the data synchronization method provided in the first aspect of the present application.

[0014] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instructions, characterized in that when the computer program / instructions are executed by a processor, the method for data synchronization provided in the first aspect of the present application is implemented.

[0015] The embodiment of the present application provides a method, device, equipment and storage medium for data synchronization, which sets groups for each slave library registered in the master library, and determines the active group from each group, and then executes the commit operation of the transaction to be synchronized when the master library receives the synchronization confirmation message of at least one slave library in each active group for the transaction to be synchronized, thereby using at least one slave library in each active group to achieve data synchronization of the transaction to be synchronized, without having to complete the data synchronization of the transaction to be synchronized in each slave library, thereby ensuring the transaction processing performance of data synchronization. Moreover, each active group has at least one slave library that completes the data synchronization of the transaction to be synchronized, so that when the entire active group where the master library is located fails, the slave libraries that have completed data synchronization in other active groups can still be used to achieve master-slave library switching, thereby improving the disaster tolerance range of data synchronization. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A flowchart of a data synchronization method shown in an embodiment of the present application;

[0018] Figure 2 A flowchart of another data synchronization method shown in an embodiment of the present application;

[0019] Figure 3 A principle block diagram of a data synchronization device shown in an embodiment of the present application;

[0020] Figure 4 It is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0023] Taking into account asynchronous replication and full synchronous replication during data synchronization, the present application designs a new data synchronization solution for situations where the disaster recovery effect and transaction processing performance of data synchronization are incompatible. By grouping the slave libraries of the master library, and then submitting the transactions to be synchronized after determining that at least one slave library in each active group in each group has completed the synchronization confirmation of the transactions to be synchronized. At this time, at least one slave library in each active group is used to realize data synchronization of the transactions to be synchronized, and there is no need to complete data synchronization of the transactions to be synchronized in each slave library, thereby ensuring the transaction processing performance of data synchronization. Moreover, there is at least one slave library in each active group that completes the data synchronization of the transactions to be synchronized, so that when the entire active group where the master library is located fails, the slave libraries that have completed data synchronization in other active groups can still be used to realize the master-slave library switching, thereby improving the disaster recovery range of data synchronization.

[0024] Figure 1 This is a flow chart of a data synchronization method shown in an embodiment of the present application. Figure 1 , the method may specifically include the following steps:

[0025] S110, determining the active group in the group to which each slave library registered in the master library belongs.

[0026] The transactions to be synchronized in this application are various database transactions pointed to by the transaction request initiated by the user to the master database and executed by the master database. At this time, in order to ensure data synchronization between the master and slave databases, for any transaction processed by the master database, the slave database is required to complete the same processing once to ensure that the same transaction data exists between the master and slave databases.

[0027] As an optional implementation scheme in the present application, when data synchronization is performed between the master and slave libraries, after the master library completes processing of a certain transaction, the transaction will be used as the transaction to be synchronized in the present application. Moreover, the master library will generate a binary log (binlog) of the transaction to be synchronized. Then, each subsequent slave library can request the binary log of the transaction to be synchronized from the master library to perform the data synchronization operation of the transaction to be synchronized on the master and slave libraries.

[0028] In this application, in order to achieve the effect of compatible transaction processing performance and data synchronization disaster recovery performance during data synchronization, that is, on the basis of improving the efficiency of transaction processing of the master database as much as possible, it is also possible to ensure that the data synchronization of transactions on each slave database has a larger disaster recovery range to improve the disaster recovery of data synchronization.

[0029] At this time, in order to ensure that data synchronization has a larger disaster tolerance range on the slave database, this application will set a grouping information for each slave database registered in the master database. When registering each slave database to the master database, each slave database is divided into different groups in advance, so that the master database can set corresponding data synchronization logic for the slave databases in different groups, so that data synchronization can have disaster tolerance performance at the group level.

[0030] It should be noted that since the slave libraries registered in the master library may be deployed in multiple computer rooms, if a computer room fails or is blocked, the data synchronization operation of all slave libraries in the computer room for the transaction may fail, and then when the master library fails, if a slave library in the computer room switches to the master library, the data of the switched master library will be incomplete, which greatly reduces the disaster recovery effect of data synchronization. Therefore, in this application, the slave libraries in the same computer room can be divided into the same group according to the computer rooms where the slave libraries are located, and then it is determined whether there is at least one slave library in different groups that successfully completes the data synchronization operation for the transaction, so as to judge the success of the data synchronization of the transaction at the group level to improve the disaster recovery range during data synchronization.

[0031] It should be understood that by configuring the Internet Protocol (IP) address, user name and password of the master library on each slave library, each slave library can be connected to the master library and registered with the master library. Moreover, when each slave library registers with the master library, a data packet is set, which contains the service identifier, IP address, port and group of the slave library. Then, after receiving the registration information of each slave library, the master library can determine the group to which each slave library belongs.

[0032] As an optional implementation scheme in the present application, considering that for data synchronization of any transaction, all slave libraries in the same group may affect the success of data synchronization of any slave library in the group due to group failure or blocking. However, the slave libraries in other groups can normally perform the data synchronization operation of the transaction, and the data synchronization disaster recovery performance when the master library fails can still be guaranteed. Therefore, the present application can divide each group into an active group that can normally perform data synchronization and a blocked group that cannot normally perform data synchronization due to group failure or blocking.

[0033] In this application, in order to avoid the impact of a certain group failure or blockage on transaction processing performance, this application can find out each active group from all the groups to which each slave library registered in the master library belongs. Then, the data synchronization status of the slave library in each active group for the transaction can be judged later.

[0034] S120, determining whether a synchronization confirmation message for the transaction to be synchronized is received from at least one slave in each active group, and if so, committing the transaction to be synchronized.

[0035] After determining the active groups in the group to which the slaves belong, for any pending synchronization transaction completed by the master, the slaves in each active group will continuously request the master for the binary log generated by each pending synchronization transaction after the master completes the processing according to their own progress in synchronizing the transaction data, so that each slave can synchronize the data of the pending synchronization transaction by executing the binary log. After a slave completes the data synchronization operation of the pending synchronization transaction, the slave will reply to the master with a synchronization confirmation message for the pending synchronization transaction, so that the master knows that the slave has completed the data synchronization of the pending synchronization transaction, that is, the data in the slave is currently consistent with that in the master.

[0036] As an optional implementation scheme in the present application, in order to achieve disaster recovery of data synchronization at the group level as much as possible while ensuring the high efficiency of transaction processing, the present application will determine whether there is at least one slave library in each active group that has completed the data synchronization of the transaction to be synchronized. Specifically, after the slave library completes the data synchronization operation of the transaction to be synchronized, it will reply a synchronization confirmation message of the transaction to be synchronized to the master library. Therefore, by determining whether the master library receives the synchronization confirmation message of the transaction to be synchronized from at least one slave library in each active group, it is determined whether there is at least one slave library in each active group that has completed the data synchronization of the transaction to be synchronized.

[0037] At this time, if the master database receives a synchronization confirmation message from at least one slave database in each active group for the synchronization transaction, it means that at least one slave database in each active group has completed data synchronization for the synchronization transaction, that is, there will be at least one slave database in each active group whose data is consistent with the master database in real time. Therefore, when the master database receives a synchronization confirmation message from at least one slave database in each active group for the synchronization transaction, it can directly submit the transaction to be synchronized, thereby improving the efficiency of transaction processing as much as possible. Moreover, even when the master database fails, there will be at least one slave database in each active group whose data is consistent with the master database in real time, so any slave database in the active group that is consistent with the master database in real time can be directly switched to the master database to ensure data integrity when the master-slave database is switched, thereby realizing data synchronization disaster recovery at the group level (computer room level).

[0038] Exemplarily, in the present application, a confirmation receiving (ack_receiver) thread is set in the master library, and the ack_receiver thread is used to receive the synchronization confirmation message for the transaction to be synchronized replied by each slave library in any group after completing the data synchronization of any transaction to be synchronized, and execute the judgment logic of whether the master library has received the synchronization confirmation message for the transaction to be synchronized from at least one slave library in each active group.

[0039] In addition, considering that the data synchronization performance of each group for transactions is uncertain at the beginning of data synchronization between the master and slave databases, each group will be regarded as an active group. Then, in the subsequent actual data synchronization process, the data synchronization performance of each group for transactions is analyzed in real time to convert the groups in the fault or blocked state from the active group to the blocked group.

[0040] Therefore, in the actual data synchronization process, the present application will also judge the data synchronization performance of each active group for the transaction in real time. That is, for any active group, if the master library still does not receive the synchronization confirmation message of any slave library in the active group for the synchronization transaction after exceeding the preset synchronization time, the active group is converted into a blocking group. After the master library sends the binary log of the transaction to be synchronized to each slave library in any active group, it will judge the data synchronization time of each slave library in the active group for the transaction to be synchronized. If the data synchronization time of each slave library in the active group for the synchronization transaction exceeds the preset synchronization time, the master library still does not receive the synchronization confirmation message of any slave library in the active group for the synchronization transaction, indicating that the data synchronization of the active group for the synchronization transaction is faulty or blocked, so the active group is directly converted into a blocking group. Exemplarily, the present application can directly dissolve the blocking group and release each slave library in the blocking group, so that the slave library uses asynchronous replication to synchronize data for the synchronization transaction, so as to restore the normal data synchronization performance of each slave library in the blocking group as much as possible.

[0041] Furthermore, if the blocking group is to be restored to an active group, it is necessary to determine whether the data synchronization progress of each slave library in the blocking group for the transaction to be synchronized can be consistent with the data synchronization progress of each slave library in each active group for the transaction to be synchronized. Therefore, if a synchronization confirmation message is received from any slave library, and the group to which the slave library belongs is a blocking group, the group synchronization point of the blocking group is determined according to the synchronization confirmation message of the slave library; when the group synchronization point of the blocking group is greater than or equal to the minimum group synchronization point of each active group in the master library, the blocking group is converted back to an active group.

[0042] At this time, by analyzing the data synchronization progress of each slave library for the transaction to be synchronized, a synchronization point can be set for each slave library, and the synchronization point is used to indicate the log offset position of the transaction to be synchronized that currently needs to perform data synchronization in each slave library in the log file. For example, the synchronization point in each slave library can be file_name+file_pos. Moreover, in this application, the binary logs of each transaction to be synchronized will be arranged in sequence in the log file according to the transaction synchronization order.

[0043] Therefore, when the master library receives a synchronization confirmation message from any slave library for a transaction to be synchronized, it first determines whether the slave library belongs to an active group or a blocking group. If the group to which the slave library belongs is an active group, it is sufficient to directly determine whether the master library has received a synchronization confirmation message from at least one slave library in each active group for a transaction to be synchronized. However, if the group to which the slave library belongs is a blocking group, it is necessary to analyze the most recent synchronization confirmation message that each slave library in the blocking group has replied to the master library. Then, according to the most recent synchronization confirmation message that each slave library in the blocking group has replied to the master library, determine the synchronization point of each slave library in the blocking group, that is, the current data synchronization progress of the transaction to be synchronized by the slave library. Furthermore, from the synchronization points of each slave library in the blocking group, the largest synchronization point, that is, the synchronization point of the slave library with the fastest data synchronization progress, can be screened out as the group synchronization point of the blocking group.

[0044] At the same time, according to the same determination method of the group synchronization point as described above, the group synchronization point of each active group can be determined by analyzing the synchronization points of each slave database in each active group. Then, considering that the group synchronization point of each active group in the master database can represent the data synchronization progress of each active group for the transaction to be synchronized, by comparing the group synchronization point of the blocking group with the minimum group synchronization point in each active group in the master database, it can be determined whether the data synchronization progress of the blocking group for the transaction to be synchronized is consistent with the data synchronization progress of the transaction to be synchronized in each active group. If the group synchronization point of the blocking group is greater than or equal to the minimum group synchronization point in each active group in the master database, it means that the data synchronization progress of the blocking group can be consistent with the data synchronization progress of each active group, so the blocking group is converted back to the active group. At this time, if the blocking group has been disbanded, then the group is recreated and configured as an active group. However, if the group synchronization point of the blocking group is less than the minimum group synchronization point in each active group in the master database, it means that the data synchronization progress of the blocking group has not caught up with the data synchronization progress of each active group, so the state of the blocking group continues to be maintained.

[0045] The technical solution provided by the embodiment of the present application is to set groups for each slave library registered in the master library, and determine the active group from each group, and then execute the commit operation of the transaction to be synchronized when the master library receives the synchronization confirmation message of at least one slave library in each active group for the transaction to be synchronized, so as to realize the data synchronization of the transaction to be synchronized by using at least one slave library in each active group, and it is not necessary to complete the data synchronization of the transaction to be synchronized in each slave library, thereby ensuring the transaction processing performance of data synchronization. Moreover, there is at least one slave library in each active group that completes the data synchronization of the transaction to be synchronized, so that when the entire active group where the master library is located fails, the slave libraries that have completed data synchronization in other active groups can still be used to realize the master-slave library switching, thereby improving the disaster tolerance range of data synchronization.

[0046] As an optional implementation scheme in the present application, in order to improve the disaster recovery range during data synchronization while ensuring transaction processing performance, the present application provides a detailed explanation of the specific process of determining whether the master database has received a synchronization confirmation message from at least one slave database in each active group regarding the synchronization transaction.

[0047] Figure 2 This is a flow chart of another data synchronization method shown in an embodiment of the present application. Figure 2 , the method may specifically include the following steps:

[0048] S210, determining the active group in the group to which each slave library registered in the master library belongs.

[0049] S220, when receiving a synchronization request of a transaction to be synchronized from any slave library, sending a synchronization log of the transaction to be synchronized to the slave library.

[0050] After the master database executes any transaction to be synchronized, a binary log of the transaction to be synchronized will be generated on demand in the log file. At this time, in the data synchronization process between the master and slave databases, considering that the data synchronization performance of each slave database for the transaction to be synchronized is different, if the master database actively sends the binary log of each transaction to be synchronized to each slave database, then there may be a problem that the slave database has a backlog of a large number of binary logs of transactions to be synchronized due to slow synchronization performance, which may cause the slave database to make mistakes and fail in data synchronization of the transaction to be synchronized. Therefore, in order to ensure the orderliness and accuracy of data synchronization of each slave database for the transaction to be synchronized, this application will have each slave database continuously send synchronization requests for each transaction to be synchronized to the master database according to its own data synchronization progress for the transaction to be synchronized.

[0051] Furthermore, each time the master database receives a synchronization request from any slave database for a transaction to be synchronized, the transaction identifier carried in the synchronization request can be parsed. Then, the binary log generated in the master database for the transaction to be synchronized specified by the transaction identifier is obtained as the synchronization log of the transaction to be synchronized in this application. Furthermore, the synchronization log of the transaction to be synchronized is sent to the slave database so that the slave database writes the synchronization log into the relay log in the slave database, thereby completing the data synchronization operation on the transaction to be synchronized.

[0052] At this time, in order to ensure the accuracy of the synchronization log sent by the master library to each slave library, this application will configure a dump thread for each slave library in the master library. When the master library receives a synchronization request from any slave library to be synchronized, the dump thread corresponding to the slave library will be used to send the synchronization log of the transaction to be synchronized to the slave library.

[0053] It should be understood that in order for the master database to know the data synchronization progress of each slave database in each active group for the transaction to be synchronized, the slave database in this application will reply a synchronization confirmation message for the transaction to be synchronized to the master database each time after completing the data synchronization operation for a transaction to be synchronized. Furthermore, the master database determines the overall synchronization progress of each slave database in each active group for the transaction to be synchronized by receiving the synchronization confirmation message for the transaction to be synchronized from each slave database in each active group.

[0054] As an optional implementation scheme in the present application, since the present application mainly performs the commit operation of the transaction to be synchronized by judging whether there is at least one slave library in each active group that completes the data synchronization of the transaction to be synchronized. That is, the present application mainly focuses on the first slave library in each active group that replies to the master library with the synchronization confirmation message of the transaction to be synchronized, without paying attention to whether other slave libraries reply to the master library with the synchronization confirmation message of the transaction to be synchronized. Therefore, when the present application sends the synchronization log of the transaction to be synchronized to any slave library, it will first judge the group synchronization point of the group to which the slave library belongs to determine the fastest synchronization progress of the group to which the slave library belongs. Moreover, the synchronization position identifier of the transaction to be synchronized that needs to be synchronized by the slave library this time is analyzed to determine the data synchronization progress of the slave library this time.

[0055] Then, by comparing the group synchronization point of the group to which the slave belongs and the synchronization position identifier of the transaction to be synchronized by the slave this time, the difference between the fastest synchronization progress of the group to which the slave belongs for the synchronization transaction and the current data synchronization progress of the slave for the synchronization transaction is determined.

[0056] At this time, the comparison results of the group synchronization point of the group to which the slave belongs and the synchronization position identifier of the transaction to be synchronized by the slave this time have the following three situations:

[0057] Case 1: If the group synchronization point of the group to which the slave belongs is empty, it means that each slave in the group to which the slave belongs has not yet replied to the master with a synchronization confirmation message for any transaction to be synchronized, so that the master needs to pay attention to the data synchronization status of each slave in the group in order to accurately determine the group synchronization point of the group later. Therefore, when the master sends the synchronization log of the transaction to be synchronized to the slave, it will not make any additional configuration to the synchronization log, so that after the slave completes the data synchronization operation of the transaction to be synchronized, it will normally reply to the master with a corresponding synchronization confirmation message, so that the master can determine the group synchronization point of the group to which the slave belongs based on the synchronization confirmation message.

[0058] Case 2: If the synchronization position identifier of the transaction to be synchronized is greater than the group synchronization point of the group to which the slave belongs, it means that the data synchronization progress of the slave after completing the data synchronization operation of the transaction to be synchronized will be greater than the current fastest synchronization progress of the group to which the slave belongs. That is, after the slave completes the data synchronization operation of the transaction to be synchronized, the fastest synchronization progress of the group to which the slave also needs to be updated accordingly. Therefore, when the master sends the synchronization log of the transaction to be synchronized to the slave, it will not make any additional configuration to the synchronization log, so that the slave can normally reply to the master with a corresponding synchronization confirmation message after completing the data synchronization operation of the transaction to be synchronized. Then, the master updates the group synchronization point of the group to which the slave belongs according to the synchronization confirmation message replied by the slave, thereby ensuring the real-time update of the fastest synchronization progress of the group to which the slave belongs.

[0059] Case three: If the synchronization position identifier of the transaction to be synchronized is less than or equal to the group synchronization point of the group to which the slave belongs, it means that the data synchronization progress of the slave after completing the data synchronization operation of the transaction to be synchronized will still not affect the current fastest synchronization progress of the group to which the slave belongs. Therefore, this application will additionally configure the corresponding synchronization confirmation cancellation information in the synchronization log of the transaction to be synchronized. The synchronization confirmation cancellation information indicates that after completing the data synchronization operation of the transaction to be synchronized, the slave does not need to reply to the master for the synchronization confirmation message of this data synchronization. Then, the additionally configured synchronization log of the transaction to be synchronized is sent to the slave.

[0060] S230: Determine the group synchronization point of each active group according to the synchronization confirmation message replied by each slave database in each active group.

[0061] After the master library sends the synchronization log of the transaction to be synchronized to each slave library based on the synchronization request of any slave library to be synchronized, it will continuously receive the synchronization confirmation messages replied by each slave library after completing the data synchronization operation of the transaction to be synchronized. At this time, for each active group, this application will obtain the synchronization confirmation messages replied by each slave library currently received in the active group. Then, by judging the synchronization position identifier of the transaction to be synchronized represented by the synchronization confirmation message replied by each slave library in the active group, the synchronization point of each slave library in the active group is determined. Finally, from the synchronization point of each slave library in the active group, the synchronization point indicating the fastest synchronization progress can be screened out as the group synchronization point of the active group.

[0062] As an exemplary solution in this application, in order to accurately record the synchronization status of each active group for the transactions to be synchronized, this application will introduce an active group (activegroup) class in the main library, and construct a GroupNode linked list (array) structure in the activegroup class. Furthermore, in the main library, various types of information of each active group can be recorded through the GroupNode linked list. For example, the GroupNode linked list of each active group may include the group name, group synchronization point, and each slave library contained in the group. Among them, each group in this application may include but is not limited to 256 slave libraries at most.

[0063] S240: Based on the group synchronization point of each active group, select the slowest synchronization group from each active group.

[0064] Considering that the master database will submit the transaction to be synchronized only when there is at least one slave database in each active group that has completed data synchronization of the transaction to be synchronized, that is, there is at least one slave database in each active group that is consistent with the master database data in real time. Therefore, this application determines the fastest synchronization progress of the active group for the transaction to be synchronized by analyzing the group synchronization point of each active group. Then, according to the fastest synchronization progress of each active group for the transaction to be synchronized, the active group with the slowest synchronization progress can be selected from each active group as the slowest synchronization group in this application, so as to analyze the overall synchronization progress of each active group for the transaction to be synchronized according to the update of the group synchronization point of the slowest synchronization group.

[0065] As an exemplary solution in the present application, when various types of information of each active group are recorded in the main library through the GroupNode linked list, the present application can also construct a specific pointer (*slowest_group) specifically for the GroupNode linked list, which points to the slowest synchronization group in the GroupNode linked list.

[0066] S250, when it is detected that the group synchronization point of the slowest synchronization group has completed an update, determine the pending synchronization transaction corresponding to the group synchronization point before the slowest synchronization group is updated, and submit the pending synchronization transaction.

[0067] Since the slowest synchronization group is the active group with the slowest data synchronization progress for the transactions to be synchronized among all active groups, it means that at least one slave in other active groups has completed data synchronization for all transactions to be synchronized before the group synchronization point of the slowest synchronization group. Therefore, it can be considered that the group synchronization point of the slowest synchronization group represents the overall synchronization progress of all active groups for the transactions to be synchronized, and all transactions to be synchronized before the group synchronization point of the slowest synchronization group have been committed.

[0068] Therefore, by judging whether the group synchronization point of the slowest synchronization group is updated, it can be determined whether each active group has at least one slave library that has completed the data synchronization of the pending synchronization transaction. At this time, when the master library detects that the group synchronization point of the slowest synchronization group has completed an update, it means that there is also a slave library in the slowest synchronization group that has completed the data synchronization operation for the pending synchronization transaction represented by the group synchronization point before the update, so that each active group will have at least one slave library that has completed the data synchronization of the pending synchronization transaction represented by the group synchronization point before the slowest synchronization group was updated. Then, the master library can directly submit the pending synchronization transaction corresponding to the group synchronization point before the slowest synchronization group was updated.

[0069] It should be noted that in order to ensure the accuracy of the slowest synchronization group during data synchronization, this application will analyze the updated group synchronization point of each active group again every time it detects that the group synchronization point of the slowest synchronization group has completed an update, and then reselect the current slowest synchronization group.

[0070] For example, if the group synchronization point of the group pointed to by the *slowest_group pointer is not updated, then the slowest synchronization group in this application remains unchanged. However, if the group synchronization point of the group pointed to by the *slowest_group pointer is updated, it is necessary to traverse the GroupNode linked list. Then, based on the group synchronization point of each active group currently recorded in the GroupNode linked list, a new slowest synchronization group is elected, thereby ensuring that the *slowest_group pointer can always point to the slowest synchronization group in the GroupNode linked list.

[0071] The technical solution provided by the embodiment of the present application is to set groups for each slave library registered in the master library, and determine the active group from each group, and then execute the commit operation of the transaction to be synchronized when the master library receives the synchronization confirmation message of at least one slave library in each active group for the transaction to be synchronized, so as to realize the data synchronization of the transaction to be synchronized by using at least one slave library in each active group, and it is not necessary to complete the data synchronization of the transaction to be synchronized in each slave library, thereby ensuring the transaction processing performance of data synchronization. Moreover, there is at least one slave library in each active group that completes the data synchronization of the transaction to be synchronized, so that when the entire active group where the master library is located fails, the slave libraries that have completed data synchronization in other active groups can still be used to realize the master-slave library switching, thereby improving the disaster tolerance range of data synchronization.

[0072] Figure 3 This is a principle block diagram of a data synchronization device shown in an embodiment of the present application. Figure 3 As shown, the device 300 may include:

[0073] The group determination module 310 is used to determine the active group in the group to which each slave library registered in the master library belongs;

[0074] The data synchronization module 320 is used to determine whether a synchronization confirmation message for a transaction to be synchronized is received from at least one slave database in each of the active groups, and if so, submit the transaction to be synchronized.

[0075] Furthermore, the data synchronization module 320 may be specifically used for:

[0076] Determine the group synchronization point of each active group based on the synchronization confirmation message replied by each slave database in each active group;

[0077] Based on the group synchronization point of each active group, selecting the slowest synchronization group from each of the active groups;

[0078] When it is detected that the group synchronization point of the slowest synchronization group has completed an update, a pending synchronization transaction corresponding to the group synchronization point before the slowest synchronization group is updated is determined, and the pending synchronization transaction is submitted.

[0079] Furthermore, the data synchronization device 300 may further include:

[0080] The slowest group update module is used to reselect the slowest synchronization group based on the updated group synchronization point of each active group.

[0081] Furthermore, the data synchronization device 300 may further include:

[0082] The synchronization log sending module is used to send the synchronization log of the transaction to be synchronized to the slave library when receiving a synchronization request for the transaction to be synchronized from any slave library.

[0083] Furthermore, the synchronization log sending module can be specifically used for:

[0084] If the group synchronization point of the group to which the slave belongs is empty, or the synchronization position identifier of the transaction to be synchronized is greater than the group synchronization point of the group to which the slave belongs, then the synchronization log of the transaction to be synchronized is sent to the slave;

[0085] If the synchronization position identifier of the transaction to be synchronized is less than or equal to the group synchronization point of the group to which the slave library belongs, corresponding synchronization confirmation cancellation information is configured in the synchronization log of the transaction to be synchronized, and the synchronization log of the transaction to be synchronized is sent to the slave library.

[0086] Furthermore, the data synchronization device 300 may further include:

[0087] The first group conversion module is used to convert any active group into a blocking group if the master database has not received a synchronization confirmation message from any slave database in the active group for the transaction to be synchronized after exceeding a preset synchronization time.

[0088] Furthermore, the data synchronization device 300 may further include:

[0089] A group synchronization point determination module, configured to determine the group synchronization point of the blocking group according to the synchronization confirmation message of any slave library if a synchronization confirmation message of the slave library is received and the group to which the slave library belongs is a blocking group;

[0090] The second group conversion module is used to convert the blocking group back to the active group when the group synchronization point of the blocking group is greater than or equal to the minimum group synchronization point of each active group in the main database.

[0091] In the embodiment of the present application, groups are set for each slave library registered in the master library, and active groups are determined from each group. Then, when the master library receives a synchronization confirmation message for the transaction to be synchronized from at least one slave library in each active group, the commit operation of the transaction to be synchronized is performed, thereby using at least one slave library in each active group to achieve data synchronization of the transaction to be synchronized, without having to complete data synchronization of the transaction to be synchronized in each slave library, thereby ensuring the transaction processing performance of data synchronization. Moreover, each active group has at least one slave library that completes data synchronization of the transaction to be synchronized, so that when the entire active group where the master library is located fails, the slave libraries that have completed data synchronization in other active groups can still be used to achieve master-slave library switching, thereby improving the disaster tolerance range of data synchronization.

[0092] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, Figure 3 The device 300 shown can execute the method embodiment provided in the present application, and the aforementioned and other operations and / or functions of each module in the device 300 are respectively for implementing the corresponding processes in each method of the embodiment of the present application. For the sake of brevity, they will not be repeated here.

[0093] The above describes the device 300 of the embodiment of the present application from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0094] Figure 4 It is a schematic block diagram of an electronic device 400 provided in an embodiment of the present application.

[0095] like Figure 4 As shown, the electronic device 400 may include:

[0096] The memory 410 and the processor 420, the memory 410 is used to store the computer program and transmit the program code to the processor 420. In other words, the processor 420 can call and run the computer program from the memory 410 to implement the method in the embodiment of the present application.

[0097] For example, the processor 420 may be configured to execute the above method embodiments according to instructions in the computer program.

[0098] In some embodiments of the present application, the processor 420 may include but is not limited to:

[0099] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0100] In some embodiments of the present application, the memory 410 includes but is not limited to:

[0101] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0102] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 410 and executed by the processor 420 to complete the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0103] like Figure 4 As shown, the electronic device may also include:

[0104] The transceiver 430 may be connected to the processor 420 or the memory 410 .

[0105] The processor 420 may control the transceiver 430 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 430 may include a transmitter and a receiver. The transceiver 430 may further include an antenna, and the number of antennas may be one or more.

[0106] It should be understood that the various components in the electronic device are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0107] The embodiment of the present application also provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, the embodiment of the present application also provides a computer program product containing instructions, and when the instructions are executed by a computer, the computer can perform the method of the above method embodiment.

[0108] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a solid state drive (solid state disk, SSD)), etc.

[0109] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0110] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the module is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0111] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0112] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for data synchronization, characterized in that: include: Determine the active group in the group to which each slave library registered in the master library belongs; Determine whether a synchronization confirmation message of a to-be-synchronized transaction from at least one slave database in each of the active groups is received, and if so, commit the to-be-synchronized transaction; The step of determining whether a synchronization confirmation message of a transaction to be synchronized is received from at least one slave in each active group, and if so, submitting the transaction to be synchronized comprises: Determine the group synchronization point of each active group according to the synchronization confirmation message replied by each slave database in each active group, wherein the group synchronization point of each active group is the synchronization point indicated by the synchronization position identifier of the transaction to be synchronized by a slave database with the fastest synchronization progress for the transaction to be synchronized among the slave databases in the active group; Based on the group synchronization point of each active group, selecting the slowest synchronization group from each of the active groups; When it is detected that the group synchronization point of the slowest synchronization group has completed an update, a pending synchronization transaction corresponding to the group synchronization point before the slowest synchronization group is updated is determined, and the pending synchronization transaction is submitted.

2. The method according to claim 1, characterized in that When it is detected that the group synchronization point of the slowest synchronization group has completed an update, the method further includes: Based on the updated group synchronization point of each active group, the slowest synchronization group is reselected.

3. The method according to claim 1, characterized in that After determining the active group in the group to which each slave library registered in the master library belongs, it also includes: When receiving a synchronization request for the transaction to be synchronized from any slave library, the synchronization log of the transaction to be synchronized is sent to the slave library.

4. The method according to claim 3, characterized in that The sending the synchronization log of the transaction to be synchronized to the slave library includes: If the group synchronization point of the group to which the slave belongs is empty, or the synchronization position identifier of the transaction to be synchronized is greater than the group synchronization point of the group to which the slave belongs, then the synchronization log of the transaction to be synchronized is sent to the slave; If the synchronization position identifier of the transaction to be synchronized is less than or equal to the group synchronization point of the group to which the slave library belongs, corresponding synchronization confirmation cancellation information is configured in the synchronization log of the transaction to be synchronized, and the synchronization log of the transaction to be synchronized is sent to the slave library.

5. The method according to claim 1, characterized in that The method further comprises: For any active group, if the master database still does not receive a synchronization confirmation message for the transaction to be synchronized from any slave database in the active group after exceeding the preset synchronization time, the active group is converted into a blocking group.

6. The method according to claim 5, characterized in that The method further comprises: If a synchronization confirmation message of any slave library is received, and the group to which the slave library belongs is a blocking group, the group synchronization point of the blocking group is determined according to the synchronization confirmation message of the slave library; When the group synchronization point of the blocking group is greater than or equal to the minimum group synchronization point of each active group in the master database, the blocking group is converted back to an active group.

7. A data synchronization device, characterized in that: include: A group determination module is used to determine the active group in the group to which each slave database registered in the master database belongs; A data synchronization module, used to determine whether a synchronization confirmation message of a to-be-synchronized transaction is received from at least one slave database in each of the active groups, and if so, submit the to-be-synchronized transaction; Wherein, the data synchronization module is specifically used for: Determine the group synchronization point of each active group according to the synchronization confirmation message replied by each slave database in each active group, wherein the group synchronization point of each active group is the synchronization point indicated by the synchronization position identifier of the transaction to be synchronized by a slave database with the fastest synchronization progress for the transaction to be synchronized among the slave databases in the active group; Based on the group synchronization point of each active group, selecting the slowest synchronization group from each of the active groups; When it is detected that the group synchronization point of the slowest synchronization group has completed an update, a pending synchronization transaction corresponding to the group synchronization point before the slowest synchronization group is updated is determined, and the pending synchronization transaction is submitted.

8. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the data synchronization method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program enables a computer to execute the data synchronization method according to any one of claims 1 to 6.

Images