A distributed database system and a synchronization method and device for member change thereof

Abstract

The specification provides a distributed database system and a member change synchronization method and device thereof, comprising a first replica group, the first replica group comprising a plurality of members, the roles of the members being master replica nodes and slave replica nodes, the master replica nodes and the slave replica nodes maintaining a member list for recording the members in the first replica group; the master replica node is configured to, in the case of receiving a member change request, generate a corresponding member change log and update the member list maintained by itself; and synchronize the member change log to the slave replica node, and determine whether the synchronization of the member change log is completed based on the confirmation information fed back by the slave replica node; the slave replica node is configured to, after receiving the member change log sent by the master replica node, update the member list maintained by itself based on the member change log.

Description

Technical Field

[0001] This specification relates to the field of database technology, and in particular to a distributed database system and a method and apparatus for synchronizing changes in its members. Background Technology

[0002] In related technologies, for member changes in a distributed database system, the primary replica node needs to write the corresponding member change information into the data log and synchronize the data log to each secondary replica node based on the data log synchronization mechanism. After receiving the data log, the secondary replica node needs to parse the corresponding member change information from it and update its own maintained member list, ultimately ensuring that the member lists maintained by the primary replica node and the secondary replica node are consistent.

[0003] However, the aforementioned method of writing member change information to the data log has several drawbacks. First, when a replica node updates its member list, it needs to parse the member change information from the data log before updating the member list, impacting the efficiency of member list updates. Second, when the data log write interval is long, member changes occurring within that interval cannot be synchronized to the replica nodes in a timely manner, affecting the consistency of the member list between the primary and secondary replica nodes. Furthermore, when data synchronization involves cross-replica groups (such as the first and second replica groups), since the first and second replica groups maintain independent member lists, member change information from the first replica group is useless to the second replica group. Therefore, the second replica group needs to parse and discard the member change information from the data log sent by the first replica group before completing data synchronization, a cumbersome operation that also affects the efficiency of data synchronization. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this specification provides a method and apparatus for synchronizing changes in a distributed database system and its members.

[0005] According to a first aspect of the embodiments of this specification, a distributed database system is provided, including a first replica group, the first replica group including a plurality of members, the roles of such members being a primary replica node and a secondary replica node, the primary replica node and the secondary replica node maintaining a member list for recording the members in the first replica group;

[0006] The primary replica node is used to generate a corresponding member change log and update its own maintained member list when it receives a member change request. The member change log and the data log belong to different log streams. It also synchronizes the member change log to the secondary replica node and determines whether the member change log synchronization is complete based on the confirmation information fed back by the secondary replica node.

[0007] The slave replica node is configured to update its own maintained member list based on the member change log sent by the primary replica node after receiving the member change log; and, upon completion of the member list update, send corresponding confirmation information to the primary replica node.

[0008] According to a second aspect of the embodiments of this specification, a method for synchronizing member changes in a distributed database system is provided. The distributed database system includes a first replica group, which includes several members, whose roles are primary replica node and secondary replica node. The primary replica node and secondary replica node maintain a member list for recording members in the first replica group, which is applied to the primary replica node. The method includes:

[0009] Upon receiving a member change request, the system generates a corresponding member change log and updates its own maintained member list. The member change log and the data log belong to different log streams.

[0010] The member change log is synchronized to the slave replica node, and the synchronization of the member change log is determined based on the confirmation information returned by the slave replica node.

[0011] According to a third aspect of the embodiments of this specification, a method for synchronizing member changes in a distributed database system is provided. The distributed database system includes a first replica group, which includes several members, whose roles are primary replica node and secondary replica node. The primary replica node and the secondary replica node maintain a member list for recording members in the first replica group, which is applied to the secondary replica node. The method includes:

[0012] After receiving the member change log sent by the primary replica node, the primary replica node updates its own maintained member list based on the member change log. The member change log and the data log belong to different log streams. The member change log is generated by the primary replica node after receiving the member change request.

[0013] Once the member list update is complete, a corresponding confirmation message is sent to the primary replica node, so that the primary replica node can determine whether the member change log synchronization is complete based on the confirmation message returned by the secondary replica node.

[0014] According to a fourth aspect of the embodiments of this specification, a synchronization device for member changes in a distributed database system is provided. The distributed database system includes a first replica group, which includes a plurality of members, the roles of which are a primary replica node and a secondary replica node. The primary replica node and the secondary replica node maintain a member list for recording members in the first replica group, which is applied to the primary replica node. The device includes:

[0015] The generation unit is used to generate a corresponding member change log and update the member list it maintains when a member change request is received. The member change log and the data log belong to different log streams.

[0016] The determining unit is used to synchronize the member change log to the slave replica node and determine whether the member change log synchronization is complete based on the confirmation information fed back by the slave replica node.

[0017] According to a fifth aspect of the embodiments of this specification, a synchronization device for member changes in a distributed database system is provided. The distributed database system includes a first replica group, which includes a plurality of members, the roles of which are primary replica nodes and secondary replica nodes. The primary replica nodes and the secondary replica nodes maintain a member list for recording members in the first replica group, which is applied to the secondary replica nodes. The device includes:

[0018] The update unit is used to update the member list it maintains based on the member change log after receiving the member change log sent by the primary replica node. The member change log and the data log belong to different log streams. The member change log is generated by the primary replica node after receiving the member change request.

[0019] The sending unit is used to send corresponding confirmation information to the primary replica node when the member list update is completed, so that the primary replica node can determine whether the member change log synchronization is completed based on the confirmation information fed back by the secondary replica node.

[0020] According to a sixth aspect of the embodiments of this specification, an electronic device is provided, comprising:

[0021] A processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the steps of the method described in the second or third aspect above.

[0022] According to a seventh aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores executable instructions thereon; wherein, when executed by a processor, the instructions implement the steps of the methods described in the second or third aspect above.

[0023] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this specification and, together with the description, serve to explain the principles of this specification.

[0025] Figure 1 This is a schematic diagram of the architecture of a distributed database system provided in an exemplary embodiment of this specification;

[0026] Figure 2 This is a flowchart illustrating a synchronization method for member changes in a distributed database system, provided in an exemplary embodiment of this specification.

[0027] Figure 3 This is a schematic diagram illustrating a primary replica node broadcasting a member change log, provided in an exemplary embodiment of this specification.

[0028] Figure 4 This is a schematic diagram illustrating how to determine the progress of data log processing from a replica node, provided in an exemplary embodiment of this specification.

[0029] Figure 5 This is a schematic diagram of a second copy group provided in an exemplary embodiment of this specification;

[0030] Figure 6 This is a flowchart illustrating a synchronization method for member changes in a distributed database system, provided in an exemplary embodiment of this specification.

[0031] Figure 7 This is a schematic structural diagram of an electronic device provided in an exemplary embodiment of this specification;

[0032] Figure 8 This is a block diagram of a synchronization device for member changes in a distributed database system provided in an exemplary embodiment of this specification;

[0033] Figure 9 This is a block diagram of a synchronization device for member changes in a distributed database system, provided in an exemplary embodiment of this specification. Detailed Implementation

[0034] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification.

[0035] It should be noted that in other embodiments, the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification. In some other embodiments, the methods may include more or fewer steps than those described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments. It should be understood that although the terms first, second, third, etc., may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0036] During operation, a distributed database system can modify its node members in response to different service needs. For example, when a node fails, it needs to be replaced with a healthy node; when the system needs to be expanded, a certain number of nodes need to be added; and when the system needs to be streamlined due to performance redundancy, a certain number of nodes need to be deleted. After the member changes are completed, the primary and replica nodes write the corresponding member change information to the data log. Based on the data log synchronization mechanism, each replica node can parse the member change information from the acquired data log, thereby updating its own maintained member list and ultimately achieving member list consistency.

[0037] However, the aforementioned method of writing member change information to the data log prevents the slave replica node from directly updating its maintained member list based on the data log. Instead, it needs to parse the corresponding member change information from the data log. This additional parsing step affects the efficiency of member list updates. Furthermore, when the data log write interval is long, member changes occurring within that interval cannot be synchronized to the slave replica node in a timely manner. For example, if the primary replica node does not perform any data log write operations within 5 minutes, but multiple member changes occur within those 5 minutes, the slave replica node will not receive the data logs and therefore cannot update its member list. This results in inconsistencies between the member lists maintained by the primary and slave replica nodes, affecting the consistency of member lists between them.

[0038] Furthermore, when data synchronization involves cross-replica groups, such as the first replica group and the second replica group, where the second replica group is used to back up the replicas maintained within the first replica group, and the node members of the second replica group are not the same as those of the first replica group, the member change information in the data log sent by the first replica group is redundant information for the second replica group. Thus, during data synchronization, the second replica group needs to parse out the corresponding member change information of the first replica group from the data log and discard the member change information in order to complete the data synchronization operation. This makes the data synchronization operation of the second replica group cumbersome and the data synchronization efficiency low.

[0039] Therefore, to solve the above problems, this specification proposes an improved synchronization method for member changes in a distributed database system, which will be described in detail below with reference to embodiments.

[0040] Figure 1 This is a schematic diagram of the architecture of a distributed database system provided in an exemplary embodiment of this specification. The system architecture may include a first replica group 10, wherein the first replica group 10 includes several members, whose roles are primary replica nodes and secondary replica nodes. Figure 1 For example, the first replica group 10 may include several members, such as node member 11, node member 12, node member 13, and node member 14. Specifically, node member 11 can be a primary replica node, while node members 12, 13, and 14 can be secondary replica nodes. Therefore, based on the division of member roles, node member 11 can also be called primary replica node 11, corresponding node member 12 can be called secondary replica node 12, node member 13 can be called secondary replica node 13, and so on. Primary replica node 11, secondary replica node 12, secondary replica node 13, and secondary replica node 14 all maintain member lists for recording the members in the first replica group 10. That is to say, Figure 1 Each node member shown can maintain a member list that records the primary replica node 11, the secondary replica node 12, the secondary replica node 13, and the secondary replica node 14.

[0041] The primary replica node 11 can be deployed on a physical server containing an independent host, or it can be deployed on a virtual server (such as a cloud server) hosted in a host cluster; this specification does not impose any restrictions on this. Unlike related technologies where member change information is written to the data log, the primary replica node 11 described in this specification can generate an independent member change log and synchronize this log to each secondary replica node. The primary replica node 11 can determine whether the member change log synchronization is complete based on the confirmation information returned by the secondary replica nodes. Through this independent member change log, each secondary replica node within the first replica group can directly synchronize the member list, eliminating the need to parse member change information from the data log, thus accelerating the efficiency of member change synchronization. Even in cross-replica group scenarios, each replica group can still use the above synchronization method, indirectly improving the data synchronization efficiency across replica groups.

[0042] The slave replica node 12 can be deployed on a physical server containing an independent host, or it can be deployed on a virtual server (such as a cloud server) hosted in a host cluster; this specification does not impose any restrictions on this. After receiving the member change log sent by the primary replica node 11, the slave replica node 12 updates its own maintained member list and sends a corresponding confirmation message to the primary replica node 11. This allows the primary replica node 11 to determine whether the member change log synchronization is complete based on the confirmation message. Because the member change log and the data log described in this specification belong to different log streams, the slave replica node 12 can directly update its member list based on the log after receiving it, without needing to parse the member change information from the data log, thus greatly accelerating the member list update efficiency. Since slave replica nodes 13 and 14 belong to the same role as slave replica node 12, their descriptions can be found above and will not be repeated here.

[0043] Based on the aforementioned distributed database system architecture, the primary replica node generates an independent member change log, preventing member change information from being written into the data log. Consequently, each secondary replica node no longer needs to parse member change information from the data log sent by the primary replica node; instead, it can directly update the member list based on the received member change log, thus accelerating the efficiency of member list updates. Furthermore, even in cross-replica group scenarios, the backup replica group (such as the second replica group) acting as the primary replica group can directly complete data synchronization based on the data log, further accelerating data synchronization efficiency.

[0044] Figure 2This is a flowchart illustrating a synchronization method for member changes in a distributed database system, provided in an exemplary embodiment of this specification. The distributed database system includes a first replica group, which comprises several members. These members act as a primary replica node and a secondary replica node. The primary and secondary replica nodes maintain a member list for recording members in the first replica group, which is applied to the primary replica node. Specifically, the method may include the following steps:

[0045] Step 202: Upon receiving a member change request, generate a corresponding member change log and update the member list maintained by the system. The member change log and the data log belong to different log streams.

[0046] This distributed database system can include corresponding primary and secondary replica nodes. The primary replica node is mainly used to respond to client read and write requests. After completing data read and write operations, the primary replica node can synchronize the updated data to each secondary replica node in the form of a data log. After receiving the data log, the secondary replica node can first persist the data log locally. After local persistence, the secondary replica node can replay the corresponding data based on the persisted data log and write it to its own storage space, such as to disk. After writing the corresponding updated data to disk, the secondary replica node can choose to delete the locally persisted data log. At this point, the secondary replica node can be considered to have completed the processing of the data log, ultimately ensuring that the secondary replica node and the primary replica node maintain the same data, thereby guaranteeing the data consistency of the distributed database system. There are various forms of primary replica node election, such as electing the node with the largest device ID or the node with the largest device memory space. This specification does not limit the specific election method.

[0047] In one embodiment, a corresponding term duration can be set for the primary replica node, for example, the term duration is 10 seconds. Before the term ends, the primary replica node can initiate an extension vote to each secondary replica node (which may include the primary replica node). When the primary replica node receives more than half of the votes in favor of the members of the first replica group, the primary replica node can confirm that the extension is successful. Otherwise, the primary replica node can be demoted to a secondary replica node, and a new primary replica node can be elected. Of course, there is a possibility that the newly elected primary replica node can still be the demoted secondary replica node.

[0048] Member change logs can contain monotonically increasing version numbers. The value of the version number indicates whether the member list is new or old. During the leader election phase, because a node with a smaller version number may no longer be in the first replica group, a node with a larger version number cannot vote for a node with a smaller version number, thus preventing the election of the node with the older version number as the leader. By using version numbers to identify nodes, it prevents nodes no longer in the first replica group from being elected as leader, ensuring the normal operation of the database system service.

[0049] Membership change requests can be sent by different nodes. For example, if the distributed database system needs to be expanded, the membership change request can be sent by the expanding node to the primary replica node, thus indicating to the primary replica node that the expanding node wants to join the first replica group. As another example, if the distributed database system needs to be downsized, the membership change request can be sent by the corresponding removed node, thus indicating to the primary replica node that the removed node has gone offline.

[0050] Upon receiving a member change request, the primary replica node can generate a corresponding member change log. This log records the specific details of the change, such as adding node A or deleting node B. Alternatively, the log can directly record the updated member list. For example, if the initial member list includes nodes A and B, and node C is added after the change, the log can directly record the updated list, including nodes A, B, and C.

[0051] After generating the aforementioned member change log, the primary replica node can update and persist its own maintained member list locally, and broadcast the member change log to each secondary replica node. Thus, each secondary replica node can receive the member change log, update and persist its own maintained member list locally. For example... Figure 3 As shown, Figure 3 This is a schematic diagram illustrating a primary / replica node broadcasting a member change log, provided in an exemplary embodiment of this specification. For example... Figure 3As shown, the first replica group may include a primary replica node 11, a secondary replica node 12, a secondary replica node 13, and a secondary replica node 14. After generating a member change log, the primary replica node 11 can broadcast it to the secondary replica nodes 12, 13, and 14. After receiving the member change log broadcast by the primary replica node, the secondary replica nodes 12, 13, and 14 can update and persist their own maintained member lists locally based on the received member change logs, and send a corresponding confirmation message to the primary replica node after persistence is complete. This confirmation message indicates that the local persistence of the secondary replica node is complete, so the primary replica node 11 can determine whether the member change log synchronization is complete based on the confirmation messages sent by each secondary replica node.

[0052] To ensure the orderly synchronization of member changes and avoid inconsistencies in the member list, for example, if a replica node can only process one member change log at a time, and upon receiving both member change log 1 and member change log 2 simultaneously, the replica node might mistakenly treat member change log 2 as the same as member change log 1, causing it to process only member change log 1 and not member change log 2. This would result in an inconsistency between the member list maintained by the replica node and that maintained by the primary replica node. Alternatively, a replica node can process both member change log 1 and member change log 2, but the order in which they are processed is uncertain. For example, the replica node might process member change log 2 before or after member change log 1. When member change log 1 and member change log 2 conflict (e.g., member change log 1 adds node A, member change log 2 deletes node A), the member list ultimately maintained by the replica node may also differ from that maintained by the primary replica node. Therefore, upon receiving a member change request, the primary replica node can check if there is an ongoing member change in the first replica group. If the check indicates that there is no ongoing member change in the first replica group, it generates a corresponding member change log; or, if the check indicates that there is an ongoing member change in the first replica group, the primary replica node waits for the ongoing member change to complete before generating a corresponding member change log; or, the primary replica node rejects the member change request. For example, suppose that when primary replica node 11 receives a member change request, it is synchronizing member change logs with secondary replica nodes 12, 13, and 14, and the primary replica node has not yet received any confirmation information from secondary replica nodes 12, 13, and 14. It can be assumed that there is an ongoing member change in the first replica group at this time. Primary replica node 11 can wait for the current member change to complete before generating a new member change log for the member change request, or the primary replica node can directly reject the member change request. By actively checking the primary replica node, the orderly progress of member changes within the first replica group can be ensured, avoiding the phenomenon mentioned earlier where a secondary replica node receives multiple member change logs simultaneously, resulting in inconsistencies between the member list maintained by the secondary replica node and that of the primary replica node.

[0053] Data log synchronization can be based on a majority-based mechanism. When the primary replica node broadcasts the data log to all secondary replica nodes, the primary replica node considers data synchronization complete once it receives confirmation messages from more than half of the member lists. Because member change information described in this specification is not written to the data log, data log synchronization based on a majority-based mechanism may result in data loss. For example, after a data update, primary replica node 11 broadcasts the corresponding data log to secondary replica nodes 12 and 13. Primary replica node 11 considers data synchronization successful once it receives confirmation messages from either secondary replica node 12 or 13 (in the case of three nodes, receiving confirmation messages from one secondary replica node and its own confirmation message is sufficient to meet the majority requirement). Therefore, although primary replica node 11 receives confirmation messages from secondary replica node 12 and considers data synchronization complete, secondary replica node 13 may not have received the data log. If a member change occurs at this time, such as removing replica node 12 and adding replica node 14, meaning the member list includes primary replica node 11, replica node 13, and replica node 14, although primary replica node 11 achieves data synchronization through a majority mechanism, in reality only primary replica node 11 stores the corresponding complete data, while replica nodes 13 and 14 do not store the corresponding complete data, causing the data loss phenomenon mentioned above, affecting the reliability of the distributed database system.

[0054] Therefore, to avoid the aforementioned data loss, the member change log can include position identifier information. This position identifier information is used to characterize the data log processing progress of the primary replica node when generating the member change log. In other words, when generating the member change log, the primary replica node can add its own persistent position in the data log stream as position identifier information. This allows the secondary replica node to determine whether its data log processing progress is consistent with that of the primary replica node based on this position identifier information. If the determination result shows that the processing progress of the data log by the secondary replica node is consistent with that of the primary replica node, the secondary replica node can update its maintained member list based on the member change log. That is, before updating the member list, the secondary replica node proactively checks whether its data log processing progress is consistent with that of the primary replica node. Under the premise of consistency, the secondary replica node updates its maintained member list, thus avoiding the aforementioned data loss caused by member changes and improving the reliability of the distributed database system.

[0055] In one embodiment, for ease of execution, the primary replica node can directly use a monotonically increasing data log ID as the corresponding site identifier information as described above. For example, assuming that when the primary replica node generates a member change log, its own data log ID is 100, then the primary replica node can directly add the data log ID 100 to the member change log, thereby allowing the data log ID 100 to represent the data log processing progress of the primary replica node when generating the member change log. Based on the aforementioned data log ID, when a replica node receives a member change log, it can perform a forward check based on the data log ID 100 added by the primary replica node in the member change log to determine whether the data log processing progress of the replica node is consistent with that of the primary replica node. If the check result indicates that the data log processing progress of the replica node is consistent with that of the primary replica node, the replica node can update its own maintained member list based on the member change log. Alternatively, if the check result indicates that the data log processing progress of the replica node has not reached that of the primary replica node, the replica node can update its own maintained member list based on the member change log after its own data log processing progress reaches that of the location identifier information. Or, the replica node can send a corresponding rejection message to the primary replica node to indicate that it rejects the member change log synchronization initiated by the primary replica node.

[0056] The following is combined Figure 4 Taking the data log ID as the location identifier as an example, this paper details how the replica node determines the progress of data log processing based on the data log ID. Figure 4This is a schematic diagram illustrating how a replica node determines the progress of data log processing, provided by an exemplary embodiment of this specification. When generating a member change log, the primary replica node 11 can add its own corresponding data log stream persistence point to the member change log, which can be the data log ID 100 as described above. After receiving the member change log, the secondary replica node, such as secondary replica node 12, can perform a forward check based on the data log ID 100. Assuming that the result of the forward check shows that the data log ID corresponding to secondary replica node 12 is also 100, it means that the data log processing progress of secondary replica node 12 is consistent with that of the primary replica node. In other words, secondary replica node 12 has written the data corresponding to the locally persisted data log 100 to the hard disk corresponding to secondary replica node 12. Since the forward check result from replica node 12 shows that the data log ID of replica node 12 is also 100, it can be assumed that the processing progress of the data log of replica node 12 is consistent with that of the primary replica node. Therefore, replica node 12 can directly update and persist the member list locally based on the member change log, and at the same time send the corresponding confirmation information back to primary replica node 11, thereby indicating that the data and member list maintained by replica node 12 are consistent with those of the primary replica node. Taking replica node 13 as an example, assuming that after receiving the member change log, the forward check shows that the data log ID of replica node 13 is 98 (it can be assumed that replica node 13 was removed from the distributed database system after processing data log 98, and then rejoined the distributed database system, resulting in replica node 13's data log ID remaining at 98), then replica node 13 can wait until it has processed data logs with data log IDs 99 and 100, ensuring that the data log processing progress of replica node 13 is consistent with that of primary replica node 11, before updating and persisting the member list locally, and simultaneously sending the corresponding confirmation information back to replica node 11. Alternatively, taking replica node 14 as an example, when the check result of the replica node shows that the processing progress of the data log by the replica node and the primary replica node is inconsistent, replica node 14 can also directly send a rejection message back to primary replica node 11 to indicate that replica node 14 rejects this member change log synchronization initiated by primary replica node 11. When there is a discrepancy in processing progress, this manual does not restrict the execution action of the slave replica nodes. They can either wait for consistency before execution, as slave replica node 13 does, or directly refuse to execute, as slave replica node 14 does. Whether to wait for execution or refuse to execute can be determined based on the majority number. When the slave replica nodes that meet the majority number of requests for execution reach consistency with the data log processing progress of the primary replica node, the subsequent slave replica nodes can refuse to execute.By introducing location identification information such as data log IDs, each replica node can perform a corresponding forward check after receiving the member change log, thereby ensuring that the data maintained by a majority of replica nodes is consistent with that of the primary replica node. This avoids the phenomenon mentioned earlier where only the primary replica node maintains the corresponding complete data, thus improving the reliability of the distributed database system.

[0057] Step 204: Synchronize the member change log to the slave replica node, and determine whether the member change log synchronization is complete based on the confirmation information returned by the slave replica node.

[0058] A primary replica node can determine that member change log synchronization is complete when it receives confirmation from more than half of the members corresponding to the updated member list it maintains. Figure 1 Taking the illustrated embodiment as an example, after receiving a member change request, the primary replica node 11 generates a corresponding member change log. This log indicates that a new secondary replica node, such as secondary replica node 15, has been added to the first replica group. The primary replica node 11 can then synchronize the member change log to secondary replica nodes 12, 13, 14, and 15. The updated member list corresponds to 5 nodes. The primary replica node 11 only needs to receive 3 or more confirmation messages to confirm that the member change log synchronization is complete. In other words, the primary replica node does not need to wait for each secondary replica node to send confirmation messages before confirming that the member change log synchronization is complete. This majority confirmation mechanism not only ensures the reliability of member change synchronization but also improves the efficiency of task execution and reduces the response time for member changes.

[0059] In one embodiment, the distributed database system may further include a second replica group, which is a backup replica group of the first replica group. The primary replica node can respond to a data change request, generate a corresponding data log, and synchronize the data log to the second replica group, so that the second replica group can complete data synchronization based on the data log. Figure 5 As shown, Figure 5This is a schematic diagram of a second replica group provided in an exemplary embodiment of this specification. The second replica group may include a primary replica node 501, a secondary replica node 502, a secondary replica node 503, and a secondary replica node 504. The second replica group can be used to back up the data stored in the first replica group. In a cross-replica group scenario, since the member change information described in this specification no longer needs to be written to the data log, but is instead generated independently as a member change log, the first replica group can directly synchronize the data log without member change information to the second replica group, and the second replica group can also directly complete data synchronization based on the data log without needing to parse and discard member change information, thus speeding up the data synchronization efficiency. Specifically, the second replica group can process the data log in the manner described above, that is, after each node in the second replica group receives the data log, it first persists the data log locally, and then each node can replay the corresponding data based on its own locally persisted data log and write the replayed data to the hard disk of each node. When each node in the second replica group has finished writing or has finished writing to disk, it can choose to delete the locally persisted data log, and consider the data synchronization to be complete. Of course, to speed up data synchronization, data synchronization is considered complete once a majority of nodes in the second replica group have finished replaying and writing the data logs to disk; it is not necessary to ensure that every node in the second replica group has completed the data log replay and writing process. It should be noted that although in Figure 5 In the embodiments shown, the second copy group is a backup copy group of the first copy group. However, in some embodiments, the second copy group can also be a data recovery copy group of the first copy group, so that the first copy group can use the second copy group to realize data recovery. This specification does not limit this.

[0060] Figure 6 This is a flowchart illustrating a method for synchronizing member changes in a distributed database system, provided in an exemplary embodiment of this specification. The distributed database system includes a first replica group comprising several members, whose roles are primary replica node and secondary replica node. The primary replica node and the secondary replica node maintain a member list for recording members in the first replica group. This list is applied to the secondary replica node and may include the following steps:

[0061] Step 602: After receiving the member change log sent by the primary replica node, update the member list maintained by itself based on the member change log. The member change log and the data log belong to different log streams. The member change log is generated by the primary replica node after receiving the member change request.

[0062] Step 604: If the member list update is complete, send the corresponding confirmation information to the primary replica node so that the primary replica node can determine whether the member change log synchronization is complete based on the confirmation information fed back by the secondary replica node.

[0063] In one embodiment, the member change log may include site identifier information, which is used to characterize the data log processing progress corresponding to the primary replica node when generating the member change log. Upon receiving the member change log, the secondary replica node can check whether its data log processing progress is consistent with that of the primary replica node based on the site identifier information in the member change log.

[0064] If the inspection results indicate that the data log processing progress of the slave replica node is consistent with that of the primary replica node, the slave replica node can update its own maintained member list based on the member change log; or,

[0065] If the inspection results indicate that the data log processing progress of the slave node has not reached the data log processing progress of the primary slave node, the slave node may update its own maintained member list based on the member change log after its own data log processing progress reaches the data log processing progress corresponding to the position identifier information; or, the slave node may send a corresponding rejection message to the primary slave node to indicate that the slave node rejects the synchronization of the member change log initiated by the primary slave node.

[0066] Corresponding to the embodiments of the foregoing methods, this specification also provides embodiments of apparatus, electronic devices, and storage media.

[0067] Figure 7 This is a schematic structural diagram of an electronic device provided in an exemplary embodiment. Please refer to... Figure 7 At the hardware level, the device includes a processor 701, a network interface 702, memory 703, non-volatile memory 704, and an internal bus 705, and may also include other hardware required for business operations. One or more embodiments of this specification can be implemented in software, such as the processor 701 reading the corresponding computer program from the non-volatile memory 704 into memory 703 and then running it. Of course, in addition to software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0068] Figure 8 This is a block diagram of a synchronization device for member changes in a distributed database system, provided in an exemplary embodiment. Please refer to... Figure 8The distributed database system may include a first replica group, which includes several members whose roles are primary replica node and secondary replica node. The primary replica node and the secondary replica node maintain a member list for recording the members in the first replica group, which is applied to the primary replica node. The device includes:

[0069] The generation unit 802 is used to generate a corresponding member change log and update the member list it maintains when a member change request is received. The member change log and the data log belong to different log streams.

[0070] The determining unit 804 is used to synchronize the member change log to the slave replica node, and determine whether the member change log synchronization is complete based on the confirmation information fed back by the slave replica node.

[0071] Optionally, the determining unit 804 is specifically used for:

[0072] The primary replica node determines that the member change log synchronization is complete upon receiving confirmation from more than half of the members corresponding to the updated member list it maintains.

[0073] Optionally, the device further includes:

[0074] The checking unit 806 is configured to, upon receiving a member change request, have the primary replica node check whether there is an ongoing member change in the first replica group; if the checking result indicates that there is no ongoing member change in the first replica group, generate a corresponding member change log; or, if the checking result indicates that there is an ongoing member change in the first replica group, have the primary replica node wait for the ongoing member change to complete before generating a corresponding member change log; or, have the primary replica node reject the member change request.

[0075] Optionally, the member change log includes site identifier information, which is used to characterize the data log processing progress of the primary replica node when generating the member change log.

[0076] Optionally, the system further includes a second replica group, which is a backup replica group of the first replica group, and the apparatus further includes:

[0077] Synchronization unit 808 is used to generate a corresponding data log in response to a data change request; and synchronize the data log to the second replica group so that the second replica group can complete the data synchronization based on the data log.

[0078] Figure 9This is a block diagram of a synchronization device for member changes in a distributed database system, provided in an exemplary embodiment. Please refer to... Figure 9 The distributed database system includes a first replica group, which comprises several members, whose roles are primary replica node and secondary replica node. The primary replica node and the secondary replica node maintain a member list for recording the members in the first replica group, which is applied to the secondary replica node. The device includes:

[0079] The update unit 902 is used to update the member list it maintains based on the member change log after receiving the member change log sent by the primary replica node. The member change log and the data log belong to different log streams. The member change log is generated by the primary replica node after receiving the member change request.

[0080] The sending unit 904 is used to send corresponding confirmation information to the primary replica node when the member list update is completed, so that the primary replica node can determine whether the member change log synchronization is completed based on the confirmation information fed back by the secondary replica node.

[0081] Optionally, the member change log includes site identifier information, which is used to characterize the data log processing progress of the primary replica node when generating the member change log. The device further includes:

[0082] The location checking unit 906 is used to check, upon receiving the member change log, whether the data log processing progress corresponding to the slave replica node is consistent with that of the master replica node based on the location identifier information in the member change log;

[0083] If the inspection results indicate that the data log processing progress of the slave replica node is consistent with that of the primary replica node, then update the member list maintained by the replica node based on the member change log; or...

[0084] If the inspection results indicate that the data log processing progress of the slave replica node has not reached the data log processing progress of the primary replica node, the slave replica node, after its own data log processing progress reaches the data log processing progress corresponding to the position identifier information, updates the member list it maintains based on the member change log; or, it sends a corresponding rejection message to the primary replica node to indicate that the slave replica node rejects the synchronization of the member change log initiated by the primary replica node.

[0085] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the solution in this specification according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0086] In a typical configuration, a computer device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0087] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0088] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0089] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0090] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0091] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0092] It should be understood that although the terms first, second, third, etc., may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0093] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

Claims

1. A distributed database system, comprising a first replica group, the first replica group comprising a plurality of members, the members being primary replica nodes and secondary replica nodes, the primary replica nodes and secondary replica nodes maintaining a member list for recording the members in the first replica group; The primary replica node is used to generate a corresponding member change log and update its own maintained member list when it receives a member change request. The member change log and the data log belong to different log streams. It also synchronizes the member change log to the secondary replica node and determines whether the member change log synchronization is complete based on the confirmation information fed back by the secondary replica node. The slave replica node is configured to update its own maintained member list based on the member change log sent by the primary replica node after receiving the member change log; and, upon completion of the member list update, send corresponding confirmation information to the primary replica node, wherein: The member change log includes site identifier information, which is used to characterize the persistence site of the data log stream corresponding to the primary replica node when generating the member change log. The secondary replica node is further used for: Upon receiving the member change log, check whether the data log processing progress of the slave replica node is consistent with that of the master replica node based on the location identifier information in the member change log; If the inspection results indicate that the data log processing progress of the slave replica node is consistent with that of the primary replica node, then update the member list maintained by the replica node based on the member change log; or... If the inspection results indicate that the data log processing progress of the slave replica node has not reached the data log processing progress of the primary replica node, the slave replica node updates its maintained member list based on the member change log after its own data log processing progress reaches the data log processing progress corresponding to the position identifier information; or, the slave replica node sends a corresponding rejection message to the primary replica node to indicate that the slave replica node rejects this member change log synchronization initiated by the primary replica node.

2. The system according to claim 1, wherein determining whether the member change log synchronization is complete based on the confirmation information fed back from the replica node includes: The primary replica node determines that the member change log synchronization is complete upon receiving confirmation from more than half of the members corresponding to the updated member list it maintains.

3. The system according to claim 1, wherein the primary replica node is further configured to: Upon receiving a member change request, the primary replica node checks whether there is an ongoing member change in the first replica group; If the inspection results indicate that there are no ongoing member changes in the first replica group, a corresponding member change log is generated. or, If the inspection results indicate that there is an ongoing member change in the first replica group, the primary replica node waits for the ongoing member change to complete before generating the corresponding member change log; or, the primary replica node rejects the member change request.

4. The system according to claim 1, further comprising a second replica group, the second replica group being a backup replica group of the first replica group, the primary replica node further configured to: In response to data change requests, generate corresponding data logs; The data logs are synchronized to the second replica group so that the second replica group can complete the data synchronization based on the data logs.

5. A method for synchronizing member changes in a distributed database system, the distributed database system comprising a first replica group, the first replica group comprising several members, the members being a primary replica node and a secondary replica node, the primary replica node and the secondary replica node maintaining a member list for recording members in the first replica group, applied to the primary replica node, the method comprising: Upon receiving a member change request, the system generates a corresponding member change log and updates its own maintained member list. The member change log and the data log belong to different log streams. The member change log is synchronized to the slave replica node, and the synchronization of the member change log is determined based on the confirmation information returned by the slave replica node. The member change log includes position identifier information, which characterizes the data log processing progress corresponding to the generation of the member change log by the primary replica node. Synchronizing the member change log to the slave replica node includes: The member change log is synchronized to the slave replica node, so that when the slave replica node receives the member change log, it checks whether the data log processing progress corresponding to the slave replica node is consistent with that of the primary replica node based on the position identifier information in the member change log; if the check result shows that the data log processing progress corresponding to the slave replica node is consistent with that of the primary replica node, the slave replica node updates its own maintained member list based on the member change log; or, if the check result shows that the data log processing progress corresponding to the slave replica node has not reached that of the primary replica node, the slave replica node updates its own maintained member list based on the member change log after its own data log processing progress reaches that of the data log processing progress corresponding to the position identifier information; or, the slave replica node sends a corresponding rejection message to the primary replica node to indicate that the slave replica node rejects this member change log synchronization initiated by the primary replica node.

6. The method according to claim 5, wherein determining whether the member change log synchronization is complete based on the confirmation information fed back from the replica node includes: The primary replica node determines that the member change log synchronization is complete upon receiving confirmation from more than half of the members corresponding to the updated member list it maintains.

7. The method according to claim 5, further comprising: Upon receiving a member change request, the primary replica node checks whether there is an ongoing member change in the first replica group; If the inspection results indicate that there are no ongoing member changes in the first replica group, a corresponding member change log is generated. or, If the inspection results indicate that there is an ongoing member change in the first replica group, the primary replica node waits for the ongoing member change to complete before generating the corresponding member change log; or, the primary replica node rejects the member change request.

8. The method according to claim 5, wherein the system further comprises a second replica group, the second replica group being a backup replica group of the first replica group, and the method further comprises: In response to data change requests, generate corresponding data logs; The data logs are synchronized to the second replica group so that the second replica group can complete the data synchronization based on the data logs.

9. A method for synchronizing member changes in a distributed database system, the distributed database system comprising a first replica group, the first replica group comprising several members, the members having the roles of primary replica node and secondary replica node, the primary replica node and the secondary replica node maintaining a member list for recording members in the first replica group, applied to the secondary replica node, the method comprising: After receiving the member change log sent by the primary replica node, the primary replica node updates its own maintained member list based on the member change log. The member change log and the data log belong to different log streams. The member change log is generated by the primary replica node after receiving the member change request. Upon completion of the member list update, a corresponding confirmation message is sent to the primary replica node, enabling the primary replica node to determine whether the member change log synchronization is complete based on the confirmation message from the secondary replica node. The member change log includes position identifier information, which characterizes the data log processing progress of the primary replica node when generating the member change log. The method further includes: Upon receiving the member change log, check whether the data log processing progress of the slave replica node is consistent with that of the master replica node based on the location identifier information in the member change log; If the inspection results indicate that the data log processing progress of the slave replica node is consistent with that of the primary replica node, then update the member list maintained by the replica node based on the member change log; or... If the inspection results indicate that the data log processing progress of the slave replica node has not reached the data log processing progress of the primary replica node, the slave replica node, after its own data log processing progress reaches the data log processing progress corresponding to the position identifier information, updates the member list it maintains based on the member change log; or, it sends a corresponding rejection message to the primary replica node to indicate that the slave replica node rejects the synchronization of the member change log initiated by the primary replica node.

10. A synchronization device for member changes in a distributed database system, the distributed database system comprising a first replica group, the first replica group comprising a plurality of members, the members being a primary replica node and a secondary replica node, the primary replica node and the secondary replica node maintaining a member list for recording members in the first replica group, applied to the primary replica node, the device comprising: The generation unit is used to generate a corresponding member change log and update the member list it maintains when a member change request is received. The member change log and the data log belong to different log streams. A determining unit is configured to synchronize the member change log to the slave replica node and determine whether the member change log synchronization is complete based on the confirmation information returned by the slave replica node, wherein: the member change log includes position identifier information, the position identifier information being used to characterize the data log processing progress corresponding to the primary replica node when generating the member change log; synchronizing the member change log to the slave replica node includes: The member change log is synchronized to the slave replica node, so that when the slave replica node receives the member change log, it checks whether the data log processing progress corresponding to the slave replica node is consistent with that of the primary replica node based on the position identifier information in the member change log; if the check result shows that the data log processing progress corresponding to the slave replica node is consistent with that of the primary replica node, the slave replica node updates its own maintained member list based on the member change log; or, if the check result shows that the data log processing progress corresponding to the slave replica node has not reached that of the primary replica node, the slave replica node updates its own maintained member list based on the member change log after its own data log processing progress reaches that of the data log processing progress corresponding to the position identifier information; or, the slave replica node sends a corresponding rejection message to the primary replica node to indicate that the slave replica node rejects this member change log synchronization initiated by the primary replica node.

11. A synchronization device for member changes in a distributed database system, the distributed database system comprising a first replica group, the first replica group comprising a plurality of members, the members being a primary replica node and a secondary replica node, the primary replica node and the secondary replica node maintaining a member list for recording members in the first replica group, applied to the secondary replica node, the device comprising: The update unit is used to update the member list it maintains based on the member change log after receiving the member change log sent by the primary replica node. The member change log and the data log belong to different log streams. The member change log is generated by the primary replica node after receiving the member change request. The sending unit is configured to send corresponding confirmation information to the primary replica node after the member list update is completed, so that the primary replica node can determine whether the member change log synchronization is complete based on the confirmation information fed back by the secondary replica node. The member change log includes position identifier information, which is used to characterize the data log processing progress corresponding to the primary replica node when generating the member change log. The sending unit is further configured to: Upon receiving the member change log, check whether the data log processing progress of the slave replica node is consistent with that of the master replica node based on the location identifier information in the member change log; If the check result indicates that the data log processing progress of the slave replica node is consistent with that of the primary replica node, the slave replica node updates its own maintained member list based on the member change log; or, if the check result indicates that the data log processing progress of the slave replica node has not reached that of the primary replica node, the slave replica node updates its own maintained member list based on the member change log after its own data log processing progress reaches that of the location identifier information; or, the slave replica node sends a corresponding rejection message to the primary replica node to indicate that it rejects the synchronization of the member change log initiated by the primary replica node.

12. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method as described in any one of claims 5 to 9.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps of the method as described in any one of claims 5 to 9.

Images