Data synchronization method and device, computing equipment and service system

Abstract

The invention discloses a data synchronization method and device, computing equipment and a service system, and belongs to the technical field of computers. The method is applied to a service system comprising a first node and a second node after split brain recovery, the first node serving as a main node is used for providing read-write service of a first database, and the second node serving as a standby node maintains a second database. The method comprises the steps that a first node obtains a target data page and a first redo log and sends the target data page and the first redo log to a second node, so that synchronization of a second database and a first database is achieved. Wherein the target data page comprises data pages which are changed for multiple times in the first database, the first redo log comprises a log provided with a skip mark, and the skip mark is used for indicating that the log provided with the skip mark is skipped when the first redo log is played back. According to the method, the synchronization of the standby node database and the main node database of the service system can be quickly completed after the split brain of the service system is recovered.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to a data synchronization method, apparatus, computing device, and service system. Background Art

[0002] A dual-active system includes a primary node and a standby node. In a database system implemented through the dual-active system architecture, the primary node and the standby node each maintain a database, and the databases maintained by the primary node and the standby node are the same. Among them, the primary node is responsible for providing read and write services for the database, and the standby node is responsible for disaster recovery. During the process of the primary node providing read and write services for the database, the standby node keeps its own database synchronized with the primary node's database through the redo log recorded by the primary node.

[0003] However, after a split-brain occurs between the primary node and the standby node, the primary node and the standby node become two independent nodes due to the loss of connection and will simultaneously provide read and write services for the database, which will cause data chaos and desynchronization between the databases maintained by the primary node and the standby node. Summary of the Invention

[0004] This application provides a data synchronization method, apparatus, computing device, and service system, which can quickly complete the synchronization of the standby node database and the primary node database of the service system after the split-brain recovery of the service system.

[0005] The technical solutions provided by this application are as follows:

[0006] In a first aspect, the present application provides a data synchronization method, which is applied to a service system. After split-brain recovery, the service system includes a first node and a second node. Among them, the first node, as the primary node, is used to provide read and write services for a first database, and the second node, as the standby node, maintains a second database. This method is executed by the first node and includes: obtaining a target data page and a first redo log; sending the target data page and the first redo log to the second node to synchronize the second database and the first database. Wherein, the target data page includes the data page in the first database indicated by a first page identifier (identifier, ID), and the first page ID is the page ID of the data page in the first database that has been changed multiple times. The multiple changes refer to the number of times of performing change operations on any data page in the first database after split-brain occurs in the service system, and this threshold is a positive integer. The first redo log is used to record the change operations performed on the first database after split-brain occurs in the service system. The first redo log includes logs with skip marks set, and the logs with skip marks set are used to record the change operations on the data pages that have been changed multiple times. The skip mark is used to indicate skipping the logs with skip marks set when replaying the first redo log.

[0007] Through the method provided by the present application, after split-brain recovery of the service system, the second node, as the standby node, updates the second database maintained by the second node after split-brain recovery by using the latest version of the "data pages with multiple changes" in the first database maintained by the first node. Among them, the "data pages with multiple changes" in the first database refer to the data pages in the first database whose number of change operations is greater than the threshold after split-brain occurs in the service system, that is, the target data pages. And, when replaying the redo log recorded by the first node after split-brain occurs in the service system at the second node, the logs with skip marks set in the redo log are skipped, that is, the logs used to record the change operations of the "data pages with multiple changes" in the first database are skipped. In this way, the efficiency of replaying the redo log at the second node can be improved, and the input / output (IO) overhead generated when the second node replays the redo log can be reduced. Furthermore, the synchronization efficiency when synchronizing the second database in the second node and the first database in the first node based on replaying the redo log is improved, and the user is unaware.

[0008] In a possible design, the above-mentioned target data page further includes the data page in the first database indicated by a second page ID, and the second page ID is the page ID of the data page on which change operations have been performed in the second database during split-brain of the service system. Before obtaining the target data page, the above method further includes: receiving the second page ID sent by the second node. The obtaining of the target data page includes: reading the target data page from the first database according to the second page ID and the first page ID.

[0009] Through this possible design, the purpose of synchronizing the data pages that have undergone change operations during the occurrence of a split-brain in the service system in the second database maintained by the second node is achieved.

[0010] In another possible design, the above-mentioned reading of the target data page from the first database according to the second page ID and the first page ID includes: performing an operation of merging and deduplicating the second page ID and the first page ID to obtain the page ID of the target data page; reading the target data page from the first database according to the page ID of the target data page.

[0011] Through this possible design, duplicate data pages in the target data page can be removed, thereby reducing the IO overhead required for the first node to read the target data page, reducing the network resources required for the first node to subsequently send the target data page, reducing the resources required for the second node to receive the target data page, and reducing the IO overhead required for the second node to update the second database according to the target data page, etc.

[0012] In yet another possible design, before obtaining the above-mentioned first redo log, the above method further includes: recording the first redo log according to the change operations performed by the first node on the first database after the occurrence of a split-brain in the service system; setting a skip flag for the first type of logs in the first redo log when the number of change operations performed by the first node on the first data page after the occurrence of a split-brain in the service system is greater than a threshold. Wherein, the first type of logs is used to record the change operations performed by the first node on the first data page after the occurrence of a split-brain in the service system, and the first data page is any data page in the first database.

[0013] Through this possible design, the purpose of setting a skip flag for the logs corresponding to the data pages whose change execution times are greater than the threshold is achieved during the process of recording the first redo log. In this way, when playing back the first redo log, the logs marked with the skip flag in the first redo log can be skipped, thereby avoiding repeatedly reading and writing the data pages that have undergone multiple change operations and consuming excessive data IO overhead.

[0014] In yet another possible design, before obtaining the above-mentioned target data page, the above method further includes: when the number of change operations performed by the first node on the first data page after the occurrence of a split-brain in the service system is greater than a threshold, adding the page ID of the first data page to a preset storage space to obtain the first page ID stored in the preset storage space. Wherein, the first data page is any data page in the first database.

[0015] Through this possible design method, it is possible to collect the page IDs of the data pages that have undergone multiple change operations during the split-brain of the service system.

[0016] In yet another possible design approach, adding the page ID of the first data page to the preset storage space includes: adding the page ID of the first data page to the preset storage space when the page ID of the first data page does not exist in the preset storage space.

[0017] Through this possible design approach, it is possible to deduplicate the page IDs of the data pages that have undergone multiple change operations during the brain split of the service system, thereby reducing the IO overhead required for the first node to read these data pages, reducing the network resources required for the first node to subsequently send these data pages, reducing the resources required for the second node to receive these data pages, and reducing the IO overhead required for the second node to update the second database based on these data pages, etc.

[0018] In yet another possible design approach, before adding the page ID of the first data page to the preset storage space, the method further includes: querying the value of the times counter in the first data page; when determining that the difference between the current value of the times counter and the initial value of the times counter is greater than or equal to a threshold, determining that the number of times the first node has performed change operations on the first data page after the brain split of the service system is greater than the threshold. Here, the times counter is used to record the number of times the first node has performed change operations on the first data page after the brain split of the service system.

[0019] Through this possible design approach, it is possible to determine the data pages for which the number of change operations is greater than the threshold.

[0020] In yet another possible design approach, the method further includes: receiving a synchronization request sent by the second node, where the synchronization request is used to request synchronization between the second database and the first database. The obtaining of the target data page and the first redo log includes: in response to the synchronization request, obtaining the target data page and the first redo log.

[0021] In yet another possible design approach, when the target data page further includes the data page indicated by the second page ID in the first database, the synchronization request includes the second page ID. Sending the second page ID and the synchronization request in one message can reduce the occupancy of network resources.

[0022] In yet another possible design approach, the above change operation includes at least one of the following operations: an operation to add data, an operation to delete data, or an operation to modify data.

[0023] In yet another possible design approach, the above service system is a dual - machine hot - standby system.

[0024] Second aspect, the present application provides a data synchronization method, which is applied to a service system. After split-brain recovery, the service system includes a first node and a second node. Among them, the first node as the primary node is used to provide read and write services for a first database, and the second node as the standby node maintains a second database. This method is executed by the second node, and the method includes: receiving a target data page and a first redo log sent by the first node; synchronizing the second database and the first database according to the target data page and the first redo log. Among them, the target data page includes the data page in the first database indicated by a first page ID, and the first page ID is the page ID of the data page in the first database that has been changed multiple times. The multiple changes refer to the number of times of performing change operations on any data page in the first database after split-brain occurs in the service system being greater than a threshold, and the threshold is a positive integer. The first redo log is used to record the change operations performed on the first database after split-brain occurs in the service system. The first redo log includes logs with skip marks set, and the logs with skip marks set are used to record the change operations on the data pages that have been changed multiple times. The skip mark is used to indicate skipping the logs with skip marks set when replaying the first redo log.

[0025] In another possible design, the above target data page further includes the data page in the first database indicated by a second page ID, and the second page ID is the page ID of the data page on which change operations have been performed in the second database during split-brain of the service system. Before receiving the target data page and the first redo log sent by the first node, the above method further includes: parsing a second redo log recorded by the second node during split-brain of the service system to obtain the second page ID; sending the second page ID to the first node. Among them, the second redo log is used to record the change operations performed on the second database during split-brain of the service system.

[0026] In another possible design, the above synchronizing the second database and the first database according to the target data page and the first redo log includes: updating the second database after split-brain recovery of the service system through the target data page; replaying the first redo log for the updated second database and skipping the logs with skip marks set in the first redo log to achieve synchronization between the second database and the first database after split-brain recovery of the service system. Or, replaying the first redo log for the second database after split-brain recovery of the service system and skipping the logs with skip marks set in the first redo log to obtain the second database updated based on the first redo log; updating the second database that has been updated based on the first redo log through the target data page to achieve synchronization between the second database and the first database after split-brain recovery of the service system.

[0027] In yet another possible design, synchronizing the second database and the first database according to the target data page and the first redo log includes: updating the second database before the occurrence of the service system split-brain through the target data page; replaying the first redo log for the updated second database and skipping the logs with skip marks set in the first redo log, so as to synchronize the second database before the occurrence of the service system split-brain and the first database after the split-brain recovery. Alternatively, replaying the first redo log for the second database before the occurrence of the service system split-brain and skipping the logs with skip marks set in the first redo log, so as to obtain the second database updated based on the first redo log; updating the second database updated based on the first redo log through the target data page, so as to synchronize the second database before the occurrence of the service system split-brain and the first database after the split-brain recovery of the service system.

[0028] In yet another possible design, before synchronizing the second database and the first database according to the target data page and the first redo log, the method further includes: parsing the second redo log recorded by the second node during the occurrence of the service system split-brain; according to the parsed second redo log, rolling back the data of the second database after the split-brain recovery of the service system to before the occurrence of the service system split-brain, so as to obtain the second database before the occurrence of the service system split-brain. Wherein, the second redo log is used to record all change operations performed on the second database during the split-brain of the service system.

[0029] In yet another possible design, before receiving the target data page and the first redo log sent by the first node, the method further includes: sending a synchronization request to the first node, and the synchronization request is used to request to synchronize the second database and the first database.

[0030] In yet another possible design, the change operation includes at least one of the following operations: an operation of adding data, an operation of deleting data, or an operation of modifying data.

[0031] In yet another possible design, the service system is a dual-active hot standby system.

[0032] It should be understood that for the description of the beneficial effects of the technical solutions provided in the second aspect and any possible design in the second aspect, reference can be made to the description of the beneficial effects of the technical solutions provided in the first aspect and the corresponding design in the first aspect, and details will not be repeated here.

[0033] In a third aspect, the present application provides a data synchronization device. The data synchronization device is used to execute any one of the methods provided in the first aspect or the second aspect above. The present application can divide the functional modules of the data synchronization device according to any one of the methods provided in the first aspect or the second aspect above. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. Exemplarily, the present application can divide the data synchronization device into a processing unit, a transceiver unit, etc. according to functions. The descriptions of the possible technical solutions and beneficial effects executed by each of the above-divided functional modules can refer to the solutions provided in the first aspect and any possible design manner in the first aspect, or can refer to the solutions provided in the second aspect and any possible design manner in the second aspect, which will not be elaborated here.

[0034] In a fourth aspect, the present application provides a computing device. The computing device includes: a memory, a network interface, and one or more processors. Among them, the one or more processors receive or send data through the network interface, and the one or more processors are configured to read program instructions stored in the memory to execute the method provided in the first aspect and any possible design manner in the first aspect, or execute the method provided in the second aspect and any possible design manner in the second aspect.

[0035] In a fifth aspect, the present application provides a service system. After split-brain recovery, the service system includes a first node and a second node. The first node, as the primary node, is used to provide read and write services for the first database, and the second node, as the standby node, maintains a second database. Among them, the first node is used to execute the method provided in the first aspect and any possible design manner in the first aspect, and the second node is used to execute the method provided in the second aspect and any possible design manner in the second aspect to achieve the synchronization of the second database and the first database.

[0036] In a sixth aspect, the present application provides a computer program product containing instructions. When the instructions run on a computing device or a processor, the computing device or the processor is enabled to execute the method provided in the first aspect and any possible design manner in the first aspect, or execute the method provided in the second aspect and any possible design manner in the second aspect.

[0037] In a seventh aspect, the present application provides a computer-readable storage medium, which is a non-volatile computer-readable storage medium. The computer-readable storage medium includes computer program instructions. When the computer program instructions are executed by a computing device or a processor, the computing device or the processor executes the methods provided in the first aspect and any possible design manner in the first aspect, or executes the methods provided in the second aspect and any possible design manner in the second aspect.

[0038] In an eighth aspect, the present application provides a chip, which includes a processor for running program instructions or code. The chip or a device including the chip can be used to execute the methods provided in the first aspect and any possible design manner in the first aspect, or execute the methods provided in the second aspect and any possible design manner in the second aspect. Exemplarily, the chip further includes: an input interface, an output interface, and a memory. Among them, the input interface, output interface, processor, and memory of the chip are connected through an internal connection path of the chip. The memory in the chip is used to store program instructions or code run by the processor, and the input interface and output interface of the chip are used for connection communication between the chip and other chips or devices.

[0039] It can be understood that any of the above-provided data synchronization devices, computing devices, service systems, computer-readable storage media, computer program products, chips, etc. can be applied to the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods, which will not be elaborated here.

[0040] In the present application, the names of the above data synchronization devices and service systems do not constitute a limitation on the devices or functional modules themselves. In actual implementation, these devices or functional modules may appear under other names. As long as the functions of each device or functional module are similar to those of the present application, they all fall within the protection scope of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a schematic diagram of two implementation manners of master-slave baseline reconstruction after split-brain recovery in a database dual-machine hot standby system;

[0042] Figure 2 is a schematic diagram of an application scenario of the method provided in an embodiment of the present application;

[0043] Figure 3 is a schematic flowchart of a data synchronization method provided in an embodiment of the present application;

[0044] Figure 4 is a schematic diagram of a process of recording a first redo log provided in an embodiment of the present application;

[0045] Figure 5 It is a schematic diagram of a process for collecting the first page ID provided by an embodiment of the present application;

[0046] Figure 6 It is a schematic diagram of flushing dirty data pages and a page ID set to disk provided by an embodiment of the present application;

[0047] Figure 7 It is another schematic flowchart of the data synchronization method provided by an embodiment of the present application;

[0048] Figure 8 It is a schematic structural diagram of a data synchronization device provided by an embodiment of the present application;

[0049] Figure 9 It is a schematic structural diagram of another data synchronization device provided by an embodiment of the present application;

[0050] Figure 10 It is a schematic structural diagram of yet another data synchronization device provided by an embodiment of the present application;

[0051] Figure 11 It is a schematic structural diagram of a computing device provided by an embodiment of the present application. Detailed implementation manners

[0052] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0053] For ease of understanding, the technologies and backgrounds involved in the embodiments of the present application will be explained first below.

[0054] 1) Dual - machine hot - standby system

[0055] The dual - machine hot - standby system includes two nodes that are backups of each other and communicate through a network connection. These two nodes are respectively the primary node in the working state (active) and the standby node in the monitoring state (standby). Among them, the primary node is responsible for providing services to users, and the standby node is responsible for disaster recovery backup. In some examples, the network used for communication between the primary node and the standby node in the dual - machine hot - standby system can be a Transmission Control Protocol (TCP) / Internet Protocol (IP) network, but is not limited thereto.

[0056] In one case, the primary node and the standby node of the dual - machine hot - standby system share a storage space. In this case, the data of both the primary node and the standby node of the dual - machine hot - standby system are stored in this shared storage space. Since there is only one copy of the data stored in the shared storage space for both the primary node and the standby node, there is no data synchronization problem between the primary node and the standby node of the dual - machine hot - standby system in this case.

[0057] In another case, the primary node and the standby node of the dual-machine hot standby system are each configured with an independent storage space, which is generally a storage space composed of non-volatile storage media, such as a mechanical hard disk, a solid-state drive, etc. In this case, the data of the primary node in the dual-machine hot standby system is stored in the storage space of the primary node, and the data of the standby node is stored in the storage space of the standby node. Therefore, data synchronization needs to be considered between the primary node and the standby node of the dual-machine hot standby system. For example, in a database system implemented using the dual-machine hot standby system architecture (abbreviated as "database dual-machine hot standby system"), the primary node maintains a database in its own storage space, and the standby node also maintains a database in its own storage space, and the standby node needs to keep the database it maintains synchronized with the database maintained by the primary node.

[0058] Taking the example that the primary node and the standby node of the dual-machine hot standby system are each configured with an independent storage space, under normal circumstances, within the same time, the primary node in the dual-machine hot standby system is responsible for providing services to users, and the standby node maintains data synchronization with the primary node through the network between it and the primary node. In this way, when the primary node fails, the original standby node will be upgraded to a new primary node to provide services to users, which can ensure the continuity and reliability of the services. Since the working mechanism of the dual-machine hot standby system actually provides a fault self-recovery ability, the dual-machine hot standby system is a common solution to improve the high availability (HA) of network devices (such as servers).

[0059] As an example, in the database dual-machine hot standby system, the primary node provides read and write services for the database to users based on the database it maintains, and the standby node only provides read services for the database to users, and maintains the synchronization of the database it maintains with the database maintained by the primary node through network communication between itself and the primary node. Moreover, when the primary node fails or the primary node actively degrades to a standby node, the original standby node will be upgraded to a new primary node to provide read and write services for the database to users based on the database it maintains.

[0060] Among them, providing read and write services for the database to users includes providing read services and write services for the database to users. Providing read services for the database to users means: receiving a database read request initiated by a user, and in response to the received read request, reading the data in the database and returning it to the user. Providing write services for the database to users means: receiving a database write request initiated by a user, and in response to the received write request, performing corresponding write operations on the database. It should be understood that the write operations performed on the database are also referred to as change operations hereinafter, and generally include at least one of the operations of adding data, deleting data, or modifying data.

[0061] 2) Redo Log

[0062] A redo log is a mechanism in a database management system. The redo log is used to record all change operations performed on the database. Redo logs are mainly used for database backup, recovery, or master - standby synchronization, etc. Among them, one change operation performed on the database corresponds to one redo log. Here, one change operation performed on the database can include at least one of the operations of adding data, deleting data, or modifying data.

[0063] Exemplarily, in a database dual - active hot - standby system, after the primary node performs corresponding change operations on the database in response to the received data write request, it will write a redo log for recording this change operation in the redo log file and send the recorded redo log to the standby node in real - time, so that the standby node can keep the database it maintains synchronized with the database maintained by the primary node according to the redo log received from the primary node.

[0064] 3) Replay Log

[0065] A replay log, also known as a playback log, refers to the process of executing / applying the change operations recorded in the redo log to the operation object (such as a database).

[0066] For example, after the host maintaining the database crashes, some data in the database may have been changed in the host memory / cache but not written to disk. This part of the data will be lost with the host crash. In this case, after the host is restored, the change operations recorded in the redo log can be applied to the database to achieve the recovery of the lost data. Among them, data not written to disk means that the data exists in memory / cache but has not been written to the disk composed of non - volatile storage media.

[0067] Another example is that in a database dual - active hot - standby system, the standby node will receive the redo log sent by the primary node. In this way, the standby node can apply the change operations recorded in the redo log received from the primary node to the database it maintains, so as to achieve the synchronization of the standby node database and the primary node database. It can be understood that in a database dual - active hot - standby system, the primary node can send the redo log to the standby machine in real - time so that the standby node can replay the redo log to achieve the consistency between the standby node database and the primary node database. Therefore, under normal circumstances, the redo logs in the primary node and the standby node in the database dual - active hot - standby system are exactly the same. In this case, it is called the primary - standby homology in the database dual - active hot - standby system.

[0068] 4) Split - brain

[0069] Brain split is a computer term, specifically referring to the system split in a cluster system caused by network unreachability between nodes. After the split, the small clusters will provide services according to their respective states, resulting in chaos in the originally integrated cluster. Network unreachability includes communication link anomalies or node anomalies, etc., not limited to this.

[0070] Taking the highly available database dual - active hot - standby system as an example, in this system, when the master node and the standby node cannot receive each other's heartbeat messages, both the master node and the standby node will determine that the other party has failed / anomalous. As a result, in this system, in addition to the original master node (referred to as the "original master node") providing read - write services of the database to users, the original standby node (referred to as the original standby node) will be upgraded to a new master node (referred to as the "new master node") and start providing read - write services of the database to users. That is to say, at this time, there are two master nodes (the original master node and the new master node) in the database dual - active hot - standby system. In this case, it is called that the database dual - active hot - standby system has experienced a brain split. Since both the original master node and the new master node of the system provide read - write services of the database to users after the database dual - active hot - standby system has a brain split, it will lead to the inconsistency between the original master node database and the new master node (original standby node) database.

[0071] Among them, the heartbeat message refers to the message (packet) used by network devices in the heartbeat detection mechanism to notify the other party that it is still online / alive. The embodiments of this application do not elaborate on the heartbeat detection mechanism.

[0072] Among them, before the database dual - active hot - standby system has a brain split, the sequence number of the last redo log reflecting the same origin of the master and standby in this system is called the log fork point. It should be understood that after the log fork point, the database dual - active hot - standby system has a brain split, and then the original master node and the new master node in the system each provide read - write services of the database to users and each record redo logs. That is to say, after the log fork point, the redo logs of the original master node and the new master node in the database dual - active hot - standby system are not of the same origin.

[0073] In addition, in the database dual - active hot - standby system that has experienced a brain split, when there are two master nodes, namely the first master node and the second master node, in the system that has a brain split, when any one of the first master node and the second master node is downgraded to a standby node (denoted as the "new standby node"), it means that the brain split of the database dual - active hot - standby system has been restored. That is, after the brain split of the system is restored, there is one master node and one standby node in the system. At this time, the new standby node will no longer provide write services of the database to users, but only provide read services of the database to users. And when the first master node is downgraded to the new standby node, the new standby node needs to synchronize the database it maintains with the database of the second master node to achieve its disaster - tolerance purpose.

[0074] It should be understood that after the split-brain recovery of the database dual-active hot standby system, the process of data synchronization between the standby node and the primary node in the system is called primary-standby baseline reconstruction. Specifically, primary-standby baseline reconstruction means that when the redo logs of the primary node and the standby node in the database dual-active hot standby system are not from the same source (such as non-same source caused by split-brain), resulting in data inconsistency between the primary node and the standby node, the redo logs of the primary node are replayed on the standby node in a full or incremental manner to make the redo logs of the primary node and the standby node finally from the same source, and then to make the data of the primary node and the standby node finally consistent.

[0075] 5), Recovery Time Objective (RTO)

[0076] RTO is the length of time that service interruption is allowed when providing services to users. For example, when the data read and write service provided by the database dual-active hot standby system to users is interrupted and the interruption needs to be recovered within half a day, it means that the RTO index of this system is 12 hours.

[0077] It should be understood that the smaller the time required for the service to recover from interruption to normal (i.e., RTO), the stronger the data recovery ability of the system and the stronger the disaster tolerance ability of the system.

[0078] In the current technology, in the scenario where the primary node and the standby node of the database dual-active hot standby system are each configured with an independent storage space, and the primary node and the standby node respectively maintain a database in their respective storage spaces, when a split-brain occurs in this system, after the split-brain recovery of this system, the efficiency of primary-standby baseline reconstruction in the system is relatively low. Refer to Figure 1 , Figure 1 shows a schematic diagram of two implementation methods of primary-standby baseline reconstruction after the split-brain recovery of the database dual-active hot standby system.

[0079] As Figure 1 shown, after the split-brain recovery of the database dual-active hot standby system, it includes a new primary node and a new standby node with a primary-standby relationship. Among them, the new primary node provides the read and write services of the database for users, and the new standby node only provides the read service of the database for users. That is to say, the new primary node can handle the read and write requests from the client, while the new standby node can only handle the read requests from the client.

[0080] Please refer to Figure 1In (a) among them, after the split-brain recovery of the database dual-active hot standby system, the new standby node in the system first determines the data pages on which change operations have been performed in the database maintained by itself during the split-brain occurrence (denoted as differential data pages), and sends the page identifiers (IDs) of the differential data pages to the new primary node. In response, the new primary node reads the corresponding differential data pages from the database maintained by itself according to the received page IDs, and sends the read differential data pages and the redo logs recorded by the new primary node during the split-brain occurrence to the new standby node. Then, the new standby node writes the received differential data pages into the database maintained by itself to overwrite the original differential data pages in the database. Moreover, the new standby node also replays the redo logs received from the new primary node based on the database maintained by itself after the split-brain recovery, so as to synchronize the database maintained by the standby node after the split-brain occurrence to the database maintained by the new primary node after the split-brain occurrence, thereby completing the purpose of reconstructing the primary-standby baseline after the split-brain recovery of the system.

[0081] Among them, the page ID of the data page is used to identify the data page in the database. Optionally, the page IDs of the data pages in the database are generally consecutive numerical numbers, but are not limited thereto. In some examples, the page ID of the data page in the database consists of a file number and a page number. The file number refers to the number of the file to which the data page belongs in the database, and the page number refers to the page number of the physical page carrying the data page.

[0082] Please refer to Figure 1 In (b) among them, after the split-brain recovery of the database dual-active hot standby system, the new primary node in the system sends the redo logs recorded by the new primary node during the split-brain occurrence to the new standby node. The new standby node first rolls back (or reverts) the database maintained by itself to the state before the split-brain occurrence according to the redo logs recorded by itself during the split-brain occurrence, so as to obtain the database maintained by the standby node before the split-brain occurrence. Then, the new standby node replays the redo logs received from the primary node based on the database maintained by itself before the split-brain occurrence, so as to synchronize the database maintained by itself before the split-brain occurrence to the database maintained by the new primary node after the split-brain occurrence, thereby completing the purpose of reconstructing the primary-standby baseline after the split-brain recovery of the system.

[0083] However, through Figure 1When the method shown is used to perform the primary and standby baseline reconstruction after split-brain recovery, the efficiency of the primary and standby baseline reconstruction depends on the efficiency of the new standby node in replaying the redo log recorded by the new primary node. The efficiency of replaying the redo log depends on the number of data pages in the database maintained by the new primary node during the split-brain occurrence and the number of change operations performed on the same data page. This is because reading and writing each data page incurs corresponding input / output (IO) overhead. Therefore, the more times the same data page is modified, the more redo log records are generated, resulting in more IO for reading and writing data pages during the replay of the redo log, thus leading to a lower efficiency in replaying the redo log and a longer time required for the primary and standby baseline reconstruction. For a highly available system, the longer the time required for the primary and standby baseline reconstruction, the larger the RTO of the system, and an excessive RTO will affect the user experience.

[0084] Based on this, an embodiment of the present application provides a data synchronization method, which is applied to a service system. This method can quickly complete the synchronization of the standby node database and the primary node database of the service system after the split-brain recovery of the service system. That is, this method can quickly complete the primary and standby baseline reconstruction after the split-brain recovery of the service system.

[0085] Among them, the service system can be a database dual-active hot standby system including one primary node and one standby node, or a database service system including one primary node and multiple standby nodes, and there is no limitation in this regard. For simplicity of description, in the following, the service system is described as an example of a database dual-active hot standby system with one primary and one standby.

[0086] Reference Figure 2 , Figure 2 shows a schematic diagram of the application scenario of the method provided by the embodiment of the present application.

[0087] As Figure 2 shown, before the split-brain occurs in the service system 20, it includes an original standby node and an original primary node. The original standby node maintains a first database, and the original primary node maintains a second database. Among them, the original primary node is used to provide read and write services for the second database to users, and the original standby node is used to provide read services for the first database to users, and synchronize the first database maintained by itself with the second database maintained by the original primary node according to the redo log received from the original primary node in real time.

[0088] Continue to refer to Figure 2, when a brain split occurs in the service system 20 at time t1, the service system 20 includes two primary nodes during the brain split, namely the new primary node of the service system 20 (denoted as the first node) and the original primary node (denoted as the second node). Specifically, when a brain split occurs in the service system 20 at time t1, it means that the original standby node of the service system 20 cannot detect the heartbeat packet of the original primary node. Therefore, the original standby node determines that the original primary node is abnormal and is upgraded to the new primary node of the service system 20 to provide read and write services for the first database to users. It should be understood that the reason why the original standby node cannot detect the heartbeat packet of the original primary node may be that the communication network between the original primary node and the original standby node is abnormal, or the original primary node has an abnormality (such as crashing), etc., and this is not limited. For example, when the communication network between the original primary node and the original standby node of the service system 20 is abnormal, the original primary node of the service system 20 itself is normal. Therefore, during the brain split of the service system 20, the original primary node will continue to maintain the working state, that is, the original primary node will continue to provide read and write services for the second database to users. Another example is that if the original primary node of the service system 20 crashes and then quickly restarts and recovers, during the brain split of the service system 20, after the original primary node recovers from the crash, it will continue to provide read and write services for the second database to users.

[0089] Among them, the embodiment of the present application does not elaborate on the process of the original standby node being upgraded to the new primary node. In one example, the service system 20 further includes a management node. When the original standby node cannot detect the heartbeat packet of the original primary node within a preset time period or a preset number of detection cycles, the original standby node sends a packet requesting to be upgraded to the new primary node to the management node. In response, the management node sends an instruction packet to the original standby node to instruct the original standby node to be upgraded to the new primary node.

[0090] Please continue to refer to Figure 2 , when the brain split of the service system 20 is restored at time t2, it means that one of the primary nodes during the brain split of the service system 20 is downgraded to a standby node. For example Figure 2 shown as the second node (i.e., the original primary node) is downgraded to a standby node (denoted as Figure 2 the new standby node shown). Therefore, after the brain split of the service system 20 is restored, the new primary node (i.e., the first node) continues to provide read and write services for the first database to users, while the new standby node (i.e., the second node) only provides read services for the second database to users and needs to synchronize the second database according to the first database. The solution provided by the embodiment of the present application is used to solve the problem of synchronizing the databases of the new standby node and the new primary node at this time.

[0091] It should be understood that the primary node that is downgraded to the new standby node after the brain split of the service system 20 can be any one of the primary nodes during the brain split of the service system 20, and the embodiment of the present application does not limit this. Figure 2Only taking the example that the original primary node in the service system 20 is downgraded to a new standby node after the split-brain recovery of the service system 20 for description.

[0092] It should be understood that the above content is an exemplary description of the application scenario of the data synchronization method provided by the embodiments of the present application, and does not constitute a limitation on the application scenario of the method. Those of ordinary skill in the art know that with the change of business requirements, its application scenario can be adjusted according to application requirements, and the embodiments of the present application do not list them one by one.

[0093] The embodiments of the present application further provide a data synchronization device, which can be applied to a service system and is used to execute the method provided by the embodiments of the present application to achieve the synchronization of the new standby node database and the new primary node database in the service system after the split-brain recovery. Among them, the service system is a database service system including a primary node and at least one standby node. Optionally, the data synchronization device can be the primary node or the standby node in the service system, or a functional module in the primary node or the standby node, which is not limited thereto. As an example, the primary node and the standby node can be two servers, but of course not limited thereto.

[0094] It should be understood that since the solution provided by the embodiments of the present application is used to solve the synchronization problem between the primary node database and the standby node database in the service system after the split-brain recovery, in the embodiments of the present application, the primary node and the standby node of the service system are each configured with an independent storage space, and their respective databases are respectively maintained in their respective storage spaces. Among them, the storage space configured by the primary node (or standby node) is a storage space composed of non-volatile storage media. Taking the primary node as an example, the storage space configured by the primary node can be a non-volatile storage medium disk (such as a mechanical hard disk, a solid-state drive, etc.) in the primary node, or the storage space configured by the primary node can be any storage device that the primary node can access, and the storage device can be an independent computer, etc., not limited thereto.

[0095] The implementation process of the data synchronization method provided by the embodiments of the present application will be described below.

[0096] Refer to Figure 3 , Figure 3 which shows a schematic flowchart of a data synchronization method provided by the embodiments of the present application. Optionally, this method is applied to the Figure 2 shown application scenario and is executed by the Figure 2 shown service system 20 for corresponding steps after a split-brain occurs in the service system 20. As Figure 3 shown, this method includes the following steps.

[0097] Step 101, the first node obtains a target data page and a first redo log.

[0098] Among them, the target data page includes the data page indicated by the first page ID in the first database. The first page ID is the page ID of the "data page with multiple changes" in the first database. The "data page with multiple changes" in the first database refers to a data page in which the number of change operations performed in the first database after the brain split of the service system is greater than the threshold. Alternatively, the aforementioned multiple changes can be understood as: after the brain split of the service system, the number of change operations performed on any data page in the first database is greater than the threshold. Among them, the threshold is a positive integer, and the specific value of the threshold is not limited in the embodiments of the present application. It should be understood that after the brain split of the service system, the first node may have performed change operations on at least one data page in the first database more than the threshold number of times. Therefore, the number of first page IDs is at least one, and thus multiple first page IDs can also be referred to as a page ID set.

[0099] The first redo log is used to record the change operations performed by the first node on the first database after the brain split of the service system. Alternatively, it can be understood that the first redo log is a redo log recorded after the log fork point. The description of the log fork point can be referred to above and will not be elaborated here. In one example, when the first node was the primary node in the system before the brain split of the service system, the first redo log is the redo log recorded after the log fork point among all the redo logs recorded by the first node. In another example, when the first node was a standby node in the system before the brain split of the service system, that is, the primary node in the system before the brain split of the service system was the second node, and the first node was upgraded to a new primary node after the brain split of the service system. In this case, the first redo log is all the redo logs recorded by the first node after receiving the last redo log (i.e., the log fork point) sent by the second node. This is because as a standby node, the first node only received the redo logs sent by the second node, which was the primary node, before the brain split of the service system and did not record redo logs by itself.

[0100] The first redo log includes logs with skip marks set. Here, the logs with skip marks set are used to record the change operations on the "data page with multiple changes", and the skip marks are used to indicate skipping the logs with skip marks set when replaying the first redo log. The embodiments of the present application do not specifically limit the implementation manner of the skip marks in the redo log. For example, the skip marks can be represented by predefined characters, but it is not limited thereto.

[0101] Optionally, after receiving the synchronization request sent by the second node, the first node obtains the target data page and the first redo log. Among them, the synchronization request is used to request the synchronization of the second database and the first database.

[0102] It can be understood that after the split-brain recovery of the service system, the second node is downgraded to a standby node. At this time, the second node needs to synchronize the second database maintained by itself with the database maintained by the first node. Therefore, when the second node determines that it is downgraded to a standby node (for example, receives an instruction from the management node to be downgraded to a standby node), the second node sends a synchronization request to the first node to synchronize the second database and the first database. In response, after receiving the synchronization request, the first node obtains the target data page and the first redo log in response to the synchronization request, so as to send the target data page and the first redo log to the second node, so that the second node synchronizes the second database and the first database according to the received target data page and the first redo log.

[0103] Specifically, after the split-brain of the service system occurs, during the process of the first node performing a change operation on the first database in response to a data write request from the client, the first redo log is recorded according to the performed change operation. Therefore, after receiving the synchronization request, the first node reads the recorded first redo log in response to the synchronization request, so as to obtain the first redo log. The process of the first node recording the first redo log refers to the description of steps 201 to 202 below and will not be elaborated here.

[0104] In addition, after the split-brain of the service system occurs, during the process of the first node performing a change operation on the first database in response to a data write request from the client, at least one of the above first page IDs is collected. The process of the first node collecting the first page ID refers to the description of steps 201 and 302 below and will not be elaborated here.

[0105] Therefore, in the first possible case, when the second node needs to synchronize the second database maintained by itself before the split-brain of the service system to the latest first database maintained by the first node after the split-brain recovery of the service system, the second node only sends the above synchronization request to the first node. In this case, after receiving the synchronization request, the first node reads the corresponding data page from the first database according to the first page ID in the page ID set in response to the synchronization request, so as to obtain the target data page. In this case, the target data page only includes the data page indicated by the first page ID in the first database, and the target data page is the latest version of the "data page with multiple changes" in the first database at the current moment.

[0106] As an example, when the page IDs of data pages in the first database are set with consecutive numbers, the first node can sort the first page ID in the set of page IDs in ascending or descending order and form a linked list, and then read the corresponding data pages from the first database one by one according to the page IDs in the linked list. Since data pages with adjacent or close page IDs may belong to the same file in the first database or be stored in the same block on the disk, when the first node reads the corresponding data pages from the first database in the order of the page IDs indicated by the linked list, data pages with adjacent or close page IDs may be read in one I / O operation. Therefore, when reading the target data pages in this way, the I / O overhead generated when reading the target data pages can be reduced.

[0107] In the second possible case, when the second node needs to synchronize the second database maintained by itself after the brain split of the service system to the latest first database maintained by the first node after the brain split of the service system, in addition to sending the above synchronization request to the first node, the second node also sends the second page ID to the first node. The second page ID is the page ID of the data page that has been changed in the second database during the brain split of the service system. Here, the change operation performed on the second database during the brain split of the service system refers to: during the brain split of the service system, the change operation performed by the second node in response to a data write request from the client on the second database. It should be understood that when the number of data pages that have been changed in the second database during the brain split of the service system is multiple, the number of second page IDs is also multiple. In some embodiments, the second node can obtain the second page ID according to the second redo log recorded by itself during the brain split of the service system. The second redo log is used to record the change operations performed by the second node on the second database during the brain split of the service system. In an example, the second node parses the second redo log to determine the data pages (denoted as differential data pages) that have been changed in the second database during the brain split of the service system, and reads the page IDs of the differential data pages. In this way, the second node can obtain the above second page ID. Further, after obtaining the second page ID, the second node sends the second page ID to the second node before, after, or at the same time as sending the above synchronization request to the first node. Of course, the second node can also add the obtained second page ID to the above synchronization request and then send the synchronization request carrying the second page ID to the first node. The embodiments of the present application do not limit this.

[0108] In this case, the second node receives a synchronization request and a second page ID, and in response to the synchronization request, reads the corresponding data page from the first database according to the first page ID in the page ID set and the received second page ID, so as to obtain the target data page. In this case, the target data page includes not only the data page indicated by the first page ID in the first database, but also the data page indicated by the second page ID in the first database. It should be understood that the target data page includes the latest version of the "data page with multiple changes" in the first database at the current moment, and also includes the latest version of the data page indicated by the second page ID in the first database.

[0109] Optionally, the second node can directly read the corresponding data page from the first database according to the first page ID in the page ID set and the received second page ID, so as to obtain the target data page. As an example, when the page IDs of the data pages in the first database are set with consecutive numbers, the first node can sort the first page ID and the second page ID in the page ID set in ascending or descending order and form a linked list, and then read the corresponding data pages from the first database one by one according to the page IDs in the linked list, which can reduce the IO overhead generated when reading the target data page.

[0110] Optionally, there may be duplicate page IDs in the set of page IDs containing at least one first page ID and the received second page ID. For example, there may be an intersection between the set of page IDs and the second page ID, or the set of page IDs includes the second page ID, or when the number of second page IDs is multiple, the set of page IDs may be a subset of the second page ID, and so on. To remove redundancy, the first node may perform a merge and deduplication operation on the first page ID in the set of page IDs and the received second page ID, so as to obtain the page ID of the target data page. For example, assume that the set of page IDs includes page IDs numbered 1, 7, 8, and 9, and the second page ID includes page IDs numbered 2, 6, 7, and 8. After the first node performs a merge and deduplication operation on the page IDs in the set of page IDs and the second page ID, page IDs numbered 1, 2, 6, 7, 8, and 9 are obtained, that is, the page IDs of the target data page include page IDs numbered 1, 2, 6, 7, 8, and 9. Then, the first node reads the corresponding data page from the first database according to the obtained page ID of the target data page, so as to obtain the target data page. As an example, when the page IDs of the data pages in the first database are set with consecutive numbers, the first node may sort the page IDs of the target data page obtained after performing the merge and deduplication operation in ascending or descending order and form a linked list, and then read the corresponding data page from the first database one by one according to the page IDs in the linked list, which can reduce the IO overhead generated when reading the target data page. It should be understood that there are no duplicate data pages in the obtained target data page, so the IO overhead required for the first node to read the target data page can be reduced, the network resources required for the first node to subsequently send the target data page can be reduced, the resources required for the second node to receive the target data page can be reduced, and the IO overhead required for the second node to update the second database according to the target data page can be reduced, etc.

[0111] Optionally, the synchronization request for requesting to synchronize the second database and the first database may also be sent from the management node of the service system to the first node, and this is not limited. For example, after the brain split recovery of the service system, after or at the same time when the management node instructs the second node to be downgraded to a standby node, a synchronization request for requesting to synchronize the second database and the first database is sent to the first node.

[0112] Step 102, the first node sends the target data page and the first redo log to the second node.

[0113] The first node sends the target data page and the first redo log to the second node through its own network interface / communication interface.

[0114] Optionally, the first node may send the target data page and the first redo log to the second node separately, or may send the target data page and the first redo log as a whole to the second node, and this is not limited.

[0115] It should be understood that since the second node can synchronize the second database and the current first database based on the target data page and the first redo log, thus solving the problem of inconsistency between the first database and the second database caused by split brain. Therefore, after the first node sends the target data page and the first redo log to the second node, when the first node subsequently records the corresponding redo log according to the change operations performed on the first database, the first node does not need to perform steps 201 to 202 described below to set skip marks for the redo log. Optionally, after sending the target data page and the first redo log to the second node, the first node can normally record the redo log and send the recorded redo log to the second node in real time, so that the second node can synchronize the second database and the first database in real time. This will not be elaborated further.

[0116] Step 103: The second node receives the target data page and the first redo log sent by the first node.

[0117] In response to step 102, the second node receives the target data page and the first redo log sent by the first node through its own network interface / communication interface.

[0118] Step 104: The second node synchronizes the second database and the first database according to the target data page and the first redo log.

[0119] In the first possible case, when the second node needs to synchronize the second database it maintained itself before the split brain of the service system to the latest first database maintained by the first node after the split brain recovery of the service system, the target data page received by the second node only includes the data page in the first database indicated by the first page ID. That is, the target data page received by the second node only includes the latest version of the "data page with multiple changes" in the first database when the first node sends the target data page.

[0120] In this case, before synchronizing the second database and the first database according to the target data page and the first redo log, the second node first parses the second redo log recorded by the second node during the split brain of the service system, and according to the parsed second redo log, rolls back the data of the second database maintained by the second node after the split brain recovery of the service system (hereinafter referred to as the second database after split brain recovery) to before the split brain of the service system, so as to obtain the second database maintained by the second node before the split brain of the service system (hereinafter referred to as the "second database before split brain"). As an example, the second node can perform the reverse operation of the change operations recorded in the second redo log on the second database after split brain recovery, so that the data of the second database after split brain recovery is rolled back to before the split brain of the service system to obtain the second database before split brain. The description of the second redo log can refer to the above description and will not be elaborated further.

[0121] Furthermore, in a possible implementation 1, the second node first updates the second database before split-brain using the target data page. The update operations performed by the second node on the second database before split-brain according to the target data page include an overwrite operation and / or a write operation. For example, when both the second database before split-brain and the target data page include a data page with a page ID of the first ID, the second node uses the data page with the page ID of the first ID in the target data page to overwrite the data page with the page ID of the first ID in the second database before split-brain. Another example is that when the target data page includes a data page with a page ID of the second ID and the second database before split-brain does not include a data page with a page ID of the second ID, the second node directly writes the data page with the page ID of the second ID in the target data page into the second database. It can be seen that through the overwrite operation, the multiple I / Os required to obtain the target data page in the second database by replaying multiple redo logs originally can be reduced to one I / O required for performing the overwrite operation. It can be seen that the solution of this application can reduce the I / O overhead of the second node.

[0122] Next, for the second database that has been updated according to the target data page, the second node replays the first redo log, and during the process of replaying the first redo log, skips the logs with skip marks set in the first redo log. In this way, it can be made that the second node performs the change operations recorded in the logs without skip marks in the first redo log on the second database that has been updated according to the target data page, thus realizing the synchronization between the second database before split-brain and the first database after split-brain recovery.

[0123] In a possible implementation 2, for the second database before split-brain, the second node can first replay the first redo log, and during the process of replaying the first redo log, skip the logs with skip marks set in the first redo log. In this way, it can be made that the second node performs the change operations recorded in the logs without skip marks in the first redo log on the second database before split-brain, thus obtaining the second database updated based on the first redo log. Next, the second node further updates the second database that has been updated based on the first redo log using the target data page, thus realizing the synchronization between the second database before split-brain and the first database after split-brain recovery. The detailed description of the further update operations performed by the second node on the second database that has been updated based on the first redo log according to the target data page can refer to the description of the update operations performed by the second node on the second database before split-brain according to the target data page above, and will not be elaborated here.

[0124] In this way, through any one of the above two possible implementation methods, the synchronization between the second database maintained by the second node before split-brain and the first database maintained by the first node after split-brain recovery can be completed according to the target data page and the first redo log.

[0125] Since the target data page includes the latest version of the "data page with multiple changes" in the first database, and the "data page with multiple changes" in the first database refers to the data page whose number of change operations in the first database after the split-brain of the service system is greater than the threshold. Therefore, updating the second database before the split-brain with the target data page can achieve the purpose of updating the second database before the split-brain with the latest version of the "data page with multiple changes" in the first database. Moreover, since the logs marked with skip marks in the first redo log are used to record the change operations of the "data page with multiple changes" in the first database, the second node skips playing back the logs with skip marks set in the first redo log when playing back the first redo log, which can achieve skipping playing back the logs used to record the change operations of the "data page with multiple changes" in the first database, and only playing back the redo logs of the data pages with fewer change operations (data pages with the number of change operations less than the threshold) in the first database. For example, when the threshold value is 1, the second node skips playing back the logs with skip marks set in the first redo log when playing back the first redo log, which can achieve only playing back the redo logs of the data pages with 1 change operation in the first database. In this way, it can be avoided that the second node performs the change operations recorded in the redo log on the data pages in the second database that have been updated to the target data page when playing back the first redo log. Therefore, by skipping playing back the logs with skip marks set in the redo log when the second node plays back the redo log, the efficiency of the second node playing back the redo log can be improved, and the IO overhead generated when the second node plays back the redo log can be reduced, thereby improving the synchronization efficiency when synchronizing the second database in the second node and the first database in the first node based on playing back the redo log.

[0126] In the second possible case, when the second node needs to synchronize the second database maintained by itself after the split-brain recovery of the service system (hereinafter referred to as the "second database after split-brain recovery") to the latest first database maintained by the first node after the split-brain recovery of the service system, the target data page received by the second node includes: the data page indicated by the first page ID in the first database, and the data page indicated by the second page ID in the first database. That is, the target data page includes the latest version of the "data page with multiple changes" in the first database when the first node sends the target data page and the latest version of the data page indicated by the second page ID in the first database.

[0127] In this case, in a possible implementation 1, the second node first updates the second database after split-brain recovery through the target data page. Among them, for the detailed description of the second node performing an update operation on the second database after split-brain recovery according to the target data page, reference can be made to the description of the second node performing an update operation on the second database before split-brain according to the target data page above, which will not be elaborated here. Then, for the second database that has been updated according to the target data page, the second node replays the first redo log, and during the process of replaying the first redo log, skips the logs with skip marks set in the first redo log. In this way, it can enable the second node to perform the change operations recorded in the logs without skip marks in the first redo log on the second database that has been updated according to the target data page, thus achieving the synchronization of the second database after split-brain recovery and the first database after split-brain recovery.

[0128] In a possible implementation 2, for the second database after split-brain recovery, the second node can first replay the first redo log, and during the process of replaying the first redo log, skip the logs with skip marks set in the first redo log. In this way, it can enable the second node to perform the change operations recorded in the logs without skip marks in the first redo log on the second database after split-brain recovery, thus obtaining a second database updated based on the first redo log. Then, the second node further updates the second database that has been updated based on the first redo log through the target data page, thereby achieving the synchronization of the second database after split-brain recovery and the first database after split-brain recovery. Among them, for the detailed description of the second node performing a further update operation on the second database that has been updated based on the first redo log according to the target data page, reference can be made to the description of the second node performing an update operation on the second database before split-brain according to the target data page above, which will not be elaborated here.

[0129] In this way, through any one of the above two possible implementation methods, the synchronization of the second database maintained by the second node and the first database maintained by the first node after split-brain recovery can be completed according to the target data page and the first redo log. Moreover, by updating the second database after split-brain recovery through the target data page, the purpose of updating the second database after split-brain recovery with the latest version of the "data page changed multiple times" in the first database can be achieved. And by skipping the logs with skip marks set in the redo log when the second node replays the redo log, the efficiency of the second node replaying the redo log can be improved, and the IO overhead generated when the second node replays the redo log can be reduced, thereby improving the synchronization efficiency when synchronizing the second database in the second node and the first database in the first node based on replaying the redo log.

[0130] Subsequently, as a standby node, the second node can receive the redo log sent by the first node in real time and synchronize the second database and the first database immediately according to the received redo log, which will not be elaborated here.

[0131] Through the data synchronization method described in steps 101 to 104, after the brain split recovery of the service system, the second node as a standby node updates the second database maintained by the second node after the brain split recovery by using the latest version of the "data page with multiple changes" in the first database maintained by the first node. Among them, the "data page with multiple changes" in the first database refers to the data page in the first database where the number of change operations executed after the brain split of the service system is greater than the threshold. And, when the second node replays the redo log recorded by the first node after the brain split occurs, the logs with skip marks set in the redo log are skipped, that is, the logs used to record the change operations of the "data page with multiple changes" in the first database are skipped. In this way, the efficiency of the second node replaying the redo log can be improved, and the IO overhead generated when the second node replays the redo log can be reduced. Furthermore, the synchronization efficiency when synchronizing the second database in the second node and the first database in the first node based on replaying the redo log is improved, and the user is unaware.

[0132] The process of "the first node records the first redo log" is described below.

[0133] Refer to Figure 4 , Figure 4 shows a schematic diagram of a process for recording the first redo log provided by an embodiment of the present application. This method is executed by the first node that serves as the primary node during both the brain split occurrence and the brain split recovery of the service system 20 shown in Figure 2 . For ease of description, in the following, taking the first node recording the corresponding redo log according to the change operation performed on any data page in the first database as an example, the process of the first node recording the first redo log is described. Taking this any data page as the first data page, the method includes the following steps.

[0134] Step 201, the first node determines whether the number of change operations performed on the first data page is greater than the threshold.

[0135] Among them, the first data page is any data page in the first database maintained by the first node. The change operations performed by the first node on the first data page include all change operations performed by the first node on the first data page in response to all data read and write requests from the client from the occurrence of the brain split of the service system to the current moment. The threshold is a positive integer.

[0136] In a possible implementation manner, the first node determines whether the number of change operations performed on the first data page is greater than the threshold by querying the number counter of the first data page.

[0137] Among them, the times counter of the first data page is used to record the number of times the first node performs a change operation on the first data page after a split-brain occurs in the service system. In some embodiments, after a split-brain occurs in the service system, during or after the first node first performs a change operation on the first data page in response to a data read / write request received from a client, the first node sets a times counter with an initial value on the first data page. Subsequently, during the process that the first node subsequently performs a change operation on the first data page in response to a data read / write request received from the client, each time the first node performs a change operation on the first data page, the value of the times counter in the first data page is incremented by 1 accordingly. In this way, the times counter of the first data page can be used to record the number of times the first node performs a change operation on the first data page after a split-brain occurs in the service system.

[0138] As an example, after a split-brain occurs in the service system, after the first node first performs a change operation on the first data page in response to a data read / write request received from a client, a times counter with an initial value of 1 is set on the first data page. When the first node performs a second change operation on the first data page in response to a data read / write request received from the client for the second time, the value of the times counter in the first data page is incremented by 1, that is, the value of the times counter in the first data page at this time is 2. Similarly, when the first node performs a third change operation on the first data page in response to a data read / write request received from the client for the third time, the value of the times counter in the first data page is incremented by 1, that is, the value of the times counter in the first data page at this time is 3.

[0139] Optionally, the first node can determine whether a split-brain has currently occurred in the service system through the heartbeat message of the second node. Furthermore, after determining that a split-brain has occurred in the service system, during or after the first node performs a change operation on the first data page in response to a data read / write request received from a client, the first node sets or updates the times counter on the first data page. In one example, assume that the first node was originally a standby node before a split-brain occurred in the service system. Then, when the first node fails to detect the heartbeat message of the second node, which is the primary node, within a preset time period or a preset number of detection cycles, the first node is upgraded to a new primary node and determines that a split-brain has occurred in the service system. In another example, assume that the first node was originally the primary node before a split-brain occurred in the service system. Then, when the first node fails to detect the heartbeat message of the second node, which is the standby node, within a preset time period or a preset number of detection cycles, the first node determines that a split-brain has occurred in the service system.

[0140] Furthermore, after determining that a split-brain situation has occurred in the service system, during the process of the first node reading and writing the first database in response to data read and write requests received from the client, each time the first node performs a change operation on the first data page, it also performs an operation to query the number counter on the first data page, so as to determine whether the number of times the first node has performed change operations on the first data page after completing the current change operation is greater than the threshold.

[0141] Optionally, after the first node has completed the current change operation on the first data page and updated the number counter of the first data page, it can query whether the difference between the current value of the number counter of the first data page and the initial value of the number counter is greater than or equal to the threshold, so as to determine whether the number of times the first node has performed change operations on the first data page after completing the current change operation is greater than the threshold. For example, taking the threshold and the initial value of the number counter both being 1 as an example, after the first node has completed the current change operation on the first data page and updated the value of the number counter of the first data page to a first value, the first node queries whether the difference between the first value and the initial value 1 of the number counter is greater than or equal to 1, so as to determine whether the number of times the first node has performed change operations on the first data page after completing the current change operation is greater than 1.

[0142] When the first node determines that the number of times it has performed change operations on the first data page after completing the current change operation is greater than the threshold, it indicates that during the period from the occurrence of the split-brain situation in the service system to the completion of the current change operation, the first node has performed a relatively large number of change operations on the first data page. In this case, the first node executes step 202.

[0143] When the first node determines that the number of times it has performed change operations on the first data page after completing the current change operation is less than the threshold, it indicates that during the period from the occurrence of the split-brain situation in the service system to the completion of the current change operation, the first node has performed a relatively small number of change operations on the first data page. For example, when the threshold value is 1 and the number of times the first node has performed change operations on the first data page after completing the current change operation is equal to 1, it indicates that during the period from the occurrence of the split-brain situation in the service system to the completion of the current change operation, the first node has only performed 1 change operation on the first data page. In this case, the first node normally records the redo log corresponding to the current change operation. Here, the redo log corresponding to the current change operation refers to the redo log used to record the current change operation.

[0144] In addition, the embodiments of the present application do not limit the situation where the number of times the first node has performed change operations on the first data page after completing the current change operation is equal to the threshold. For example, in this case, the first node can execute step 202, or the first node normally records the redo log corresponding to the current change operation.

[0145] Step 202: When the number of change operations performed on the first data page is greater than the threshold, the first node sets a skip flag for the first type of logs in the first redo log.

[0146] The first redo log is the redo log recorded by the first node after the brain split of the service system. The first redo log is used to record all change operations performed by the first node on the first database after the brain split of the service system. Specifically, the first redo log is used to record all change operations performed by the first node on the data pages in the first database in response to data read and write requests from the client after the brain split of the service system.

[0147] The first type of logs is used to record the change operations performed by the first node on the first data page after the brain split of the service system. The skip flag of the first type of logs is used to indicate skipping the first type of logs when replaying the first redo log.

[0148] Specifically, in Step 201, when the first node reads and writes the first database in response to the data read and write requests received from the client, each time the first node performs a change operation on the first data page of the first database, the first node performs an operation to query the count counter on this data page, so as to determine whether the number of change operations performed by the first node on the first data page after executing this change operation is greater than the threshold. Therefore, when the first node performs this change operation on the first data page, queries the value of the count counter on the first data page, and determines that the number of change operations performed by the first node on the first data page after executing this change operation is greater than the threshold, the first node generates a redo log (denoted as the first log) in the file of the first redo log for this change operation, and the first node also sets a skip flag in the first log. In addition, the first node also sets a skip flag for the logs in the first redo log that are used to record the change operations performed on the first data page but do not have a skip flag set. This is because when the number of change operations performed by the first node on the first data page after executing a certain change operation is less than or equal to the threshold, the first node does not set a skip flag in the log generated in the first redo log to record this change operation.

[0149] As an example, assume that the initial values of the threshold and the count counter are both 1. Then, after the first node performs the first change operation on the first data page, the value of the count counter queried on the first data page is 1. The first node determines that the number of change operations performed by the first node on the first data page after executing the first change operation is 1, and is equal to the threshold 1. Therefore, the first node generates log 1 in the file of the first redo log for this first change operation, and log 1 is used to record the first change operation performed by the first node on the first data page.

[0150] Next, after the first node performs the second change operation on the first data page, when it queries that the value of the counter on the first data page is 2, the first node determines that the number of times the first node performs the change operation on the first data page until the second change operation is completed is 2, and it is greater than the threshold 1. Therefore, the first node generates log 2 in the file of the first redo log for the second change operation. Log 2 is used to record the second change operation performed by the first node on the first data page. Moreover, the first node sets a skip flag in log 2 and also sets a skip flag in log 1.

[0151] Similarly, after the first node performs the third change operation on the first data page, when it queries that the value of the counter on the first data page is 3, the first node determines that the number of times the first node performs the change operation on the first data page until the third change operation is completed is 3, and it is greater than the threshold 1. Therefore, the first node generates log 3 in the file of the first redo log for the third change operation. Log 3 is used to record the third change operation performed by the first node on the first data page. Moreover, the first node sets a skip flag in log 3.

[0152] In this way, through steps 201 to 202, after a brain split occurs in the service system, the first node, which is the primary node, records the first redo log including the skip flag according to the change operations performed on the data pages in the first database.

[0153] Next, the process of "the first node collects the first page ID" will be described.

[0154] Reference Figure 5 , Figure 5 shows a schematic diagram of a process for collecting the first page ID provided by an embodiment of the present application. This method is executed by the first node of the service system 20 shown in Figure 2 which acts as the primary node both during a brain split and after the brain split is restored. For ease of description, in the following, taking the first node collecting the first page ID for any data page in the first database maintained by itself as an example, the process of the first node collecting the first page ID will be described. Taking this any data page as the first data page as an example, this method first executes step 201, and when the first node determines through step 201 that the number of times it performs the change operation on the first data page until the current change operation is completed is greater than the threshold, it executes the following step 302.

[0155] Step 302: When the number of times of performing the change operation on the first data page is greater than the threshold, the first node adds the page ID of the first data page to a preset storage space to obtain the first page ID stored in the preset storage space.

[0156] Among them, the first page ID is the page ID of the "data page with multiple changes" in the first database. The "data page with multiple changes" in the first database specifically refers to the data page in which the number of change operations performed in the first database after the brain split of the service system is greater than the threshold.

[0157] It should be understood that when the first node determines that a brain split has occurred in the service system, it allocates a preset storage space in its own storage space and uses this as the storage space for storing the page ID set containing at least one first page ID. The detailed description of how the first node determines that a brain split has occurred in the service system can refer to the relevant description in step 201 and will not be elaborated here.

[0158] In a possible implementation, when the number of change operations performed on the first data page is greater than the threshold, the first node can directly add the page ID of the first data page to the preset storage space to obtain a first page ID stored in the preset storage space, that is, the first node collects a first page ID.

[0159] In another possible implementation, when the page ID of the first data page does not exist in the preset storage space, the first node adds the page ID of the first data page to the preset storage space to obtain a first page ID stored in the preset storage space. Or it can be understood that when the page ID of the first data page does not exist in the page ID set stored in the preset storage space, the first node adds the page ID of the first data page to the page ID set stored in the preset storage space to collect a first page ID. For example, when the first node determines that the number of change operations it has performed on the first data page after completing the current change operation is greater than the threshold, it first reads the page ID of the first data page and traverses the page ID set stored in the preset storage space to determine whether the page ID of the first data page already exists in the page ID set stored in the preset storage space. Then, when the page ID of the first data page does not exist in the page ID set stored in the preset storage space, the first node writes the page ID of the first data page into the storage space for storing the page ID set to receive a first page ID. In one example, a deduplication process / thread can be added to the kernel of the first node, so that the first node can add the page ID of the first data page to the preset storage space when the page ID of the first data page does not exist in the page ID set stored in the preset storage space through the deduplication process / thread.

[0160] In this way, it is possible to avoid duplicate page IDs in the set of page IDs stored in the preset storage space. This is because after a brain split occurs in the service system, the first node may perform change operations on the first data page multiple times. That is, for the first data page, the first node may perform step 201 multiple times, so the first node may determine multiple times that the number of change operations performed on the first data page is greater than the threshold. Therefore, the first node determines multiple times that it is necessary to add the page ID of the first data page to the preset storage space for storing the set of page IDs. In this case, to avoid duplicate page IDs in the set of page IDs, the first node adds the page ID of the first data page to the preset storage space for storing the set of page IDs only when the page ID of the first data page does not exist in the set of page IDs.

[0161] In some examples, for ease of operation, the above set of page IDs is stored in a preset storage space in the memory / cache of the first node. To ensure data reliability, the first node periodically or in real time flushes the latest set of page IDs in the memory / cache to the disk of the first node. Here, the disk is composed of a non-volatile storage medium, such as a hard disk, etc. Among them, the process of the first node flushing the set of page IDs in the memory / cache to the disk can be referred to as disk dropping the latest set of page IDs in the memory / cache. It should be understood that the disk dropping operation performed on the latest set of page IDs in the memory / cache includes: writing the first page ID that exists in the set of page IDs in the memory / cache but does not exist in the set of page IDs on the disk, to the set of page IDs on the disk.

[0162] Since the first node needs to read the data pages of the first database stored on the disk into the memory / cache during the process of responding to the data read / write requests from the client, so as to perform corresponding change operations on the data pages in the memory / cache. Therefore, the first node also needs to periodically or in real time flush the changed data pages (i.e., dirty data pages) in the memory / cache into the first database stored on the disk, and this process is called flushing the dirty data pages in the memory / cache to disk. It should be understood that the dirty data pages in the memory / cache include: the data pages obtained by modifying the data pages in the first database read from the disk and / or the newly added data pages for the first database. Therefore, flushing the dirty data pages in the memory / cache into the first database on the disk includes: using the dirty data pages in the memory / cache to overwrite the data pages with the same page ID stored in the first database on the disk, and / or writing the newly added data pages into the first database on the disk. Therefore, optionally, in the embodiments of the present application, the first node may, when performing the disk flushing operation on the dirty data pages in the memory / cache each time, simultaneously perform disk flushing on the memory / cache or the set of page IDs that are the most recently updated in the cache. For example, when the first node performs the disk flushing operation on the dirty data pages in the memory / cache, it triggers the execution of the disk flushing operation on the set of page IDs that are the most recently updated currently. In this way, the consistency between the disk-flushed data pages and the set of page IDs can be ensured.

[0163] Reference Figure 6 , Figure 6 shows a schematic diagram of flushing dirty data pages and a set of page IDs provided by an embodiment of the present application. As Figure 6 shown, the first node includes a first thread (or denoted as ckpt_proc) and a second thread (or denoted as bitmap_file_proc). Among them, when the first thread performs the disk flushing operation on the dirty data pages in the memory / cache of the first node to implement flushing the dirty data pages in the memory / cache into the first database stored on the disk, it triggers the second thread to perform the disk flushing operation on the set of page IDs that are the most recently updated in the memory / cache of the first node, so as to implement writing the most recently updated page IDs in the set of page IDs in the memory / cache into the set of page IDs on the disk.

[0164] Through step 201 and step 302, the process of collecting the set of page IDs by the first node as the primary node according to the number of times of performing change operations on the data pages in the first database after the brain split of the service system is realized.

[0165] It should be noted that since both step 202 and step 302 are executed after step 201, the execution timing of the first node executing step 202 and step 302 in the embodiments of the present application is not limited. For example, the first node may first execute step 202 and then execute step 302. Or, the first node first executes step 302 and then executes step 202. Or, the first node executes step 202 and 302 simultaneously.

[0166] To deepen the understanding of the method provided by the embodiments of the present application, the following describes with examples.

[0167] Refer to Figure 7 , Figure 7 which shows another schematic flowchart of the data synchronization method provided by the embodiments of the present application. For ease of description, in this example, it is defined that the primary and standby nodes of the service system are switched after the split-brain recovery, that is, the primary node of the service system after the split-brain recovery is the standby node before the split-brain occurs, and the standby node of the service system after the split-brain recovery is the primary node before the split-brain occurs. For example, this method is applied to Figure 2 the service system 20 shown in Figure 7 As shown, this method includes the following steps.

[0168] Step 401: After the split-brain of the service system occurs, the first node collects the first page ID and records the first redo log including skip marks.

[0169] Among them, for the detailed description of how the first node determines that the service system has a split-brain, reference can be made to the relevant description of step 201 above. For the detailed description of the first page ID and the first redo log, reference can be made to the description of step 101 above. For the detailed description of how the first node collects the first page ID, reference can be made to the descriptions of step 201 and step 302. For the detailed description of how the first node records the first redo log, reference can be made to the descriptions of step 201 to step 202 above, and will not be elaborated here.

[0170] Step 402: After the split-brain of the service system is recovered, the second node obtains the second page ID and sends a synchronization request for requesting to synchronize the second database and the first database to the first node, and the synchronization request includes the second page ID.

[0171] Among them, for the detailed description of the second page ID, and for the detailed description of how the second node sends a synchronization request including the second page ID to the first node after the split-brain of the service system is recovered, reference can be made to the relevant description in step 101 above, and will not be elaborated here.

[0172] Step 403: The first node receives a synchronization request including the second page ID. In response to the synchronization request, the first node obtains the first redo log, performs an operation of merging and deduplicating the first page ID and the second page ID, and obtains the target data page from the first database according to the page ID obtained after performing the operation of merging and deduplicating the first page ID and the second page ID.

[0173] For the detailed description of the operation of the first node for merging and deduplicating the first page ID and the second page ID, and the detailed description of obtaining the first redo log and the target data page, reference can be made to the relevant descriptions in step 101 above, which will not be elaborated here.

[0174] Step 404: The first node sends the target data page and the first redo log to the second node.

[0175] In response, the second node receives the target data page and the first redo log sent by the first node.

[0176] Step 405: The second node updates the second database according to the target data page, replays the first redo log for the second database, and skips replaying the logs marked with skip marks in the first redo log during the process of replaying the first redo log.

[0177] For the detailed description of step 405, reference can be made to the description in step 104 above.

[0178] In this way, through steps 401 to 405, it is possible to improve the synchronization efficiency when synchronizing the second database maintained by the second node as the standby node and the first database maintained by the first node as the master node after the split-brain recovery of the service system.

[0179] The above mainly introduces the solution provided in the embodiments of the present application from the perspective of the method.

[0180] To implement the above functions, refer to Figure 8 , Figure 8 shows a schematic structural diagram of a data synchronization device provided in an embodiment of the present application. As Figure 8 shown, the data synchronization device 800 is applied to the first node of the service system. The service system includes the first node and the second node after split-brain recovery. Among them, the first node as the master node is used to provide read and write services for the first database, and the second node as the standby node maintains the second database. The data synchronization device 800 is used to execute the data synchronization method described above, for example, used to execute Figure 3 , Figure 4 , Figure 5 , Figure 6 or Figure 7 the part executed by the first node in the method shown. The data synchronization device 800 may include an acquisition unit 801 and a sending unit 802.

[0181] An acquisition unit 801 is configured to acquire a target data page and a first redo log. A sending unit 802 is configured to send the target data page and the first redo log to a second node to achieve synchronization between a second database and a first database. Wherein, the target data page includes the data page indicated by a first page ID in the first database, and the first page ID is the page ID of the data page in the first database that has been changed multiple times. The multiple changes refer to the number of times of performing a change operation on any data page in the first database after a brain split of the service system occurs, and the threshold is a positive integer. The first redo log is used to record the change operations performed on the first database after the brain split of the service system occurs. The first redo log includes logs with skip marks set. The logs with skip marks set are used to record the change operations on the data page that has been changed multiple times. The skip mark is used to indicate skipping the logs with skip marks set when replaying the first redo log.

[0182] As an example, in combination with Figure 3 , the acquisition unit 801 can be used to execute step 101, and the sending unit 802 can be used to execute step 102.

[0183] Optionally, the target data page further includes the data page indicated by a second page ID in the first database, and the second page ID is the page ID of the data page on which a change operation has been performed in the second database during the brain split of the service system. Before the acquisition unit 801 acquires the target data page, the data synchronization device 800 further includes: a receiving unit 803, configured to receive the second page ID sent by the second node. The acquisition unit 801 is specifically configured to read the target data page from the first database according to the second page ID and the first page ID.

[0184] As an example, in combination with Figure 7 , the receiving unit 803 and the acquisition unit 801 can be used to execute step 403.

[0185] Optionally, the data synchronization device 800 further includes: a deduplication unit 804, configured to perform a merge deduplication operation on the second page ID and the first page ID to obtain the page ID of the target data page. The acquisition unit 801 is specifically configured to read the target data page from the first database according to the page ID of the target data page.

[0186] As an example, in combination with Figure 7 , the deduplication unit 804 and the acquisition unit 801 can be used to execute step 403.

[0187] Optionally, before the obtaining unit 801 obtains the first redo log, the data synchronization device 800 further includes: a redo log generation unit 805, configured to record the first redo log according to the change operations performed by the first node on the first database after a brain split occurs in the service system, and to set a skip flag for the first type of logs in the first redo log when the number of change operations performed by the first node on the first data page after a brain split occurs in the service system is greater than a threshold. Wherein, the first type of logs is used to record the change operations performed by the first node on the first data page after a brain split occurs in the service system, and the first data page is any data page in the first database.

[0188] As an example, in combination with Figure 4 , the redo log generation unit 805 can be used to execute step 202.

[0189] Optionally, before the obtaining unit 801 obtains the target data page, the data synchronization device 800 further includes: a page ID collection unit 806, configured to add the page ID of the first data page to a preset storage space to obtain a first page ID stored in the preset storage space when the number of change operations performed by the first node on the first data page after a brain split occurs in the service system is greater than a threshold. Wherein, the first data page is any data page in the first database.

[0190] As an example, in combination with Figure 5 , the page ID collection unit 806 can be used to execute step 302.

[0191] Optionally, the page ID collection unit 806 is specifically configured to add the page ID of the first data page to the preset storage space when the page ID of the first data page does not exist in the preset storage space.

[0192] Optionally, before the page ID collection unit 806 adds the page ID of the first data page to the preset storage space, the data synchronization device 800 further includes: a change times statistics unit 807, configured to query the value of the times counter in the first data page, and to determine that the number of change operations performed by the first node on the first data page after a brain split occurs in the service system is greater than a threshold when it is determined that the difference between the current value and the initial value of the times counter is greater than or equal to the threshold. Wherein, the times counter is used to record the number of change operations performed by the first node on the first data page after a brain split occurs in the service system.

[0193] As an example, in combination with Figure 4 or Figure 5 , the change times statistics unit 807 can be used to execute step 201.

[0194] Optionally, the receiving unit 803 is further configured to receive a synchronization request sent by a second node, where the synchronization request is used to request to synchronize a second database and a first database. The obtaining unit 801 is specifically configured to, in response to the synchronization request, obtain a target data page and a first redo log.

[0195] As an example, in combination with Figure 7 , the receiving unit 803 and the obtaining unit 801 may be used to execute step 403.

[0196] Optionally, the above change operation includes at least one of the following operations: an operation of adding data, an operation of deleting data, or an operation of modifying data.

[0197] Optionally, the above service system is a dual-active hot standby system.

[0198] For the specific description of the above optional manner, reference may be made to the foregoing method embodiments, which will not be elaborated here. In addition, the explanations and descriptions of the beneficial effects of any of the above-provided data synchronization devices 800 may refer to the corresponding method embodiments above and will not be elaborated.

[0199] As an example, in combination with the following Figure 11 , the functions implemented by the sending unit 802 and the receiving unit 803 in the data synchronization device 800 may be implemented through Figure 11 the network interface 1103 shown. The functions implemented by the obtaining unit 801, the deduplication unit 804, the redo log generation unit 805, the page ID collection unit 806, and the change count statistics unit 807 in the data synchronization device 800 may be implemented by the processor 1101 in Figure 11 executing the program code in the memory 1102 in Figure 11 .

[0200] Referring to Figure 9 , Figure 9 shows a schematic structural diagram of another data synchronization device provided in an embodiment of the present application. As shown in Figure 9 , the data synchronization device 900 is applied to a second node of a service system, and the service system includes a first node and the second node after split-brain recovery. Among them, the first node as the primary node is used to provide read and write services for a first database, and the second node as the standby node maintains a second database. The data synchronization device 900 is used to execute the data synchronization method described above, for example, for executing Figure 3 , Figure 4 , Figure 5 , Figure 6 or Figure 7 the part executed by the second node in the method shown. The data synchronization device 900 may include a receiving unit 901 and a synchronization unit 902.

[0201] A receiving unit 901 is configured to receive a target data page and a first redo log sent by a first node. A synchronization unit 902 is configured to synchronize a second database and a first database according to the target data page and the first redo log. The target data page includes the data page indicated by a first page ID in the first database, and the first page ID is the page ID of the data page in the first database that has been changed multiple times. The multiple changes refer to the number of times of performing a change operation on any data page in the first database after a brain split of the service system occurs, and the threshold is a positive integer. The first redo log is used to record the change operations performed on the first database after the brain split of the service system occurs. The first redo log includes logs with skip marks, and the logs with skip marks are used to record the change operations on the data pages that have been changed multiple times. The skip mark is used to indicate skipping the logs with skip marks when replaying the first redo log.

[0202] As an example, in combination with Figure 3 , the receiving unit 901 can be used to execute step 103, and the synchronization unit 902 can be used to execute step 104.

[0203] Optionally, the target data page further includes the data page indicated by a second page ID in the first database, and the second page ID is the page ID of the data page on which a change operation has been performed in the second database during the brain split of the service system. Before the receiving unit 901 receives the target data page and the first redo log sent by the first node, the data synchronization device 900 further includes: a parsing unit 903, configured to parse a second redo log recorded by a second node during the brain split of the service system to obtain the second page ID; a sending unit 904, configured to send the second page ID to the first node. The second redo log is used to record the change operations performed on the second database during the brain split of the service system.

[0204] As an example, in combination with Figure 7 , the parsing unit 903 and the sending unit 904 can be used to execute step 402.

[0205] Optionally, the synchronization unit 902 is specifically configured to update the second database after the brain split recovery of the service system through the target data page, and replay the first redo log for the updated second database, and skip replaying the logs with skip marks in the first redo log, so as to implement the synchronization of the second database and the first database after the brain split recovery of the service system. Or, the synchronization unit 902 is specifically configured to replay the first redo log for the second database after the brain split recovery of the service system, and skip replaying the logs with skip marks in the first redo log, so as to obtain the second database updated based on the first redo log, and update the second database that has been updated based on the first redo log through the target data page, so as to implement the synchronization of the second database and the first database after the brain split recovery of the service system.

[0206] As an example, in combination with Figure 7 , the synchronization unit 902 can be used to execute step 405.

[0207] Optionally, the synchronization unit 902 is specifically configured to update the second database before the occurrence of the split brain of the target data page update service system, and replay the first redo log for the updated second database, and skip the logs with skip marks set in the replay of the first redo log, so as to synchronize the second database before the occurrence of the split brain of the service system and the first database after the split brain recovery of the service system. Alternatively, the synchronization unit 902 is specifically configured to replay the first redo log for the second database before the occurrence of the split brain of the service system, and skip the logs with skip marks set in the replay of the first redo log, so as to obtain the second database updated based on the first redo log, and update the second database updated based on the first redo log through the target data page, so as to synchronize the second database before the occurrence of the split brain of the service system and the first database after the split brain recovery of the service system.

[0208] As an example, in combination with Figure 7 , the synchronization unit 902 can be used to execute step 405.

[0209] Optionally, before the synchronization unit 902 synchronizes the second database and the first database according to the target data page and the first redo log, the parsing unit 903 is configured to parse the second redo log recorded by the second node during the split brain, and the second redo log is used to record all change operations performed on the second database during the split brain. The rollback unit 905 is configured to roll back the data of the second database after the split brain recovery of the service system to before the occurrence of the split brain of the service system to obtain the second database before the occurrence of the split brain of the service system.

[0210] Optionally, before the receiving unit 901 receives the target data page and the first redo log sent by the first node, the sending unit 904 is further configured to send a synchronization request to the first node, and the synchronization request is used to request to synchronize the second database and the first database.

[0211] As an example, in combination with Figure 7 , the sending unit 904 can be used to execute step 402.

[0212] Optionally, the above change operations include at least one of the following operations: an operation of adding data, an operation of deleting data, or an operation of modifying data.

[0213] Optionally, the above service system is a dual-machine hot standby system.

[0214] For the specific description of the above optional manner, reference may be made to the foregoing method embodiments, which will not be elaborated herein. In addition, for the explanations of any of the data synchronization devices 900 provided above and the descriptions of the beneficial effects, reference may be made to the corresponding method embodiments above, which will not be elaborated.

[0215] As an example, in combination with the following Figure 11 , the functions implemented by the receiving unit 901 and the sending unit 904 in the data synchronization device 900 can be implemented through the Figure 11 shown network interface 1103. The functions implemented by the synchronization unit 902, the parsing unit 903, and the rollback unit 905 in the data synchronization device 900 can be implemented by the processor 1101 in the Figure 11 executing the program code in the memory 1102 in the Figure 11 .

[0216] Refer to Figure 10 , Figure 10 shows a schematic structural diagram of another data synchronization device provided by an embodiment of the present application. As shown in Figure 10 , the data synchronization device 1000 is applied to the first node or the second node in the service system. After the brain split recovery, the service system includes a first node and a second node. Among them, the first node as the primary node is used to provide read and write services for the first database, and the second node as the standby node maintains a second database. When the data synchronization device 1000 is applied to the first node, the data synchronization device 1000 is used to execute the part of the data synchronization method executed by the first node described above. When the data synchronization device 1000 is applied to the second node, the data synchronization device 1000 is used to execute the part of the data synchronization method executed by the second node described above.

[0217] The data synchronization device 1000 may include a transceiver unit 1001 and a processing unit 1002. Among them, the transceiver unit 1001 is used to execute the operations related to receiving and / or sending in the data synchronization method described above. The processing unit 1002 is used to execute other operations except for the operations related to receiving and / or sending in the data synchronization method described above.

[0218] As an example, in combination with the following Figure 11 , the functions implemented by the transceiver unit 1001 in the data synchronization device 1000 can be implemented through the Figure 11 shown network interface 1103. The functions implemented by the processing unit 1002 in the data synchronization device 1000 can be implemented by the processor 1101 in the Figure 11 executing the program code in the memory 1102 in the Figure 11 .

[0219] Those skilled in the art should easily realize that for each example unit and algorithm step described in combination with the embodiments disclosed herein, this application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described function for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0220] It should be noted that Figure 8 、 Figure 9 and Figure 10 the division of modules / units in [document name] is illustrative, only a logical function division, and there can be other division methods in actual implementation. For example, two or more functions can also be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of a software function module.

[0221] The embodiments of this application provide a computing device, which is used to implement some or all of the functions in the data synchronization method provided by the embodiments of this application. For example, the computing device is the first node described above, or the second node described above. Figure 11 is a schematic structural diagram of a computing device provided by the embodiments of this application. As Figure 11 shown, the computing device 1100 includes a processor 1101, a memory 1102, a network interface 1103, and a bus 1104. Among them, the processor 1101, the memory 1102, and the network interface 1103 are communicatively connected to each other through the bus 1104.

[0222] The processor 1101 may include a general-purpose processor and / or a dedicated hardware chip. The general-purpose processor may include: a central processing unit (CPU), a microprocessor, or a graphics processing unit (GPU). The CPU is, for example, a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The dedicated hardware chip is a high-performance processing hardware module. The dedicated hardware chip includes at least one of a digital signal processor (DSP), a data processing unit (DPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, a neural processing unit (NPU), a tensor processing unit (TPU), an artificial intelligent chip, or a network processor (NP). The processor 1101 may also be an integrated circuit chip with signal processing capabilities. In the implementation process, some or all of the functions of the method provided by the embodiments of the present application may be completed by the integrated logic circuit in the hardware of the processor 1101 or instructions in software form.

[0223] The memory 1102 is used to store computer programs, which include an operating system 1102a and executable code (i.e., program instructions) 1102b. The memory 1102 is, for example, a read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, or other types of static storage devices that can store static information and instructions, or is a static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), or other types of dynamic storage devices that can store information and instructions, or is a read-only optical disc or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium that can be used to carry or store the desired executable code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. For example, the memory 1102 is used to store the first database or the second database described above. The memory 1102 exists independently, for example, and is connected to the processor 1101 through the bus 1104. Alternatively, the memory 1102 and the processor 1101 are integrated together. The memory 1102 can store executable code. When the executable code stored in the memory 1102 is executed by the processor 1101, the processor 1101 is used to execute some or all of the functions of the data synchronization method provided in the embodiments of the present application. For the implementation manner of the processor 1101 to execute this process, please refer to the relevant descriptions in the foregoing embodiments accordingly. The memory 1102 may also include software modules and data required for other running processes, such as an operating system.

[0224] The network interface 1103 uses a transceiver module such as, but not limited to, a transceiver to implement communication with other devices or communication networks. For example, the network interface 1103 can be any one or any combination of the following devices: network interfaces (such as Ethernet interfaces), wireless network cards, and other devices with network access functions. Among them, the network interface 1103 includes a receiving unit for receiving data / messages and a sending unit for sending data / messages.

[0225] The bus 1104 is of any type and is used to implement the interconnection of internal devices (such as the memory 1102, the processor 1101, and the network interface 1103) of the computing device 1100. For example, a system bus. In the embodiments of the present application, the above-mentioned devices inside the computing device 1100 are interconnected through the bus 1104 as an example. Optionally, the above-mentioned devices inside the computing device 1100 can also communicate with each other using other connection methods in addition to the bus 1104. For example, the above-mentioned devices inside the computing device 1100 are interconnected through an internal logical interface.

[0226] It should be noted that the above-mentioned multiple devices can be respectively arranged on independent chips, or at least partially or entirely arranged on the same chip. Whether to arrange each device on a different chip or to integrate and arrange them on one or more chips often depends on the needs of product design. The embodiments of the present application do not limit the specific implementation forms of the above-mentioned devices. And the descriptions of the processes corresponding to the above-mentioned respective drawings have different focuses. For parts not detailed in a certain process, reference can be made to the relevant descriptions of other processes.

[0227] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product providing the program development platform includes one or more computer instructions. When these computer program instructions are loaded and executed on the computing device 1100, part or all of the functions of the data synchronization method provided by the embodiments of the present application are implemented in whole or in part.

[0228] Moreover, the computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium stores the computer program instructions providing the program development platform.

[0229] An embodiment of the present application further provides a service system. After split-brain recovery, the service system includes a first node and a second node. The first node, serving as the primary node, is used to provide read and write services for a first database, and the second node, serving as the standby node, maintains a second database. Among them, the first node is used to execute the part of the data synchronization method described above that is executed by the first node, and the second node is used to execute the part of the data synchronization method described above that is executed by the second node, so as to synchronize the second database and the first database.

[0230] An embodiment of the present application further provides a computer-readable storage medium. The computer-readable storage medium is a non-volatile computer-readable storage medium. The computer-readable storage medium includes computer program instructions. When the computer program instructions run on a computing device, a computer system, or a processor, the computing device, the computer system, or the processor is caused to implement the data synchronization method provided by the embodiment of the present application.

[0231] An embodiment of the present application further provides a computer program product containing instructions. When the instructions run on a computing device, a computer system, or a processor, the computing device, the computer system, or the processor is caused to implement the data synchronization method provided by the embodiment of the present application.

[0232] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above embodiments can be completed by hardware, or can be completed by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk, an optical disc, or the like.

[0233] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.), and signals involved in the present application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant laws, regulations, and standards of relevant countries and regions.

[0234] An embodiment of the present application further provides a chip. The chip includes a processor for running program instructions or code. The chip or a device including the chip can be used to execute the data synchronization method provided by the embodiment of the present application. Exemplarily, the chip further includes: an input interface, an output interface, and a memory. Among them, the input interface, output interface, processor, and memory of the chip are connected through an internal connection path of the chip. The memory in the chip is used to store program instructions or code run by the processor, and the input interface and output interface of the chip are used for connection and communication between the chip and other chips or devices.

[0235] In the embodiments of the present application, the terms "first", "second", and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "at least one" means one or more, and the term "a plurality" means a plurality, unless otherwise clearly defined.

[0236] The term "and / or" in the present application is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. Additionally, the character " / " in this document generally represents an "or" relationship between the associated objects before and after.

[0237] It should be understood that the terms used in the description of various examples herein are only for describing specific examples and are not intended to be limiting. As used in the description of various examples and the appended claims, the singular forms "a", "an", and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0238] It should be understood that determining B based on A does not mean determining B solely based on A. B can also be determined based on A and / or other information.

[0239] It should be understood that when the term "comprises" (also known as "includes", "including", "comprises", and / or "comprising") is used in this specification, it specifies the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or their groups.

[0240] It should also be understood that in the various embodiments of the present application, the magnitudes of the serial numbers of the various processes do not mean the sequence of execution is prior or subsequent. The execution sequence of each process should be determined by its function and internal logic and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0241] The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the concept and principles of the present application should be included within the protection scope of the present application.

Claims

1. A data synchronization method, characterized in that, Applied to a service system, the service system includes a first node and a second node after split-brain recovery. Among them, the first node as the primary node is used to provide read and write services for a first database, and the second node as the standby node maintains a second database. The method is executed by the first node, and the method includes: Obtain a target data page and a first redo log; wherein, the target data page includes the data page in the first database indicated by a first page identifier ID, and the first page ID is the page ID of the data page in the first database that has been changed multiple times. The multiple changes refer to the number of times of performing change operations on any data page in the first database after the split-brain occurs, and the number of times is greater than a threshold. The threshold is a positive integer; the first redo log is used to record the change operations performed on the first database after the split-brain occurs. The first redo log includes logs with skip marks set, and the logs with skip marks set are used to record the change operations on the data page that has been changed multiple times. The skip mark is used to indicate skipping the logs with the skip mark set when replaying the first redo log; Send the target data page and the first redo log to the second node to synchronize the second database and the first database.

2. The method according to claim 1, characterized in that, The target data page further includes the data page in the first database indicated by a second page ID, and the second page ID is the page ID of the data page in the second database that has been changed during the split-brain; Before obtaining the target data page, the method further includes: Receive the second page ID sent by the second node; The obtaining of the target data page includes: Read the target data page from the first database according to the second page ID and the first page ID.

3. The method according to claim 2, characterized in that, The reading of the target data page from the first database according to the second page ID and the first page ID includes: Perform an operation of merging and deduplicating the second page ID and the first page ID to obtain the page ID of the target data page; Read the target data page from the first database according to the page ID of the target data page.

4. The method according to any one of claims 1 to 3, characterized in that, Before obtaining the first redo log, the method further includes: Record the first redo log according to the change operations performed by the first node on the first database after the split-brain occurs; In the case where the number of times the first node performs change operations on a first data page after the split-brain occurs is greater than the threshold, set the skip mark for the first type of logs in the first redo log. The first type of logs is used to record the change operations performed by the first node on the first data page after the split-brain occurs, and the first data page is any data page in the first database.

5. The method according to any one of claims 1 to 4, characterized in that, Before obtaining the target data page, the method further includes: When the number of change operations performed by the first node on the first data page after the split-brain occurs is greater than the threshold, add the page ID of the first data page to a preset storage space to obtain the first page ID stored in the preset storage space; wherein, the first data page is any data page in the first database.

6. The method according to claim 5, characterized in that, The adding the page ID of the first data page to a preset storage space includes: When the page ID of the first data page does not exist in the preset storage space, add the page ID of the first data page to the preset storage space.

7. The method according to claim 5 or 6, characterized in that, Before adding the page ID of the first data page to the preset storage space, the method further includes: Query the value of the count counter in the first data page, where the count counter is used to record the number of change operations performed by the first node on the first data page after the split-brain occurs; When it is determined that the difference between the current value of the count counter and the initial value of the count counter is greater than or equal to the threshold, determine that the number of change operations performed by the first node on the first data page after the split-brain occurs is greater than the threshold.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: Receive a synchronization request sent by the second node, where the synchronization request is used to request synchronization between the second database and the first database; The obtaining the target data page and the first redo log includes: In response to the synchronization request, obtain the target data page and the first redo log.

9. The method according to any one of claims 1 to 8, characterized in that, The change operation includes at least one of the following operations: an operation of adding data, an operation of deleting data, or an operation of modifying data.

10. The method according to any one of claims 1 to 9, characterized in that, The service system is a dual-active hot standby system.

11. A data synchronization method, characterized in that, Applied to a service system, the service system includes a first node and a second node after split-brain recovery, where the first node, as the primary node, is used to provide read and write services for the first database, and the second node, as the standby node, maintains a second database. The method is executed by the second node, and the method includes: Receive the target data page and the first redo log sent by the first node; wherein, the target data page includes the data page in the first database indicated by the first page identifier ID, the first page ID is the page ID of the data page in the first database that has been changed multiple times, the multiple changes refer to the number of change operations performed on any data page in the first database after the split-brain occurs being greater than the threshold, and the threshold is a positive integer; the first redo log is used to record the change operations performed on the first database after the split-brain occurs, and the first redo log includes logs with skip marks set, and the logs with skip marks set are used to record the change operations on the data page that has been changed multiple times, and the skip mark is used to indicate skipping the logs with skip marks set when replaying the first redo log; Synchronize the second database and the first database according to the target data page and the first redo log.

12. The method according to claim 11, wherein The target data page further includes the data page in the first database indicated by a second page ID, where the second page ID is the page ID of the data page in the second database on which a change operation was performed during the split-brain occurrence; Before receiving the target data page and the first redo log sent by the first node, the method further includes: Parsing the second redo log recorded by the second node during the split-brain occurrence to obtain the second page ID; the second redo log is used to record the change operations performed on the second database during the split-brain occurrence; Sending the second page ID to the first node.

13. The method according to claim 12, wherein Synchronizing the second database and the first database according to the target data page and the first redo log includes: Updating the second database after split-brain recovery through the target data page; Replaying the first redo log for the updated second database and skipping the logs with the skip flag set in the first redo log to synchronize the second database after split-brain recovery and the first database; or, Replaying the first redo log for the second database after split-brain recovery and skipping the logs with the skip flag set in the first redo log to obtain the second database updated based on the first redo log; Updating the second database updated based on the first redo log through the target data page to synchronize the second database after split-brain recovery and the first database.

14. The method according to claim 11, wherein Synchronizing the second database and the first database according to the target data page and the first redo log includes: Updating the second database before split-brain occurrence through the target data page; Replaying the first redo log for the updated second database and skipping the logs with the skip flag set in the first redo log to synchronize the second database before split-brain occurrence and the first database after split-brain recovery; or, Replaying the first redo log for the second database before split-brain occurrence and skipping the logs with the skip flag set in the first redo log to obtain the second database updated based on the first redo log; Updating the second database updated based on the first redo log through the target data page to synchronize the second database before split-brain occurrence and the first database after split-brain recovery.

15. The method according to claim 14, wherein Before synchronizing the second database and the first database according to the target data page and the first redo log, the method further includes: Parsing the second redo log recorded by the second node during the split-brain occurrence; the second redo log is used to record all change operations performed on the second database during the split-brain; According to the parsed second redo log, rolling back the data of the second database after split-brain recovery to before the split-brain occurrence to obtain the second database before the split-brain occurrence.

16. The method according to any one of claims 11 to 15, wherein Before receiving the target data page and the first redo log sent by the first node, the method further includes: Sending a synchronization request to the first node, where the synchronization request is used to request synchronization between the second database and the first database.

17. The method according to any one of claims 11 to 16, wherein The change operation includes at least one of the following operations: an operation of adding data, an operation of deleting data, or an operation of modifying data.

18. The method according to any one of claims 11 to 17, wherein The service system is a dual-active hot standby system.

19. A data synchronization device, wherein Applied to the first node of a service system, after split-brain recovery, the service system includes the first node and the second node. The first node, as the primary node, is used to provide read and write services for the first database, and the second node, as the standby node, maintains a second database. The apparatus includes: A processing unit, configured to perform other operations other than the operations related to receiving and / or sending in the method according to any one of claims 1 to 10; A transceiver unit, configured to perform the operations related to receiving and sending in the method according to any one of claims 1 to 10.

20. A data synchronization device, wherein Applied to the second node of a service system, after split-brain recovery, the service system includes the first node and the second node. The first node, as the primary node, is used to provide read and write services for the first database, and the second node, as the standby node, maintains a second database. The apparatus includes: A transceiver unit, configured to perform the operations related to receiving and sending in the method according to any one of claims 11 to 18; A processing unit, configured to perform other operations other than the operations related to receiving and / or sending in the method according to any one of claims 11 to 18.

21. A computing device, wherein Comprising: A memory, a network interface, and one or more processors. The one or more processors receive or send data through the network interface, and the one or more processors are configured to read program instructions stored in the memory to execute the method according to any one of claims 1 to 10, or execute the method according to any one of claims 11 to 18.

22. A service system, whereinAfter split-brain recovery, the service system includes the first node and the second node. The first node, as the primary node, is used to provide read and write services for the first database, and the second node, as the standby node, maintains a second database; wherein, the first node is configured to execute the method according to any one of claims 1 to 10, and the second node is configured to execute the method according to any one of claims 11 to 18 to achieve synchronization between the second database and the first database.

23. A computer program product containing instructions, characterized in that, When the instruction is run by a computing device or a processor, the computing device or the processor executes the method according to any one of claims 1 to 10, or executes the method according to any one of claims 11 to 18.

24. A computer-readable storage medium, characterized in that, Including computer program instructions, when the computer program instructions are executed by a computing device or a processor, the computing device or the processor executes the method according to any one of claims 1 to 10, or executes the method according to any one of claims 11 to 18.

Images