Data synchronization method and device

Abstract

An embodiment of the present application provides a data synchronization method that can be applied to database-related fields. The data synchronization method provided by the present application includes: the host sends the first data generated by the first process to the backup machine through the data synchronization process, and the backup machine synchronizes the first data; the data synchronization process completes the synchronization of the first data based on the backup machine, wakes up the second process, and then the second process wakes up the first process to perform subsequent operations. The data synchronization process in the present application does not need to wake up the first process that has completed data synchronization, but only needs to wake up the second process, and the second process wakes up the first process that has completed data synchronization, so that the data synchronization process can focus more on triggering the backup machine to synchronize data and receiving the synchronization progress of the backup machine, thereby alleviating the pressure on the data synchronization process and reducing the time it takes for the data synchronization process to wake up the transaction process.

Description

Technical Field

[0001] The present application relates to the field of computer databases, and in particular to a data backup method and device. Background Art

[0002] To improve database system availability, a backup server is typically used to synchronize data, including operation logs, and perform the same operations as the primary server. If the primary server fails, the backup server can be used to provide services.

[0003] There are generally two methods for synchronizing data between the master and standby servers. One is synchronous replication, where the master transaction commits, including data operation records, not only to files on the master server but also to files on the standby server before the transaction is considered committed. The other is asynchronous replication, where after the transaction completes writing data, including data operation records, to local files, the transaction is declared committed without waiting for synchronization on the standby server.

[0004] In comparison, synchronous replication significantly slows down transaction submission latency due to the need to wait for the standby server to synchronize, resulting in significantly longer transaction submission delays and, therefore, significantly lower performance than asynchronous replication. However, if the primary server fails, asynchronous replication may cause committed transactions to not yet be fully synchronized on the standby server, potentially leading to data loss during a failover, which can be problematic. Therefore, synchronous replication is generally used for scenarios requiring high reliability and availability.

[0005] Specifically, the data synchronization process of the host can trigger the standby machine to synchronize data, and when the standby machine is synchronizing data, the transaction process related to the host is in a dormant state. When the standby machine completes the synchronization, the data synchronization process of the host needs to wake up the corresponding transaction process in turn. After completing the wake-up of the transaction process, the data synchronization process can continue to trigger the standby machine to synchronize data.

[0006] However, in some scenarios, the data synchronization process needs to wake up many transaction processes. In this case, the data synchronization process needs to spend a lot of time to wake up the transaction processes, which causes many transaction processes to enter a dormant state while waiting for the synchronization machine to synchronize data, increasing the delay in the transaction process's transaction processing. Summary of the Invention

[0007] In a first aspect, an embodiment of the present application provides a data synchronization method, the method comprising:

[0008] A data synchronization process sends first data to a backup machine. The first data is generated by the first process. The first process may be a transaction process running on the primary machine. When the primary machine receives a user-issued operation instruction to perform a read or write operation, for example, a corresponding transaction process may be generated. The transaction process includes operations corresponding to the user-issued operation instruction. The transaction process may perform corresponding operations on storage devices in the database, such as adding data, deleting data, modifying data, and querying data. The first data may be, but is not limited to, a database transaction log (XLOG) for dealing with crash recovery. Based on this data, the backup machine can obtain the operations performed by the primary machine's transaction process and perform the same operations, thereby achieving data synchronization with the primary machine on the backup machine. The data synchronization process completes synchronization of the first data on the backup machine and wakes up the second process. After receiving the synchronization progress sent by the backup machine, the data synchronization process may wake up the second process. The second process wakes up the first process, thereby waking up the transaction process. Specifically, the data synchronization process may traverse a waiting queue, separate completed transaction processes (including the first process) from the waiting queue based on completion progress, and generate a completion queue (including the first process). This completion queue can be temporarily generated, and after all transaction processes in this completion queue are awakened, this completion queue may no longer exist. The data synchronization process can wake up the second process, and the second process can wake up the first process in the completion queue, that is, the second process wakes up the transaction process. It should be noted that the data synchronization process can first generate a completion queue including the first process, and then wake up the second process; or wake up the second process first, and then generate a completion queue including the first process. The order of completion queue generation and second process awakening is not limited here. The above-mentioned queues (completion queues and waiting queues) are only one implementation method. In the embodiments of the present application, queues can also be replaced by other data structures with the same or similar data structures.

[0009] In this embodiment, the second process may not be a transaction process, but a process used solely to wake up the transaction process. Specifically, the second process may be started when the database service is started. The second process is in a dormant state when there is no transaction process that needs to be woken up. After being woken up by the data synchronization process, the second process may wake up the transaction process. After completing the wake-up of the transaction process, the second process may enter a dormant state.

[0010] In an embodiment of the present application, the data synchronization process does not need to wake up the transaction process whose data synchronization has been completed, but only needs to wake up a process specifically used to wake up the completed data synchronization, so that the data synchronization process can focus more on triggering the backup machine to synchronize data and receiving the synchronization progress of the backup machine, thereby alleviating the pressure on the data synchronization process and reducing the time it takes for the data synchronization process to wake up the transaction process.

[0011] In an optional implementation, the method further includes: before the data synchronization process sends the first data to the standby machine, the data synchronization process receives a data synchronization request corresponding to the first process.

[0012] In an optional implementation, the first process is configured to perform a first operation on a database, and the first data includes an operation record corresponding to the first operation.

[0013] In an optional implementation, the method further includes: the data synchronization process sends second data to the backup machine, where the second data is generated by the second process; the data synchronization process completes synchronization of the first data and the second data based on the backup machine, and wakes up the second process.

[0014] In an optional implementation, the second process is configured to perform a second operation on the database, and the second data includes an operation record corresponding to the second operation.

[0015] In an optional implementation, the host includes a completion queue, and the identifier of the second process is located in the first element of the completion queue; and waking up the second process includes:

[0016] A process corresponding to the identifier included in the first element is awakened.

[0017] In an optional implementation, before waking up the second process, after the standby machine completes synchronization of the first data, a completion queue is generated at least according to an identifier of the second process, wherein the identifier of the second process is located in the first element of the completion queue.

[0018] In an optional implementation, the first element is the head element of the completion queue.

[0019] In an optional implementation, the identifier of the first process is located in the second element of the completion queue, and in the completion queue, the second element is located after the first element; the second process wakes up the process corresponding to the identifier included in the second element.

[0020] In an optional implementation, the second process may wake up processes corresponding to identifiers included in a plurality of elements following the first element in the completion queue.

[0021] In the embodiment of the present application, the second process can replace the data synchronization process to wake up other processes in the completion queue. Specifically, the second process needs to detect whether the subsequent elements of the completion queue contain other transaction processes. If so, it is responsible for waking up the remaining transaction processes in the completion queue.

[0022] In an optional implementation, the second element is adjacent to the first element; the second process may wake up a process included in an element in the completion queue that is adjacent to the first element and located after the first element.

[0023] In an optional implementation, the host includes a waiting queue, and before the standby machine completes the synchronization, the identifier of the second process is located at the third element in the waiting queue, and the identifier of the first process is located at the fourth element in the waiting queue, and the fourth element is after the third element; wherein the completion queue is generated based on the waiting queue.

[0024] In this embodiment, the generated completion queue maintains the order of processes in the original waiting queue, ensuring the first-in-first-out of transaction processing.

[0025] In an optional implementation, the completion queue is all or part of the waiting queue.

[0026] In an optional implementation, the relative positions of the third element and the fourth element in the waiting queue are the same as the relative positions of the first element and the second element in the completion queue. That is, the interval and order between the elements where the second process is located and the elements where the first process is located remain unchanged.

[0027] In an optional implementation, the identifier of the second process is located at the head element of the waiting queue, and the identifier of the first process is located at an element other than the head element in the waiting queue.

[0028] In this embodiment, if the identifier of the second process is located in an element other than the head element in the completion queue, the data synchronization process can wake up the process included in the head element and the processes corresponding to the identifiers included in the adjacent elements until the second process is woken up.

[0029] In an optional implementation, the host includes a completion queue, the identifier of the first process is located in the first element of the completion queue, and the second process is located outside the completion queue; the second process wakes up the first process, including: the second process wakes up the process corresponding to the identifier included in the first element.

[0030] In this embodiment, the second process may not be a transaction process, but a separate process for waking up a transaction process. The first process is the process corresponding to the identifier included in any element of the waiting queue (waiting queue). In this embodiment of the present application, the completion queue is generated based on at least the first process, and the identifier of the first process is located in the first element of the completion queue; the second process wakes up the process corresponding to the identifier included in the first element.

[0031] In an optional implementation, the method further includes:

[0032] Before the second process wakes up the first process, after the standby machine completes synchronization of the first data, a completion queue is generated at least according to the identifier of the first process, wherein the identifier of the first process is located in the first element of the completion queue.

[0033] In an optional implementation, the host includes a waiting queue, and before the standby machine completes the synchronization, the identifier of the first process is located at the fourth element in the waiting queue; wherein the completion queue is generated based on the waiting queue.

[0034] In an optional implementation, the completion queue is all or part of the waiting queue.

[0035] In an optional implementation, the first process and the second process run on a host, the host includes multiple NUMA nodes, and the second process and the first process belong to processes included in the same NUMA node.

[0036] In an optional implementation, the first process and the second process run on a host, the host includes multiple NUMA nodes, and at least one of the second process and the first process belongs to the same NUMA node as the completion queue.

[0037] In an optional implementation, the first process and the second process run on a host, the host includes multiple NUMA nodes, and at least one of the second process and the first process belongs to the same NUMA node as the waiting queue; and / or the completion queue and the waiting queue belong to the same NUMA node.

[0038] Optionally, in an embodiment of the present application, the completion queue and the wait queue belong to the same NUMA node. Data associated with the second process and the first process occupy memory resources on the same NUMA node. Data associated with the completion queue and the wait queue occupy memory resources on the same NUMA node.

[0039] In this embodiment of the present application, when the database system is initialized, a corresponding wait queue can be assigned to each NUMA node, and the transaction process (the second process and the first process) can be assigned to the wait queue of that node. Similarly, the completion queue (completion queue) separated from the wait queue must also be located on the node. Based on this, when the completion queue traverses and wakes up the transaction process, the process and the accessed data are located on the same NUMA node, avoiding costly remote memory access and further improving the efficiency of the parallel wakeup mechanism.

[0040] In a second aspect, the present application provides a data writing method, which includes: a data writing process wakes up a second process based on completing a write operation on first data, wherein the first data is generated by the first process; and the second process wakes up the first process.

[0041] In this embodiment of the present application, after the data writing process completes the write operation on the first data, it can wake up the second process. The second process then completes the wake-up of the process or completes it in conjunction with other processes in the completion queue. This can reduce the time the local data writing process spends waking up the transaction process, thereby improving the efficiency of the file write operation.

[0042] In an optional implementation, the first process is configured to perform a first operation on a database, and the first data includes an operation record corresponding to the first operation.

[0043] In an optional implementation, the data writing process wakes up the second process based on completing the write operation on the first data, including:

[0044] The data writing process wakes up the second process based on completing the writing operation of the first data and the second data; wherein the second data is generated by the second process.

[0045] In an optional implementation, the second process is configured to perform a second operation on the database, and the second data includes an operation record corresponding to the second operation.

[0046] In an optional implementation, the host includes a completion queue, and the identifier of the second process is located in the first element of the completion queue; and waking up the second process includes:

[0047] A process corresponding to the identifier included in the first element is awakened.

[0048] In an optional implementation, the method further includes:

[0049] Before waking up the second process, after the data writing process completes the writing operation on the first data, the data writing process generates a completion queue at least according to the identifier of the second process, wherein the identifier of the second process is located in the first element of the completion queue.

[0050] In an optional implementation, the first element is the head element of the completion queue.

[0051] In an optional implementation, the identifier of the first process is located in the second element of the completion queue, and in the completion queue, the second element is located after the first element;

[0052] Accordingly, the second process wakes up the first process, including:

[0053] The second process wakes up the process corresponding to the identifier included in the second element.

[0054] In an optional implementation, the second process waking up the process included in the second element includes:

[0055] The second process wakes up processes corresponding to identifiers included in a plurality of elements following the first element in the completion queue.

[0056] In an optional implementation, the second element is adjacent to the first element; accordingly, the second process waking up the process included in the second element includes:

[0057] The second process wakes up a process included in an element in the completion queue that is adjacent to and subsequent to the first element.

[0058] In an optional implementation, the host includes a waiting queue, and before the data writing process completes the writing operation on the first data, the identifier of the second process is located at the third element in the waiting queue, and the identifier of the first process is located at the fourth element in the waiting queue, and the fourth element is after the third element; wherein the completion queue is generated based on the waiting queue.

[0059] In an optional implementation, the completion queue is all or part of the waiting queue.

[0060] In an optional implementation, the relative positions of the third element and the fourth element in the waiting queue are the same as the relative positions of the first element and the second element in the completion queue.

[0061] In an optional implementation, the identifier of the second process is located at the head element of the waiting queue, and the identifier of the first process is located at an element other than the head element in the waiting queue.

[0062] In an optional implementation, the host includes a completion queue, the identifier of the first process is located in a first element of the completion queue, and the second process is located outside the completion queue; and the second process waking up the first process includes:

[0063] The second process wakes up the process corresponding to the identifier included in the first element.

[0064] In an optional implementation, the method further includes:

[0065] Before the second process wakes up the first process, after the standby machine completes synchronization of the first data, a completion queue is generated at least according to the identifier of the first process, wherein the identifier of the first process is located in the first element of the completion queue.

[0066] In an optional implementation, the host includes a waiting queue, and before the data writing process completes the writing operation of the first data, the identifier of the first process is located at the fourth element in the waiting queue; wherein the completion queue is generated based on the waiting queue.

[0067] In an optional implementation, the completion queue is all or part of the waiting queue.

[0068] In an optional implementation, the first process and the second process run on a host, the host includes multiple NUMA nodes, and the second process and the first process belong to processes included in the same NUMA node.

[0069] In an optional implementation, the first process and the second process run on a host, the host includes multiple NUMA nodes, and at least one of the second process and the first process belongs to the same NUMA node as the completion queue.

[0070] In an optional implementation, the first process and the second process run on a host, the host includes multiple NUMA nodes, and at least one of the second process and the first process belongs to the same NUMA node as the waiting queue; and / or the completion queue and the waiting queue belong to the same NUMA node.

[0071] In a third aspect, the present application provides a data synchronization device that may include a processor coupled to a memory, the memory storing program instructions. When the program instructions stored in the memory are executed by the processor, the first aspect and any of the methods described above are implemented. For details of the steps performed by the processor in each possible implementation of the first aspect, please refer to the first aspect and will not be repeated here.

[0072] In a fourth aspect, the present application provides a data writing device, which may include a processor coupled to a memory, wherein the memory stores program instructions. When the program instructions stored in the memory are executed by the processor, any of the methods described in the second aspect is implemented. For details regarding the steps performed by the processor in executing the second aspect and its various possible implementations, please refer to the second aspect and will not be repeated here.

[0073] In a fifth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect above.

[0074] In a sixth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute any of the methods described in the second aspect above.

[0075] In a seventh aspect, the present application provides a circuit system, which includes a processing circuit, and the processing circuit is configured to execute the above-mentioned first aspect and any one of the methods described.

[0076] In an eighth aspect, the present application provides a circuit system, which includes a processing circuit, and the processing circuit is configured to execute the second aspect and any of the methods described above.

[0077] In a ninth aspect, the present application provides a computer program which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect above.

[0078] In a tenth aspect, the present application provides a computer program which, when executed on a computer, enables the computer to execute any of the methods described in the second aspect above.

[0079] In an eleventh aspect, the present application provides a chip system, which includes a processor for supporting a server or a threshold value acquisition device to implement the functions involved in the above aspects, for example, sending or processing the data and / or information involved in the above methods. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for the server or communication device. The chip system can be composed of a chip or can include a chip and other discrete devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 A schematic diagram of the architecture of a NUMA system provided in an embodiment of the present application;

[0081] Figure 2 A schematic diagram of the connection between two nodes in a NUMA architecture;

[0082] Figure 3 It is a schematic diagram of a process processing flow;

[0083] Figure 4a It is a schematic diagram of a waiting queue structure;

[0084] Figure 4b It is a schematic diagram of a waiting queue structure;

[0085] Figure 4c It is a schematic diagram of a waiting queue structure;

[0086] Figure 5a It is a schematic diagram of the structure of a waiting queue and a completion queue;

[0087] Figure 5b It is a schematic diagram of the structure of a waiting queue and a completion queue;

[0088] Figure 5c It is a schematic diagram of the structure of a waiting queue and a completion queue;

[0089] Figure 6a It is a schematic diagram of a process processing flow;

[0090] Figure 6b It is a schematic diagram of a process processing flow;

[0091] Figure 6c It is a schematic diagram of a process processing flow;

[0092] Figure 6d It is a schematic diagram of a process processing flow;

[0093] Figure 7 It is a schematic diagram of a process processing flow;

[0094] Figure 8 It is a schematic diagram of a process processing flow;

[0095] Figure 9 It is a schematic diagram of a process processing flow;

[0096] Figure 10 It is a schematic diagram of a process processing flow;

[0097] Figure 11 It is a schematic diagram of a process processing flow;

[0098] Figure 12 It is a schematic diagram of a process processing flow;

[0099] Figure 13 It is a schematic diagram of a process processing flow;

[0100] Figure 14 It is a schematic diagram of a process processing flow;

[0101] Figure 15 This is a schematic diagram of the structure of a data synchronization device;

[0102] Figure 16 A schematic diagram of the structure of a computer program product;

[0103] Figure 17 The figure shows the structure of a host. DETAILED DESCRIPTION

[0104] The following describes the embodiments of the present application in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present application, rather than all the embodiments. Those skilled in the art will appreciate that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0105] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or that are inherent to these processes, methods, products or devices. The naming or numbering of steps in this application does not mean that the steps in the method flow must be executed in the time or logical sequence indicated by the naming or numbering. The process steps that have been named or numbered can be changed in the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0106] The database system in the embodiment of the present application may include a host and a standby machine. The application scenario of the present application is first introduced below. The host and the standby machine in the embodiment of the present application may be a NUMA-based system, wherein in the architecture of the NUMA system 100, it may be a multi-socket system 100. Figure 1 , Figure 1 A schematic diagram of the architecture of a NUMA system 100 provided in an embodiment of the present application is shown in FIG. Figure 1 As shown in FIG, , NUMA system 100 includes a socket 101a and a socket 101b. Sockets 101a and 101b may be collectively referred to herein as sockets and may be used to install a central processing unit (CPU). The sockets may be communicatively coupled to each other via an interconnect 104. For example, each socket may be connected to each of the other sockets via a point-to-point quick path interconnect (QPI) link. It should be noted that QPI is an interconnect architecture, and the interconnection between the sockets in the embodiments of the present application may also be implemented using other interconnect architectures, such as other point-to-point architectures, ring architectures, and bus architectures, which are not limited here. The number of sockets depicted in NUMA system 100 is merely an example, and those skilled in the art will appreciate that a different number of sockets may exist. For example, NUMA system 100 may include, for example, six, four, or fewer sockets, or NUMA system 100 may include, for example, sixteen, thirty-two, or more sockets.

[0107] The slot may include multiple nodes, each node has its own CPU and memory, and the nodes are connected and communicated with each other through the interconnection 104. Figure 1 As shown in FIG, socket 101a includes nodes 102a and 102b, and socket 101b includes nodes 103a and 103b. Each node includes one memory and six CPUs. It should be noted that the number of nodes and CPUs depicted in NUMA system 100 is only an example, and those skilled in the art will understand that each socket may include other numbers of nodes, and each node may include other numbers of CPUs.

[0108] In the embodiment of the present application, the software of the host involves the transaction process, data synchronization process, waiting queue, etc.; the software of the standby machine mainly involves the data synchronization process. In an existing implementation, the main process of data synchronization is as follows:

[0109] 1. When the host transaction process is preparing to submit a transaction, it first completes the local data submission, then sends a synchronization request to the host's data synchronization process and adds itself to the waiting queue.

[0110] 2. The data synchronization process on the host responds to the request and sends data to the standby machine;

[0111] 3. The data synchronization process on the standby machine receives and writes it into the standby machine's file system;

[0112] 4. The data synchronization process on the standby machine will report the synchronization progress to the master machine;

[0113] 5. The host data synchronization process traverses the waiting queue according to the synchronization progress and wakes up the transaction processes that have completed synchronization one by one. At this point, the transaction process can complete the transaction commit operation.

[0114] The host data synchronization process is continuously executing the following steps:

[0115] 1. Awakened by the transaction process, reads unsynchronized data, and sends it to the standby machine through the network;

[0116] 2. Wait for the progress indication sent by the standby machine;

[0117] 3. According to the current progress, traverse the waiting queue and wake up the completed transaction process from the queue.

[0118] However, as the number of central processing unit (CPU) cores and non-uniform memory access (NUMA) nodes continues to increase, the capabilities of single machines are becoming increasingly powerful, but some shortcomings are gradually exposed: the long wake-up time of transaction processes affects the efficiency of host data synchronization processes.

[0119] The reasons for these deficiencies include:

[0120] 1. Process wakeup operations are performed within the host's data synchronization process. With the advancement of multi-core capabilities on single machines, the concurrency of supported transaction processes has also increased dramatically. Consequently, the number of processes in the waiting queue has increased significantly, significantly lengthening the time it takes to traverse the queue and wake up. This results in a longer processing cycle for the host data synchronization process.

[0121] 2. Due to the impact of the first point above, transaction process wakeup will also be relatively inefficient. Even if the backup server data synchronization has been completed, the transaction process at the end of the queue will need to wait a long time before being woken up.

[0122] 3. For servers with multiple NUMA nodes, access to this waiting queue involves a large amount of memory access across NUMA nodes. Compared with single-node servers, the access cost will be further increased.

[0123] In order to solve the above problems, this application provides a data backup method, referring to Figure 2 , Figure 2 This is a flow chart of a data backup method provided by an embodiment of the present application. This embodiment of the present application provides a data synchronization method, such as Figure 2 As shown, the data synchronization method provided by this application specifically includes:

[0124] 201. A data synchronization process sends first data to a backup machine, where the first data is generated by a first process.

[0125] In an embodiment of the present application, the first process is configured to perform a first operation on a database, and the first data includes an operation record corresponding to the first operation. In one implementation, the first process is a transaction process, and the first data may be, but is not limited to, a database transaction log for dealing with crash recovery (XLOG). Based on this data, the backup machine can obtain the operations performed by the transaction process of the primary machine and perform the same operations, thereby achieving data synchronization with the primary machine on the backup machine.

[0126] In an embodiment of the present application, the second process can be configured to perform a second operation on the database and generate a second log including an operation record corresponding to the second operation. The second process and the first process can be transaction processes in the host, wherein the host can have one or more transaction processes. When the host receives an operation instruction issued by a user to perform a transaction such as reading and writing, it can generate a corresponding transaction process. The transaction process includes an operation corresponding to the operation instruction issued by the user. The transaction process can perform corresponding operations on the storage device in the database, wherein the operation can include adding data, deleting data, modifying data, and querying data, etc., which is not limited by the present application. After completing the corresponding operation, the transaction process can generate data including an operation record corresponding to the operation. Through this data, the backup machine can perform the same operation on the storage device as the host. In one implementation, the log can be, but is not limited to, a transaction log for dealing with crash recovery (XLOG). Based on this log, the backup machine can obtain the operations performed by the transaction process of the host and perform the same operations, thereby realizing the backup of the host on the backup machine.

[0127] Optionally, in an embodiment of the present application, the first process and the second process may be transaction processes, wherein the second process is configured to perform a second operation on a database (a storage device or memory in a database system, etc.) and generate second data including an operation record corresponding to the second operation, wherein the second process is configured to perform a second operation on the database, and the second data includes an operation record corresponding to the second operation. The first process is configured to perform a first operation on the database and generate first data including an operation record corresponding to the first operation. The second process and the first process may be different transaction processes on the host, generated in response to different user operation instructions.

[0128] After generating data, the transaction process can store the data in a cache, and the data writing process will write the data from the cache to the corresponding file. The data writing process can be a transaction process that competes for the lock, or a separate process dedicated to data writing, which is not limited here.

[0129] In an embodiment of the present application, after the second process generates second data including an operation record corresponding to the second operation, the data writing process can write the second data into a corresponding file; and after the first process generates first data including an operation record corresponding to the first operation, the data writing process can write the first data into a corresponding file. The data writing process can be the second process, the first process, or another transaction process, or another process specifically used for data writing.

[0130] After generating data, the transaction process can be added to a specific waiting queue; wherein, the transaction process can be sorted in the waiting queue based on the data queue number LSN (LOG sequence number) where the data is located, and added to the corresponding queue element in sequence. Figure 3 , assuming that there are 4 transaction processes joining the waiting queue, coded as T1 to T4, and the data positions LSN of data T1 to T4 are marked as 10 to 40 in sequence, such as Figure 3 As shown in FIG, when transaction processes T1 to T4 are added to the waiting queue, they can be sorted according to LSN and added to the waiting queue in sequence.

[0131] In an embodiment of the present application, after the second process and the first process generate data, they can add the identifiers of the second process and the first process to a waiting queue. Specifically, after the second process generates the second data, it can add the identifier corresponding to the second process to the waiting queue; after the first process generates the first data, it can add the identifier corresponding to the first process to the waiting queue. It should be noted that the identifier of the first process can uniquely indicate the first process, and the identifier of the second process can uniquely indicate the second process. The above identifiers can be process pointers or other character strings, which are not limited by this application.

[0132] Wherein, the identifier of the second process is located at the third element in the waiting queue, the identifier of the first process is located at the fourth element in the waiting queue, and the fourth element is after the third element. Wherein, the waiting queue may include a plurality of elements connected in series, the next pointer of the head element points to the adjacent element, and so on, the next pointer of each element points to the adjacent element in the queue, until the tail element. In one implementation, the next pointer of the tail element may point to NULL. If the waiting queue is a bidirectional loopback queue, it is used to more conveniently support some other operations, such as when a transaction process needs to be temporarily canceled, so the tail element may also point to other queue elements. In an embodiment of the present application, the fourth element is after the third element, that is, there is a next pointer path in the waiting queue, which can be pointed to by the third element.

[0133] In an optional implementation, the waiting queue may include multiple elements, each of which may include an identifier of a process, the identifier of the second process is located in an element other than the head element in the waiting queue, and the fourth element is adjacent to the third element, that is, the next pointer of the third element points to the fourth element. Figure 4a , the identifier of the second process is located in the element (the third element) other than the head element of the waiting queue, and the identifier of the first process is located in the element (the fourth element) adjacent to the third element, wherein the identifier of the first process may also be located at the tail element of the waiting queue.

[0134] In an optional implementation, the waiting queue may include multiple elements, each of which may include an identifier of a process, the identifier of the second process is located at the head element of the waiting queue, and the identifier of the first process is located at an element other than the head element in the waiting queue. It should be noted that the head element here refers to the head in the queue. For example, referring to Figure 4b, the identifier of the second process is located at the head element of the waiting queue, and the identifier of the first process is located at an element other than the head element in the waiting queue, wherein other elements may exist between the element where the identifier of the second process is located and the element where the identifier of the first process is located, and the identifier of the first process may also be located at the tail element of the waiting queue.

[0135] In one optional implementation, referring to Figure 4c The waiting queue may include multiple elements, each of which may include the identifier of a process, the identifier of the second process is located in the element other than the head element in the waiting queue, the identifier of the first process is located in the element other than the head element in the waiting queue, and the fourth element is not adjacent to the third element, that is, there are elements including other processes between the fourth element and the third element.

[0136] In an embodiment of the present application, after generating data, the transaction process, in addition to entering the waiting queue, can also send a data synchronization request to the data synchronization process of the host to instruct the data synchronization process to trigger the backup machine to synchronize data. Specifically, after receiving the synchronization request, if the data synchronization process is in an idle state, it can send the data that currently needs to be synchronized to the backup machine. The data synchronization process of the backup machine can receive the data and write it to the corresponding backup machine file system. After completing the data writing (i.e., completing the data synchronization), the data synchronization process of the backup machine can feedback the synchronization progress to the host to indicate which data has been synchronized.

[0137] In an embodiment of the present application, the data synchronization process may be the aforementioned data synchronization process. After receiving a data synchronization request, if the data synchronization process is in an idle state, the data (first data and second data) that currently needs to be synchronized may be sent to the standby machine. The data synchronization process of the standby machine may receive the first data and the second data and write them to the corresponding standby machine file system. After completing the writing of the first data and the second data (i.e., completing the synchronization of the first data and the second data), the data synchronization process of the standby machine may feed back the synchronization progress to the data synchronization process of the master machine to indicate that the synchronization of the first data and the second data has been completed.

[0138] 202. The data synchronization process completes synchronization of the first data based on the standby machine and wakes up the second process.

[0139] After receiving the synchronization progress from the standby server, the data synchronization process can awaken the secondary process. Specifically, the data synchronization process can traverse the waiting queue, separate completed transaction processes from the waiting queue based on their completion progress, and generate a completion queue. This completion queue can be temporary and may cease to exist after all transaction processes in it are awakened. It should be noted that if the data synchronization process is fast enough, multiple completion queues may be separated from the same waiting queue and coexist.

[0140] In an embodiment of the present application, the data synchronization process triggers the standby machine to synchronize the second data with the first data; and based on the standby machine, the synchronization of the second data with the first data is completed, and a completion queue is generated at least according to the second process, wherein the identifier of the second process is located in the first element of the completion queue, then the data synchronization process can wake up the process corresponding to the identifier included in the first element.

[0141] Next, we describe how the data synchronization process generates a completion queue based on at least the second process:

[0142] In an embodiment of the present application, the completion queue is generated based on at least the second process and the first process, the identifier of the first process is located in the second element of the completion queue, and in the completion queue, the second element is located after the first element.

[0143] In an embodiment of the present application, the completion queue can be generated by the data synchronization process based on the waiting queue. In an optional implementation, the completion queue is all or part of the waiting queue. Among them, if the logs corresponding to only a part of the processes in the waiting queue have been backed up on the standby machine, the next pointer of the element where the process at the end of the waiting queue is located can be changed from pointing to the adjacent process to pointing to NULL. At this time, the element where the process at the end of the waiting queue is located in the above part of the processes becomes the tail element.

[0144] In the embodiment of the present application, the relative positions of the third element and the fourth element in the waiting queue are the same as the relative positions of the first element and the second element in the completion queue. That is, the interval and order between the elements where the second process is located and the elements where the first process is located remain unchanged.

[0145] Reference Figure 5a, the identifier of the second process is located in the element other than the head element of the waiting queue (the third element), and the identifier of the first process is located in the element adjacent to the third element (the fourth element). The first process can also be located at the tail element of the waiting queue, that is, the next pointer of the element where the first process is located points to NULL. Accordingly, the completion queue is part of the waiting queue. In the waiting queue, the third and fourth elements are adjacent, and the fourth element is located after the third element. Correspondingly, in the completion queue, the third and fourth elements are adjacent, and the fourth element is located after the third element.

[0146] Reference Figure 5b The identifier of the second process is located at the head element of the waiting queue, and the identifier of the first process is located at an element other than the head element in the waiting queue. There may be other elements between the element where the second process is located and the element where the first process is located. The first process may also be located at the tail element of the waiting queue, that is, the next pointer of the element where the first process is located points to NULL. Accordingly, the completion queue is part of the waiting queue. In the waiting queue, the third element is the head element, and there are N elements between the third and fourth elements, and the fourth element is located after the third element. Accordingly, in the completion queue, the third element is the head element, and there are N elements between the third and fourth elements, and the fourth element is located after the third element.

[0147] Reference Figure 5c , the identifier of the second process is located in an element other than the head element in the waiting queue, the identifier of the first process is located in an element other than the head element in the waiting queue, and the fourth element is not adjacent to the third element, that is, there is an element including other processes between the fourth element and the third element. Accordingly, the completion queue is part of the waiting queue, and in the waiting queue, there are N elements between the third and fourth elements, and the fourth element is located after the third element. Accordingly, in the completion queue, there are N elements between the third and fourth elements, and the fourth element is located after the third element.

[0148] In an embodiment of the present application, the data synchronization process can wake up the second process in the completion queue, that is, the data synchronization process can wake up the process corresponding to the identifier included in the first element in the completion queue, and the first element includes the identifier of the second process.

[0149] In an optional implementation, the identifier of the second process is located in the head element of the completion queue, and the data synchronization process can wake up the process corresponding to the identifier included in the head element of the completion queue.

[0150] In an optional implementation, the identifier of the second process is located in an element other than the head element in the completion queue, and the data synchronization process can wake up the process included in the head element and the processes corresponding to the identifiers included in the adjacent elements until the second process is woken up.

[0151] 203. The second process wakes up the first process.

[0152] In the embodiment of the present application, after being awakened, the second process can wake up the first process.

[0153] In an optional implementation, the second process wakes up processes included in one or more elements in the completion queue that follow the first element. In this embodiment of the present application, the second process can replace the data synchronization process to wake up other processes in the completion queue. Specifically, the second process needs to detect whether subsequent elements in the completion queue contain other transaction processes. If so, it is responsible for waking up the remaining transaction processes in the completion queue.

[0154] In an optional implementation, the identifier of the second process is located at the head element of the completion queue. For details, refer to Figure 6a After the data synchronization process wakes up the second process, the second process can wake up the Nth process (the Nth process may or may not exist. When it does not exist, the element where the second process is located is adjacent to the element where the first process is located, and the number of Nth processes is not limited, that is, there can be one or more elements between the second process and the first process). After completing the wake-up of the Nth process, the first process can be woken up, and the processes after the first process can be woken up (if the identifier of the first process is located at the tail element of the completion queue, this is not necessary).

[0155] In an optional implementation, the identifier of the second process is located in an element other than the head element in the completion queue. For details, refer to Figure 6b The data synchronization process needs to wake up the Nth process first (the number of the Nth process is not limited, that is, there can be one or more elements before the second process). After that, the data synchronization process can wake up the second process. Then, the second process can wake up the first process and wake up the processes after the first process (if the identifier of the first process is at the tail element of the completion queue, this is not necessary).

[0156] In an optional implementation, the second element is adjacent to the first element; the second process wakes up the process corresponding to the identifier included in the element adjacent to the first element and located after the first element in the completion queue.

[0157] In an optional implementation, the identifier of the second process is located at the head element of the completion queue. For details, refer to Figure 6cAfter the data synchronization process wakes up the second process, the second process can wake up the first process included in the adjacent element. Then, the first process can wake up the Nth process adjacent to it, and so on, until the transaction process included in the tail element is woken up.

[0158] In an optional implementation, the identifier of the second process is located in an element other than the head element in the completion queue. For details, refer to Figure 6d The data synchronization process needs to wake up the Nth process first (the number of the Nth process is not limited, that is, there can be one or more elements before the second process). After that, the Nth process can wake up the second process, and then the second process can wake up the first process, and so on, until the transaction process included in the tail element is woken up.

[0159] In an embodiment of the present application, if the identifier of the second process is located at an element other than the head element (not the tail element) of the completion queue, then the awakening of subsequent processes of the second process is not performed by the data synchronization process (it can be done by the second process or the subsequent process of the second process), so that the data synchronization process (data synchronization process) only needs to wake up a part of the processes that have completed data synchronization, so that the data synchronization process can focus more on triggering the backup machine to synchronize data and receiving the synchronization progress of the backup machine, thereby alleviating the pressure on the data synchronization process and reducing the time for the data synchronization process to wake up the transaction process.

[0160] If the identifier of the second process is located at the head element of the completion queue, and the subsequent processes are not awakened by the data synchronization process (they can be done by the second process or the subsequent process of the second process), the data synchronization process (data synchronization process) only needs to wake up one process (the second process), thereby further reducing the pressure on the data synchronization process and shortening the time it takes for the data synchronization process to wake up the transaction process.

[0161] Reference Figure 7 In the embodiment of the present application, after waking up the second process, the data synchronization process can trigger the backup machine to synchronize data, and the rest of the transaction process can be completed by the second process. Figure 8 In the existing implementation, after waking up the second process, the data synchronization process needs to wake up the remaining transaction processes. Only after the awakening of the remaining transaction processes is completed can the backup machine be triggered to perform data synchronization. At this time, there may be many transaction processes waiting to be awakened in the waiting queue, which makes the process awakening efficiency of the data synchronization process poor.

[0162] In an embodiment of the present application, the data synchronization process does not need to wake up all processes that have completed data synchronization, but only needs to wake up a part of the processes that have completed data synchronization, so that the data synchronization process can focus more on triggering the backup machine to synchronize data and receiving the synchronization progress of the backup machine, thereby alleviating the pressure on the data synchronization process and reducing the time for the data synchronization process to wake up the transaction process.

[0163] In one embodiment, the second process may not be a transaction process, but a separate process for waking up the transaction process. The first process is the process corresponding to the identifier included in any element of the waiting queue (waiting queue). In this embodiment of the present application, the completion queue is generated based on at least the first process, and the identifier of the first process is located in the first element of the completion queue; the second process wakes up the process corresponding to the identifier included in the first element.

[0164] In an embodiment of the present application, a completion queue can be generated based on at least the first process, wherein the identifier of the first process is located in the first element of the completion queue. The master includes a waiting queue, and the method further includes: before the backup completes the backup, adding the first process to the waiting queue, wherein the completion queue is generated based on the waiting queue. Optionally, the completion queue can be all or part of the waiting queue.

[0165] It should be noted that, referring to Figure 9 , the second process can wake up the transaction processes included in all the elements in the completion queue; or, when the identifier of the first process is located at the head element of the completion queue, the second process can wake up the first process, and the first process wakes up the transaction processes included in the remaining elements in the completion queue.

[0166] Or, refer to Figure 10 The second process can wake up the process corresponding to the identifier included in the element before the element where the first process is located until the first process is woken up, and the first process wakes up the transaction processes included in the remaining elements in the completion queue.

[0167] Or, refer to Figure 11When the identifier of the first process is located at the head element of the completion queue, the second process can wake up the first process, and the first process wakes up the process corresponding to the identifier included in the adjacent and subsequent elements, and then the process corresponding to the identifier included in the adjacent and subsequent elements awakened above completes the transaction processes included in the remaining elements in the queue; or, when the identifier of the first process is located at the head element of the completion queue, the second process can wake up the first process, and the first process wakes up the process corresponding to the identifier included in the adjacent and subsequent elements, and then the process corresponding to the identifier included in the adjacent element awakened above wakes up the process corresponding to the identifier included in the adjacent element, and so on, until all the transaction processes included in the elements in the completion queue are awakened.

[0168] In an embodiment of the present application, the data synchronization process does not need to wake up the process that has completed data synchronization, but only needs to wake up a process specifically used to wake up the completed data synchronization, so that the data synchronization process can focus more on triggering the backup machine to synchronize data and receiving the synchronization progress of the backup machine, thereby alleviating the pressure on the data synchronization process and reducing the time for the data synchronization process to wake up the transaction process.

[0169] Optionally, in an embodiment of the present application, the host includes multiple NUMA nodes, the second process and the first process belong to processes included in the same NUMA node, and data related to the second process and the first process occupy memory resources under the same NUMA node.

[0170] Optionally, in an embodiment of the present application, the host includes multiple NUMA nodes, and the second process, at least one of the first process, and the completion queue belong to the same NUMA node. Data related to the second process, at least one of the first process, and the completion queue occupy memory resources on the same NUMA node.

[0171] Optionally, in an embodiment of the present application, the host includes multiple NUMA nodes, and the second process, at least one of the first process, and the waiting queue belong to the same NUMA node. Data related to the second process, at least one of the first process, and the waiting queue occupy memory resources on the same NUMA node.

[0172] Optionally, in an embodiment of the present application, the completion queue and the wait queue belong to the same NUMA node. Data associated with the second process and the first process occupy memory resources on the same NUMA node. Data associated with the completion queue and the wait queue occupy memory resources on the same NUMA node.

[0173] In this embodiment of the present application, when the database system is initialized, a corresponding wait queue can be assigned to each NUMA node, and the transaction process (the second process and the first process) can be assigned to the wait queue of that node. Similarly, the completion queue (completion queue) separated from the wait queue must also be located on the node. Based on this, when the completion queue traverses and wakes up the transaction process, the process and the accessed data are located on the same NUMA node, avoiding costly remote memory access and further improving the efficiency of the parallel wakeup mechanism.

[0174] Next, an application example of this application is described. Figure 12 and Figure 13 , if there are two NUMA nodes (node ​​1 and node 2) in the NUMA system.

[0175] During system initialization, two waiting queues (wait queues) are prepared for the data synchronization process, one for Node 1 and one for Node 2. When transaction processes start, they can be bound to different NUMA nodes using an even-split strategy. Here, we assume that odd-numbered processes are bound to Node 1, while even-numbered processes are bound to Node 2.

[0176] When the transaction process is ready to commit and has completed writing to the local log file, it sends a request to the data synchronization process and joins the corresponding waiting queue. For example, processes numbered 1 / 3 / 5 / 7 / 9 are added to the waiting queue under node 1; processes numbered 2 / 4 / 6 / 8 / 10 are added to the waiting queue under node 2.

[0177] The master's data synchronization process sends logs to the slave and receives progress reports from the slave. Upon detecting a synchronization update on the slave, the master process prepares to wake up the relevant transaction processes. During this wakeup, it iterates through each waiting queue and disconnects them based on the synchronization completion position, separating the completed transaction queues. This step requires holding a lock to protect the waiting queues.

[0178] As above Figure 12 As shown in the figure, transaction processes numbered 1 / 3 / 5 and transaction processes numbered 2 / 4 / 6 are separated into different completion queues (completion queues). The data synchronization process can wake up the first transaction process in each completion queue and is no longer responsible for waking up other processes. Figure 13 As shown, only processes numbered 1 and 2 are awakened. The data synchronization process then immediately proceeds to the next round of work, and no longer processes the remaining processes. Each process that completes the head element of the queue (processes numbered 1 and 2) is responsible for completing the process awakening task for that queue. The awakening operations for processes numbered 1 and 2 are executed concurrently.

[0179] Reference Figure 14 , Figure 14A data writing method according to an embodiment of the present invention is shown in FIG. Figure 14 As shown in the figure, the data writing method provided by the embodiment of the present application includes:

[0180] 1401. The data writing process wakes up the second process based on completing a write operation on first data, wherein the first data is generated by the first process.

[0181] 1402. The second process wakes up the first process.

[0182] In an embodiment of the present application, the first process is configured to perform a first operation on the database, and the first data includes an operation record corresponding to the first operation.

[0183] and Figures 2 to 13 The difference between the embodiment described in the embodiment is that this embodiment is applied in the scenario of data writing, wherein the data can be and Figure 2 In the corresponding embodiment, the first data and the second data are of the same type, and writing the data of the database system to the local file is a very important processing step. In order to avoid the additional consumption caused by competition, a dedicated data service process is used to write data for other transaction processes (that is, the data writing process in this embodiment).

[0184] In existing implementations, after the data writing process completes the write operation on the first data, it needs to wake up the first process. This takes a long time to wake up the transaction process, and the data writing process can only write the data corresponding to the remaining transaction processes after waking up the transaction process, which is inefficient.

[0185] In the embodiment of the present application, after the data writing process completes the writing operation of the first data, the second process can be awakened, and then the second process completes or combines with the other processes in the completion queue to complete the awakening of the process. Figures 2 to 13 The description of the embodiment in will not be repeated here.

[0186] In the embodiment of the present application, the time spent by the local data writing process on waking up the transaction process can be reduced, thereby improving the efficiency of the file writing operation.

[0187] In an optional implementation, the first process is configured to perform a first operation on a database, and the first data includes an operation record corresponding to the first operation.

[0188] In an optional implementation, the data writing process wakes up the second process based on completing the write operation on the first data, including:

[0189] The data writing process wakes up the second process based on completing the writing operation of the first data and the second data; wherein the second data is generated by the second process.

[0190] In an optional implementation, the second process is configured to perform a second operation on the database, and the second data includes an operation record corresponding to the second operation.

[0191] In an optional implementation, the host includes a completion queue, and the identifier of the second process is located in the first element of the completion queue; and waking up the second process includes:

[0192] A process corresponding to the identifier included in the first element is awakened.

[0193] In an optional implementation, the method further includes:

[0194] Before waking up the second process, after the data writing process completes the writing operation on the first data, the data writing process generates a completion queue at least according to the identifier of the second process, wherein the identifier of the second process is located in the first element of the completion queue.

[0195] In an optional implementation, the first element is the head element of the completion queue.

[0196] In an optional implementation, the identifier of the first process is located in the second element of the completion queue, and in the completion queue, the second element is located after the first element;

[0197] Accordingly, the second process wakes up the first process, including:

[0198] The second process wakes up the process corresponding to the identifier included in the second element.

[0199] In an optional implementation, the second process waking up the process included in the second element includes:

[0200] The second process wakes up processes corresponding to identifiers included in a plurality of elements following the first element in the completion queue.

[0201] In an optional implementation, the second element is adjacent to the first element; accordingly, the second process waking up the process included in the second element includes:

[0202] The second process wakes up a process included in an element in the completion queue that is adjacent to and subsequent to the first element.

[0203] In an optional implementation, the host includes a waiting queue, and before the data writing process completes the writing operation on the first data, the identifier of the second process is located at the third element in the waiting queue, and the identifier of the first process is located at the fourth element in the waiting queue, and the fourth element is after the third element; wherein the completion queue is generated based on the waiting queue.

[0204] In an optional implementation, the completion queue is all or part of the waiting queue.

[0205] In an optional implementation, the relative positions of the third element and the fourth element in the waiting queue are the same as the relative positions of the first element and the second element in the completion queue.

[0206] In an optional implementation, the identifier of the second process is located at the head element of the waiting queue, and the identifier of the first process is located at an element other than the head element in the waiting queue.

[0207] In an optional implementation, the host includes a completion queue, the identifier of the first process is located in a first element of the completion queue, and the second process is located outside the completion queue; and the second process waking up the first process includes:

[0208] The second process wakes up the process corresponding to the identifier included in the first element.

[0209] In an optional implementation, the method further includes:

[0210] Before the second process wakes up the first process, after the standby machine completes synchronization of the first data, a completion queue is generated at least according to the identifier of the first process, wherein the identifier of the first process is located in the first element of the completion queue.

[0211] In an optional implementation, the host includes a waiting queue, and before the data writing process completes the writing operation of the first data, the identifier of the first process is located at the fourth element in the waiting queue; wherein the completion queue is generated based on the waiting queue.

[0212] In an optional implementation, the completion queue is all or part of the waiting queue.

[0213] In an optional implementation, the first process and the second process run on a host, the host includes multiple NUMA nodes, and the second process and the first process belong to processes included in the same NUMA node.

[0214] In an optional implementation, the first process and the second process run on a host, the host includes multiple NUMA nodes, and at least one of the second process and the first process belongs to the same NUMA node as the completion queue.

[0215] In an optional implementation, the first process and the second process run on a host, the host includes multiple NUMA nodes, and at least one of the second process and the first process belongs to the same NUMA node as the waiting queue; and / or the completion queue and the waiting queue belong to the same NUMA node.

[0216] Reference Figure 15 , the present application also provides a data synchronization device 1500, the device 1500 includes: a processing circuit 1501 and a communication interface 1502; wherein,

[0217] The processing circuit 1501 is configured to perform Figure 2 as well as Figure 14 The above-mentioned communication interface 1502 is configured to execute the sending action or receiving action related to the interaction between the standby machines in the above-mentioned embodiment.

[0218] Reference Figure 16 , the present application also provides a computer-readable storage medium, in some embodiments, the above Figures 2 to 14 The disclosed methods may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of manufacture. Figure 16 Schematically illustrates a conceptual partial view of an example computer program product arranged according to at least some embodiments presented herein, the example computer program product comprising a computer program for executing a computer process on a computing device. In one embodiment, the example computer program product 1600 is provided using a signal-bearing medium 1601. The signal-bearing medium 1601 may include one or more program instructions 1602 that, when executed by one or more processors, may provide the above-described instructions for executing a computer process. Figures 2 to 14 Thus, for example, reference to Figure 2 In the embodiment shown in FIG, one or more features of blocks 201-203 may be undertaken by one or more instructions associated with signal bearing medium 1601, referring to FIG. Figure 14 In the embodiment shown in , one or more features of blocks 1401-1402 may be performed by one or more instructions associated with signal bearing medium 1601. In addition, Figure 16 Program instructions 1602 in also describe example instructions.

[0219] In some examples, signal-bearing medium 1601 may include computer-readable media 1603, such as, but not limited to, a hard drive, a compact disc (CD), a digital video disc (DVD), a digital tape, memory, read-only memory (ROM), random access memory (RAM), and the like. In some embodiments, signal-bearing medium 1601 may include computer-recordable media 1604, such as, but not limited to, memory, a read / write (R / W) CD, a R / W DVD, and the like. In some embodiments, signal-bearing medium 1601 may include communication media 1605, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, and the like). Thus, for example, signal-bearing medium 1601 may be communicated via a wireless form of communication medium 1605 (e.g., a wireless communication medium that complies with the IEEE 802.11 standard or other transmission protocols). One or more program instructions 1602 may be, for example, computer-executable instructions or logic-implemented instructions. In some examples, the computing device of computing device can be configured to, in response to one or more program instructions 1602 communicated to computing device by computer-readable medium 1603, computer recordable medium 1604 and / or communication medium 1605, provide various operations, functions or actions.It should be understood that the arrangement described here is only for the purpose of example.Thus, it will be understood by those skilled in the art that other arrangements and other elements (e.g., machines, interfaces, functions, sequences, and functional groups, etc.) can be used instead, and some elements can be omitted together according to the desired result.In addition, many of the described elements can be implemented as discrete or distributed components or in any appropriate combination and position to combine the functional entities implemented by other components.

[0220] Based on the same inventive concept, the embodiment of the present invention provides a host, see Figure 17 , the host 1700 includes: a transmitter 1703, a receiver 1704, a memory 1702 and a processor 1701 coupled to the memory 1702. The transmitter 1703, the receiver 1704, the memory 1702 and the processor 1701 can be connected via a bus or other means ( Figure 17 The bus connection is used as an example). Where:

[0221] Processor 1701 may be one or more central processing units (CPUs). Figure 17Taking a processor as an example, when the processor 1701 is a CPU, the CPU can be a single-core CPU or a multi-core CPU.

[0222] The memory 1702 includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a portable read-only memory (CD-ROM). The memory 1702 is used for related instructions and data, and is also used to store program codes, which are specifically used to implement Figures 2 to 14 The method described in the embodiment;

[0223] The transmitter 1703 is used to send data to the outside (e.g., a standby device);

[0224] The receiver 1704 is used to receive data from the outside (eg, a standby machine).

[0225] It should be noted that, in an embodiment of the present invention, the host can be an independent device in the database system, for example, the host is an independent physical server. The host can also be a non-independent device. In this case, in one application scenario, the host can be built into the host, or exist as a functional module of the host (for example, the host and the host are different virtual machines, connected through an I / O interface, and coexist on the same physical server); in another application scenario, the host can be built into a backup machine, or exist as a functional module of a backup machine (for example, a backup machine and the host are different virtual machines, connected through an I / O interface, and coexist on the same physical server).

[0226] It should also be noted that the execution steps of the processor 1701 and other technical features involved in the processor 1701 can also be referred to. Figures 2 to 17 The relevant contents of the host described in the method embodiment will not be repeated here.

[0227] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0228] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0229] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0230] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0231] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, server, or other network device, etc.) to execute the present application. Figure 2 All or part of the steps of the method described in the embodiment. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0232] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A data synchronization method, characterized in that: The method comprises: The data synchronization process sends first data to the standby machine, where the first data is generated by the first process running on the master machine; The data synchronization process sends second data to the standby machine, where the second data is generated by the second process; The data synchronization process completes synchronization of the first data and the second data based on the standby machine, and wakes up the second process; and The second process wakes up the first process.

2. The method according to claim 1, characterized in that The method further comprises: Before the data synchronization process sends the first data to the standby machine, the data synchronization process receives a data synchronization request corresponding to the first process.

3. The method according to claim 1 or 2, characterized in that The first process is configured to perform a first operation on a database, and the first data includes an operation record corresponding to the first operation.

4. The method according to claim 1, wherein The second process is configured to perform a second operation on the database, and the second data includes an operation record corresponding to the second operation.

5. The method according to claim 1 or 4, characterized in that The host includes a completion queue, and the identifier of the second process is located in the first element of the completion queue; The waking up the second process includes: A process corresponding to the identifier included in the first element is awakened.

6. The method according to any one of claims 1 to 2, characterized in that: The method further comprises: Before waking up the second process, after the standby machine completes synchronization of the first data, a completion queue is generated at least according to an identifier of the second process, wherein the identifier of the second process is located in a first element of the completion queue.

7. The method according to claim 5, characterized in that The first element is the head element of the completion queue.

8. The method according to claim 5, characterized in that The identifier of the first process is located in the second element of the completion queue, and in the completion queue, the second element is located after the first element; Accordingly, the second process wakes up the first process, including: The second process wakes up the process corresponding to the identifier included in the second element.

9. The method according to claim 8, characterized in that The second process waking up the process included in the second element includes: The second process wakes up processes corresponding to identifiers included in a plurality of elements following the first element in the completion queue.

10. The method according to claim 8, characterized in that The second element is adjacent to the first element; accordingly, the second process wakes up the process included in the second element, including: The second process wakes up a process included in an element in the completion queue that is adjacent to and subsequent to the first element.

11. The method according to claim 8, characterized in that The host includes a waiting queue. Before the standby machine completes the synchronization, the identifier of the second process is located at the third element in the waiting queue, and the identifier of the first process is located at the fourth element in the waiting queue, and the fourth element is after the third element; wherein, the completion queue is generated based on the waiting queue.

12. The method according to claim 11, characterized in that The completion queue is all or part of the waiting queue.

13. The method according to claim 11, characterized in that The relative positions of the third element and the fourth element in the waiting queue are the same as the relative positions of the first element and the second element in the completion queue.

14. The method according to claim 11, characterized in that The identifier of the second process is located at the head element of the waiting queue, and the identifier of the first process is located at an element other than the head element in the waiting queue.

15. The method according to any one of claims 1 to 2, characterized in that: The host includes a completion queue, the identifier of the first process is located in a first element of the completion queue, and the second process is located outside the completion queue; The second process waking up the first process includes: The second process wakes up the process corresponding to the identifier included in the first element.

16. The method according to any one of claims 1 to 2, characterized in that: The method further comprises: Before the second process wakes up the first process, after the standby machine completes synchronization of the first data, a completion queue is generated at least according to the identifier of the first process, wherein the identifier of the first process is located in the first element of the completion queue.

17. The method according to claim 15, characterized in that The host includes a waiting queue. Before the standby machine completes the synchronization, the identifier of the first process is located at the fourth element in the waiting queue. The completion queue is generated according to the waiting queue.

18. The method according to claim 17, characterized in that The completion queue is all or part of the waiting queue.

19. The method according to any one of claims 1 to 2, characterized in that: The first process and the second process run on a host, the host includes multiple NUMA nodes, and the second process and the first process are processes included in the same NUMA node.

20. The method according to claim 5, characterized in that The first process and the second process run on a host, the host includes multiple NUMA nodes, and at least one of the second process and the first process belongs to the same NUMA node as the completion queue.

21. The method according to claim 11, wherein The first process and the second process run on a host, the host includes multiple NUMA nodes, and at least one of the second process and the first process belongs to the same NUMA node as the waiting queue; And / or, the completion queue and the waiting queue belong to the same NUMA node.

22. A data writing method, characterized in that: The method comprises: The data writing process wakes up the second process based on completing a write operation on first data and second data, wherein the first data is generated by the first process running on the host, and the second data is generated by the second process; and The second process wakes up the first process.

23. The method according to claim 22, characterized in that The first process is configured to perform a first operation on a database, and the first data includes an operation record corresponding to the first operation.

24. The method according to claim 22, characterized in that The second process is configured to perform a second operation on the database, and the second data includes an operation record corresponding to the second operation.

25. The method according to claim 22, wherein The host includes a completion queue, and the identifier of the second process is located in the first element of the completion queue; waking up the second process includes: A process corresponding to the identifier included in the first element is awakened.

26. The method according to claim 22, characterized in that The method further comprises: Before waking up the second process, after the data writing process completes the writing operation on the first data, the data writing process generates a completion queue at least according to the identifier of the second process, wherein the identifier of the second process is located in the first element of the completion queue.

27. The method according to claim 25, characterized in that The first element is the head element of the completion queue.

28. The method according to claim 25, characterized in that The identifier of the first process is located in the second element of the completion queue, and in the completion queue, the second element is located after the first element; Accordingly, the second process wakes up the first process, including: The second process wakes up the process corresponding to the identifier included in the second element.

29. The method according to claim 28, characterized in that The second process waking up the process included in the second element includes: The second process wakes up processes corresponding to identifiers included in a plurality of elements following the first element in the completion queue.

30. The method according to claim 28, wherein The second element is adjacent to the first element; accordingly, the second process wakes up the process included in the second element, including: The second process wakes up a process included in an element in the completion queue that is adjacent to and subsequent to the first element.

31. The method according to claim 28, wherein The host includes a waiting queue. Before the data writing process completes the writing operation on the first data, the identifier of the second process is located at the third element in the waiting queue, and the identifier of the first process is located at the fourth element in the waiting queue, and the fourth element is after the third element; wherein, the completion queue is generated based on the waiting queue.

32. The method according to claim 31, characterized in that The completion queue is all or part of the waiting queue.

33. The method according to claim 31, characterized in that The relative positions of the third element and the fourth element in the waiting queue are the same as the relative positions of the first element and the second element in the completion queue.

34. The method according to claim 31, wherein The identifier of the second process is located at the head element of the waiting queue, and the identifier of the first process is located at an element other than the head element in the waiting queue.

35. The method according to claim 22 or 23, characterized in that The host includes a completion queue, the identifier of the first process is located in a first element of the completion queue, and the second process is located outside the completion queue; The second process waking up the first process includes: The second process wakes up the process corresponding to the identifier included in the first element.

36. The method according to any one of claims 22 and 23, characterized in that: The method further comprises: Before the second process wakes up the first process, after the standby machine completes synchronization of the first data, a completion queue is generated at least according to the identifier of the first process, wherein the identifier of the first process is located in the first element of the completion queue.

37. The method according to claim 35, characterized in that The host includes a waiting queue, and before the data writing process completes the writing operation on the first data, the identifier of the first process is located at the fourth element of the waiting queue; wherein the completion queue is generated according to the waiting queue.

38. The method according to claim 37, wherein The completion queue is all or part of the waiting queue.

39. The method according to any one of claims 22 to 23, characterized in that The first process and the second process run on a host, the host includes multiple NUMA nodes, and the second process and the first process are processes included in the same NUMA node.

40. The method according to claim 25, wherein The first process and the second process run on a host, the host includes multiple NUMA nodes, and at least one of the second process and the first process belongs to the same NUMA node as the completion queue.

41. The method according to any one of claims 31 to 34, characterized in that The first process and the second process run on a host, the host includes multiple NUMA nodes, and at least one of the second process and the first process belongs to the same NUMA node as the waiting queue; And / or, the completion queue and the waiting queue belong to the same NUMA node.

42. A computer-readable storage medium, characterized in that A computer program is stored, and when the program is executed by a computer, the method according to any one of claims 1 to 21 is implemented.

43. A computer-readable storage medium, characterized in that A computer program is stored, and when the program is executed by a computer, the method according to any one of claims 22 to 41 is implemented.

Images