Generation method of global consistency snapshot of distributed database and related product

Abstract

The invention provides a method for generating a consistency snapshot of a distributed database and a related product. The method comprises the following steps: respectively acquiring own directory snapshots and other directory snapshots of other nodes in the distributed database by each node of the distributed database; and each node generates a global directory snapshot of the distributed database according to each own directory snapshot and the other directory snapshots, and generates a consistent snapshot of the distributed database in the node according to the global directory snapshot. According to the technical scheme, the consistency snapshots of the distributed database are generated by adopting a parallel method, and compared with a serial method in the prior art, the problem that loads of all nodes are unbalanced can be prevented.

Description

Technical Field

[0001] The present invention relates to a database technical solution, and in particular to a method for generating a consistent snapshot of a distributed database and related products. Background Art

[0002] The global consistency of a database means that all data in the database system should be kept consistent at any point in time, which is the key to ensuring data reliability and normal operation of the system. The global consistency data of a database includes the consistency snapshot and consistency point of the database. The consistency snapshot refers to the read-only copy of the database at a specific point in time, which records the state of the database at that point in time; the consistency point refers to the time point when incremental replication starts, that is, the data submitted by the database after the consistency point is incrementally replicated; through the consistency snapshot and consistency point of the database, the incremental data of the database in a specific time period can be obtained.

[0003] The global consistency data of the database is achieved through the logical replication function. For a single database, the consistency snapshot of the database can be generated based on the pre-write log. However, for a distributed database, since transactions are executed in shards on each node and each node has its own independent transaction segment, it is necessary to obtain the pre-write log of each node to generate a consistent snapshot of the distributed database.

[0004] The method for constructing a globally consistent snapshot of a distributed database in the prior art is to set a unique global sequence number in the distributed database to record the generation order of the pre-write logs on each node, then one of the nodes obtains the pre-write logs of other nodes, and sorts the pre-write logs according to the global sequence number, and finally parses the pre-write logs to obtain a consistent snapshot of the distributed database.

[0005] The above method for generating a consistent snapshot of a distributed database is a serial method. Since this method concentrates the workload of generating a consistent snapshot of a distributed database on one of the nodes, it will increase the workload of the node, resulting in a load imbalance problem among the nodes of the distributed database. Summary of the invention

[0006] An object of the present invention is to solve the problem of unbalanced loads on nodes of the distributed database caused by the need to concentrate the workload of building a consistent snapshot on one node in the existing distributed database, so as to improve the performance of the distributed database.

[0007] A further object of the present invention is to improve the accuracy of the obtained consistent snapshot of the distributed database.

[0008] Another further object of the present invention is to improve the work efficiency of generating consistent snapshots of a distributed database while ensuring the accuracy of the consistent snapshots.

[0009] In particular, in a first aspect, the present invention provides a method for generating a consistent snapshot of a distributed database, comprising:

[0010] Each node of the distributed database obtains its own directory snapshot and other directory snapshots of other nodes in the distributed database;

[0011] Each of the nodes generates a global directory snapshot of the distributed database according to each of the own directory snapshots and the other directory snapshots, and generates a consistent snapshot of the distributed database in the node according to the global directory snapshot.

[0012] Furthermore, the step of each of the nodes generating a global directory snapshot of the distributed database according to each of the own directory snapshots and the other directory snapshots includes:

[0013] Each of the nodes obtains the global sequence number of the write-ahead log of other nodes respectively, and determines whether the global sequence number of its own write-ahead log is the maximum global sequence number;

[0014] If not, the self-directory snapshot is aligned with another directory snapshot having the largest global sequence number to generate the global directory snapshot.

[0015] Furthermore, after the step of each of the nodes generating a global directory snapshot of the distributed database according to each of the own directory snapshots and the other directory snapshots, the method further includes:

[0016] Each of the nodes obtains the global directory snapshots of other nodes respectively, and generates a new global directory snapshot respectively according to each of the global directory snapshots.

[0017] Furthermore, after the step of each of the nodes generating a global directory snapshot of the distributed database according to each of the own directory snapshots and the other directory snapshots, the method further includes:

[0018] Each of the nodes sends its own global directory snapshot to the coordination node of the distributed database;

[0019] The step of each of the nodes obtaining a global directory snapshot of other nodes respectively includes:

[0020] Each of the nodes sends first preset request information to the coordinating node respectively, so as to obtain global directory snapshots of other nodes from the coordinating node.

[0021] Furthermore, the step of each node of the distributed database obtaining its own directory snapshot includes:

[0022] Each of the nodes obtains its own write-ahead log respectively, and generates its own directory snapshot respectively according to its own write-ahead log.

[0023] Furthermore, the step of each node of the distributed database obtaining its own directory snapshot and other directory snapshots of other nodes in the distributed database includes:

[0024] After acquiring the snapshot of its own directory, each of the nodes sends the snapshot of its own directory to the coordination node of the distributed database; and

[0025] Each of the nodes sends second preset request information to the coordinating node respectively, so as to obtain the other directory snapshots from the coordinating node.

[0026] Furthermore, after the step of each of the nodes generating a global directory snapshot of the distributed database according to each of the own directory snapshots and the other directory snapshots, the method further includes:

[0027] Each of the nodes generates a consistency point of the distributed database at the node according to its own global directory snapshot.

[0028] In a second aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the above-mentioned generation methods.

[0029] In a third aspect, the present invention further provides a computer program product, comprising a computer program, which implements the steps of any of the above generation methods when executed by a processor.

[0030] In a fourth aspect, the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any of the above generation methods.

[0031] The technical solution provided by the present invention adopts a parallel method to generate a consistent snapshot of a distributed database, that is, the work of generating a consistent snapshot is no longer concentrated on one node of the distributed database, but each node of the distributed database generates a global directory snapshot of the distributed database respectively, and then generates a consistent snapshot of the distributed database according to its own global directory snapshot. The technical solution of the present invention can balance the workload of each node in the distributed database, so compared with the serial method in the prior art, it can prevent the problem of unbalanced load of each node from occurring in the process of generating a consistent snapshot of the distributed database.

[0032] Based on the following detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings, those skilled in the art will become more aware of the above and other objects, advantages and features of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Hereinafter, some specific embodiments of the present invention will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings:

[0034] Figure 1 It is a schematic diagram of logical replication of a database in the related art;

[0035] Figure 2 It is a schematic diagram of a stand-alone database in the related art writing the transaction being executed into WAL;

[0036] Figure 3 It is a schematic diagram of reading WAL during the logical replication process of a stand-alone database in the related art;

[0037] Figure 4 It is a schematic diagram of a single-machine database constructing globally consistent data in the related technology;

[0038] Figure 5 It is a schematic diagram of a distributed database constructing globally consistent data in related technologies;

[0039] Figure 6 It is a schematic diagram of the work of constructing globally consistent data in a distributed database in related technologies;

[0040] Figure 7 is a schematic flow chart of a method for generating a consistent snapshot of a distributed database according to an embodiment of the present invention;

[0041] Figure 8 is a schematic diagram of converting a consistent snapshot of a distributed database from a serial construction mode to a parallel construction mode according to an embodiment of the present invention;

[0042] Fig. 9 is a schematic diagram of a work of constructing a consistent snapshot of a distributed database in a generation method according to an embodiment of the present invention;

[0043] Fig.10 It is a schematic diagram showing the dependency relationship between directory snapshots of nodes in a distributed database in a generation method according to an embodiment of the present invention;

[0044] Fig.11 is a schematic diagram of generating a consistent snapshot of a distributed database in a generation method according to an embodiment of the present invention;

[0045] Fig.12 is a schematic flow chart of generating a global directory snapshot of a distributed database in a construction method according to an embodiment of the present invention;

[0046] Fig.13 is a schematic flow chart of a method for generating a consistent snapshot of a distributed database according to another embodiment of the present invention;

[0047] Fig.14 is a schematic diagram of a distributed database constructing a consistent snapshot in a generation method according to another embodiment of the present invention;

[0048] Fig.15 is a schematic diagram of a computer program product according to an embodiment of the present invention;

[0049] Fig.16 is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention; and

[0050] Fig.17 is a schematic diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In the prior art, the global consistency of database data is achieved through the logical replication function, where the logical replication function breaks the limitation that only data replication is allowed through the database, and is mainly applied to data synchronization between heterogeneous databases and different versions of the same structure. The key point of the logical replication function is to construct a directory snapshot used for logical decoding, a consistency snapshot used for stock replication, and a log sequence number used for incremental replication. The log sequence number refers to the sequence number of the database's WAL (Write-Ahead Logging). Figure 1As shown, Txn102, Txn103, Txn104, Txn105, and Txn106 represent transactions numbered 102, 103, 104, 105, and 106, respectively; operator1 represents the first operation in the corresponding transaction, and operator2 represents the second operation in the corresponding transaction, for example, Txn102: operator1 represents the first operation in Txn102; Committed indicates that the corresponding transaction has been executed; LSN represents the log sequence number of WAL. From the transaction perspective, the database may have running transactions at any time, and from the WAL perspective, the database will append in the order of operations of all transactions. What logical replication needs to do is to generate a consistent snapshot of the database by traversing the WAL to ensure seamless connection between the existing data and incremental data in the database.

[0052] For a stand-alone database, the prerequisites for implementing stand-alone logical replication include: the historical data version that the logical replication depends on will not be deleted or recycled, and the dependent write-ahead log will not be deleted or recycled. The implementation method of stand-alone database logical replication includes: first, the main process of the database logical replication writes the running transactions in the database memory to the WAL, such as Figure 2 As shown, Txn1, Txn2, Txn3, and Txn4 represent the first transaction, the second transaction, the third transaction, and the fourth transaction, respectively. Begin indicates that the corresponding transaction starts to execute, and commit indicates that the corresponding transaction is completed. Then the main process reads and parses the pre-write log to generate a consistent snapshot of the database, as shown in Figure 3 As shown, Record1Running Txns indicates that the running transaction Txns is recorded in the WAL record Record1, Record2Txn1operator1 indicates that the first operation operator1 of the transaction Txn1 is recorded in the WAL record Record2, Record3 Txn2 operator2 indicates that the second operation operator2 of the transaction Txn2 is recorded in the WAL record Record2, and Record4 Running Txns indicates that the running transaction Txns is recorded in the WAL record Record4.

[0053] The process of generating a consistent snapshot based on WAL can be divided into four stages, namely START_SNPASHOT, BUILDING_SNAPSHOT, FULL_SNAPSHOT and CONSISTENT_SNAPSHOT. START_SNPASHOT is the beginning of the logical replication construction. At this time, the WAL of the database begins to be protected and will not be recycled; BUILDING_SNAPSHOT is the stage where the first RunningTxns is discovered and the dependent construction logic begins to enter. Since the WAL of some transactions is not protected, the executing transactions are replicated; FULL_SNAPSHOT is the stage where the first directory snapshot has been constructed. At this time, the executing transactions still cannot be used, but can be used as data in the directory snapshot; CONSISTENT_SNAPSHOT is the stage where the final consistent snapshot and consistency point of the database are constructed.

[0054] by Figure 4 Take the application scenario shown as an example, where T1, T2, T3, T4, T5 and T6 are transaction numbers, and ChekPoint is the checkpoint in WAL; Running_XAXTS T1: T2 indicates that the transactions being executed include transactions T1 and T2, and Running_XAXTS T3: T4 indicates that the transactions being executed include transactions T3 and T4, T1 Commit indicates that transaction T1 is completed, T2 Commit indicates that transaction T2 is completed, T3 Commit indicates that transaction T3 is completed, and T4 Commit indicates that transaction T4 is completed. Logical replication begins in the START_SNPASHOT stage; in the BUILDING_SNAPSHOT stage, it can be found that the transactions being executed include transactions T1 and T2, and the dependency construction logic begins at this time; in the FULL_SNAPSHOT stage, the first directory snapshot of the database is constructed; finally, in the CONSISTENT_SNAPSHOT stage, the consistency snapshot and consistency point of the database can be generated based on the constructed directory snapshot.

[0055] Distributed databases are different from stand-alone databases. Stand-alone databases only have one node, so the global order of their database operations can be guaranteed by the log sequence number of the write-ahead log. The log sequence number of the write-ahead log reflects the execution order of transactions in the database in a monotonically increasing manner. However, each node in the database cluster maintains its own write-ahead log separately, and the log sequence numbers of different nodes are not comparable. Therefore, it is necessary to introduce a unique global sequence number and use the global sequence number to compare the execution order of transactions on different nodes. On the other hand, each node in a distributed database can perform read and write operations, so any node can receive concurrent transactions. In the concurrent control design of a distributed database, each node has its own independent transaction segment, and different transaction segments are used to distinguish which node has changed the data.

[0056] In the prior art, the distributed database uses a serial method to generate a consistent snapshot. This method is based on the logical replication of a single-machine database and the introduction of a global sequence number and transaction segments of each node in the distributed database. The global snapshot of the distributed database is constructed according to the single-machine snapshot of each node, and each node has its own separate transaction segment, namely:

[0057]

[0058] Among them, SnapShot represents the global directory snapshot of the distributed database, Node1, Node2, Node3 and Node4 are the first node, the second node, the third node and the fourth node in the distributed database respectively, xmin1 represents the smallest transaction number in the transaction segment of the first node Node1, xmax1 represents the largest transaction number in the transaction segment of the first node Node1, and active txns1 represents the transaction number being executed in the transaction segment of the first node Node1. xmin2 represents the smallest transaction number in the transaction segment of the second node Node2, xmax2 represents the largest transaction number in the transaction segment of the second node Node2, and active txns2 represents the transaction number being executed in the transaction segment of the second node Node2. xmin3 represents the smallest transaction number in the transaction segment of the third node Node3, xmax3 represents the largest transaction number in the transaction segment of the third node Node3, and active txns3 represents the transaction number being executed in the transaction segment of the third node Node3. xmin4 represents the smallest transaction number in the transaction segment of the fourth node Node4, xmax4 represents the largest transaction number in the transaction segment of the fourth node Node4, and active txns4 represents the transaction number being executed in the transaction segment of the fourth node Node4.

[0059] Then the log sequence number of each node in the distributed database is changed to a global sequence number, and each node has its own WAL:

[0060]

[0061] Among them, LSN is the global sequence number of the distributed database, LSN1 is the log sequence number of the first node Node1, LSN2 is the log sequence number of the second node Node2, LSN3 is the log sequence number of the third node Node3, and LSN4 is the log sequence number of the fourth node Node4.

[0062] In the process of completing the global logical replication initialization by the distributed database serial construction method, the transformation process of each state is no longer just waiting for the transaction running on this node, but waiting for the transaction running on all nodes in the distributed database. Through the stable switching of the distributed database state, the global directory snapshot, consistency snapshot and consistency point of the distributed database are gradually generated.

[0063] by Figure 5 Taking the application scenario shown as an example, in the BUILDING_SNAPSHOT stage, not only the transactions T1 and T2 being executed by the current node need to be obtained, but also the transactions T101, T102, T201, T202, T201, and T302 being executed by other nodes need to be obtained. Accordingly, in the FULL_SNAPSHOT stage, not only the transactions T3 and T4 being executed by the current node need to be obtained, but also the transactions T101, T102, T201, T202, T201, and T302 being executed by other nodes need to be obtained.

[0064] Although the serial method of building a consistent snapshot of a distributed database is logically complete, it has a major drawback, that is, all tasks in the global logical replication initialization process are completed by one node. Assume that there are four nodes in the distributed database, such as Figure 6 As shown, the WALs of the first node, the second node, the third node and the fourth node are the first WAL, the second WAL, the third WAL and the fourth WAL respectively. During the operation, the main process of the node receiving the user request sorts the operations in all WAL logs according to the global sequence number and builds a consistent snapshot of the distributed database. Therefore, the serial method will concentrate the workload of the distributed database to generate consistent data on the node receiving the user request, resulting in an unbalanced load on each node of the distributed database.

[0065] In order to solve the above technical problems, the present invention provides a method for generating a consistent snapshot of a distributed database. The process of the generation method is as follows: Figure 7As shown, each node generates a consistent snapshot of the distributed database in parallel to complete the initialization of the distributed database logical replication. Since the parallel method can reduce the burden of a single node operation, it can prevent the distributed database from having a load imbalance problem.

[0066] exist Figure 7 In the process shown, the method for generating a consistent snapshot of a distributed database of the present invention may generally include:

[0067] Step S102: Each node of the distributed database obtains its own directory snapshot and other directory snapshots of other nodes respectively;

[0068] Step S104: Each node of the distributed database generates a global directory snapshot of the distributed database according to its own directory snapshot and other directory snapshots;

[0069] Step S106: Each node of the distributed database generates a consistent snapshot of the distributed database in the node according to its own global directory snapshot.

[0070] The technical solution of this embodiment is to change the method for generating consistent snapshots of distributed databases in the prior art from a serial method to a parallel method, such as Figure 8 As shown, the purpose of balancing the workload of each node in the distributed database is achieved.

[0071] In the above step S102, the directory snapshot of each node in the distributed database is a directory snapshot generated by each node in the distributed database according to the transaction executed by itself. After each node in the distributed database completes the initialization of logical replication, each node can only guarantee to generate its own directory snapshot according to its own data. Therefore, if each node wants to construct a global directory snapshot of the distributed database, it is also necessary to solve the problem of directory snapshot dependency between nodes.

[0072] In this embodiment, each node of the distributed database obtains the transaction execution status of other nodes by obtaining snapshots of other directories of other nodes. Fig. 9 As shown, the first main process, the second main process, the third main process and the fourth main process are the main processes that execute the logical replication function in the first node, the second node, the third node and the fourth node of the distributed database respectively. The main process of each node in the distributed database generates a snapshot of its own directory according to the transaction executed by its own node, and obtains snapshots of other directories of other nodes from other nodes.

[0073] In the above step S104, during the initialization of the logical replication of the distributed database, a single node needs to rely on other directory snapshots of other nodes during the logic decoding, and each node completes the initialization of the logical replication independently, ignoring the association between the directory snapshots of each node. Therefore, after obtaining other directory snapshots of other nodes, each node of the distributed database can generate a global directory snapshot of the distributed database according to the dependency relationship between the nodes.

[0074] by Fig.10 Taking the application scenario shown as an example, the DML transaction Txn of the first node Node1 in the distributed database needs to rely on the DDL transaction Txn of the second node Node2, where DML (Data Manipulation Language) is a subset of structured query language, used to manage and operate data in relational databases, including data insertion, deletion, update and other operations; DDL (Data Definition Language) is also a subset of structured query language, used to define and manage data structures or patterns in databases, and its statements are usually used to create, modify and delete database objects, such as tables, indexes, views, stored procedures, triggers and the database itself. Therefore, during the logical replication initialization process of the distributed database on the first node Node1, it is necessary to generate a global directory snapshot of the distributed database based on the dependency relationship between its own transaction Txn and the transaction Txn of the second node Node2.

[0075] In the above step S106, after generating the global directory snapshot of the distributed database, each node of the distributed database can obtain the execution status of the transaction in the distributed database according to the respective global directory snapshots on the local node to generate the consistency snapshot of the distributed database.

[0076] A consistent snapshot of a distributed database refers to a read-only copy of the database at a specific point in time, which records the state of the distributed database at that specific point in time and ensures that the distributed database can be restored to the state at that specific point in time when needed.

[0077] According to the above content, the technical solution of this embodiment adopts a parallel method to generate a consistent snapshot of a distributed database, that is, the work of generating a consistent snapshot is no longer concentrated on one node of the distributed database, but each node of the distributed database generates a global directory snapshot of the distributed database respectively, and then generates a consistent snapshot of the distributed database according to its own global directory snapshot. The technical solution of the present invention can balance the workload of each node in the distributed database, so compared with the serial method in the prior art, it can prevent the problem of unbalanced load of each node in the distributed database during the process of generating a consistent snapshot.

[0078] In some embodiments of the present invention, in the above step S104, after each node of the distributed database generates a global directory snapshot of the distributed database according to the directory snapshot of each node, the following further includes:

[0079] Each node in the distributed database generates a global consistency point of the distributed database at its own node based on its own global directory snapshot.

[0080] The consistency point of a distributed database is the time point at which the distributed database generates a consistent snapshot. Based on the consistent snapshot and consistency point of the distributed database, an accurate restoration of the data in the distributed database can be generated.

[0081] like Fig.11 As shown, the global sequence number of the second node Node2 in the distributed database is greater than the global sequence number of Node1 in the first stage, so the first node Node1 needs to be aligned according to the second node Node2. The old consistency point of the first node Node1 is the consistency point before the alignment operation, and the new consistency point is the consistency point after the alignment operation. Since the data before the consistency point is the stock copy, and the data after the consistency point is the incremental copy, the "seamless connection" of the stock copy and the incremental copy can be achieved in the logical replication process.

[0082] When each node in a distributed database generates a global directory snapshot, it is not just a simple logical addition of its own directory snapshot and other directory snapshots, but also needs to resolve the transaction dependencies of each node. Take the following application scenario as an example:

[0083] RAC Node1 Txn1: CREATE TABLE t(a INT); / / The transaction Txn1 executed by the first node Node1 of the distributed database RAC is to create a data table t, and the data table t has a data column named a, and the data type of the data column a is integer;

[0084] RAC Node2 Txn2: INSERT INTO t VALUES(100); / / The transaction Txn2 executed by the second node Node2 of the distributed database RAC is to insert a row of data into the data table t. The value of the row of data in the data column a is 100.

[0085] In this application scenario, in the process of obtaining the global directory snapshot of the distributed database based on the directory snapshots of the first node Node1 and the second node Node2, since the transaction Txn2 executed by the second node Node2 depends on the transaction Txn1 executed by the first node Node1, the consistency snapshot generated based on the global directory snapshot needs to ensure that the transaction Txn1 does not undergo incremental replication and the transaction Txn2 does not undergo stock replication.

[0086] To resolve the transaction dependency of the above nodes, in some embodiments of the present invention, in the above step S104, after each node of the distributed database generates a global directory snapshot of the distributed database according to its own directory snapshot and other directory snapshots, the following steps are further included:

[0087] Each node of the distributed database obtains the global directory snapshot of each other node respectively, and generates a new global directory snapshot according to the global directory snapshot of each node respectively.

[0088] In the above step S104, each node of the distributed database generates a global directory snapshot based on each directory snapshot, which may contain missing information, and the accuracy and reliability of the global directory snapshot generated on each node cannot be guaranteed.

[0089] Therefore, after obtaining the global directory snapshot, each node of the distributed database in this embodiment updates its own global directory snapshot a set number of times, and during the update process, each node in the distributed database generates a new global directory snapshot based on the global directory snapshot of each node, thereby resolving the transaction dependencies of each node in the distributed database, so that the updated global directory snapshot can include more transaction execution information, thereby achieving the purpose of improving the accuracy and reliability of the global directory snapshot generated by each node in the distributed database.

[0090] In some embodiments of the present invention, a coordination node is provided in the distributed database. The coordination node is a key component in the distributed database architecture and is responsible for parsing and forwarding SQL statements, acting as a role similar to that of an agent to ensure that data query and data processing tasks can be executed efficiently and accurately.

[0091] In this embodiment, after each node of the distributed database generates a global directory snapshot of the distributed database, the following steps are further performed:

[0092] Each node of the distributed database sends the global directory snapshot generated by itself to the coordination node of the distributed database;

[0093] Accordingly, the method for each node of the above-mentioned distributed database to obtain the global directory snapshot of each other node includes:

[0094] Each node of the distributed database sends first preset request information to the coordination node respectively to obtain global directory snapshots of other nodes.

[0095] According to the technical solution of this embodiment, each node in the distributed database can quickly obtain the global directory snapshots of other nodes through the coordination node, thereby improving the work efficiency of constructing the consistent snapshot of the distributed database.

[0096] In some embodiments of the present invention, in the above step S104, each node of the distributed database generates a global directory snapshot of the distributed database according to the directory snapshot of each node. Fig.11 As shown, the following steps are included:

[0097] Step S112: Each node of the distributed database obtains the global sequence number of the write-ahead log of other nodes respectively, and determines whether its own global sequence number is the maximum global sequence number;

[0098] If yes, execute step S116; if no, execute step S114;

[0099] Step S114: Use the other directory snapshots with the largest global sequence number to align the own directory snapshot, so as to generate a global directory snapshot in the local node.

[0100] Step S116: Generate a global directory snapshot based on its own directory snapshot.

[0101] In the above step S112, each node of the distributed database can obtain the global sequence number of the write-ahead log of each other node according to other directory snapshots of other nodes.

[0102] Since the global sequence number of each node's write-ahead log is unique and increments as each node generates a write-ahead log, the node with the largest global sequence number has the latest transactions recorded in its own directory snapshot. In the above step S114, the self-directory snapshot is aligned with other directory snapshots with the largest global sequence number to ensure that the generated global directory snapshot contains all transaction information.

[0103] In this embodiment, aligning one's own directory snapshot with other directory snapshots with the largest global sequence number means merging event information in other directory snapshots of other nodes into one's own directory snapshot according to the global sequence numbers of the pre-write logs of other nodes, until one's own directory snapshot includes event information in other directory snapshots with the largest global sequence number.

[0104] In the above step S116, generating a global directory snapshot based on its own directory snapshot means that each node of the distributed database merges the event information in other directory snapshots of other nodes into its own directory snapshot according to the global sequence number of the pre-write log of each other node, so as to generate a global directory snapshot.

[0105] The technical solution of this embodiment can align each node in the distributed database with other directory snapshots corresponding to the maximum global sequence number to ensure the accuracy of the global directory snapshot of the distributed database generated at this node.

[0106] In some embodiments of the present invention, the method in step S102 in which each node of the distributed database obtains its own directory snapshot includes:

[0107] Each node of the distributed database obtains its own write-ahead log and generates its own directory snapshot based on its own write-ahead log.

[0108] Since the write-ahead log of each node of the distributed database records its own transaction execution information, each node of the distributed database can accurately and quickly generate its own directory snapshot according to its own write-ahead log.

[0109] In some embodiments of the present invention, after each node of the distributed database obtains its own directory snapshot in the above step S102, the following further includes:

[0110] Each node in the distributed database sends its own directory snapshot to the coordination node of the distributed database.

[0111] Accordingly, in the above step S102, the method for each node of the distributed database to obtain other directory snapshots of other nodes includes:

[0112] Each node of the distributed database sends second preset request information to the coordination node of the distributed database respectively, so as to obtain other directory snapshots of other nodes from the coordination node respectively.

[0113] According to the technical solution of this embodiment, each node in the distributed database can quickly obtain other directory snapshots of other nodes through the coordination node, thereby improving the work efficiency of constructing consistent snapshots of the distributed database.

[0114] In some embodiments of the present invention, a method for constructing global consistency data in a distributed database is as follows: Fig.13 As shown, the following steps are included:

[0115] Step S202: Each node of the distributed database obtains its own write-ahead log, generates its own directory snapshot according to its own write-ahead log, and then sends its own directory snapshot to the coordination node of the distributed database;

[0116] Step S204: each node of the distributed database sends a second preset request message to the coordinating node respectively, so as to obtain other directory snapshots of other nodes from the coordinating node;

[0117] Step S206: Each node of the distributed database obtains the global sequence number of the write-ahead log of other nodes respectively, and determines whether its own global sequence number is the maximum global sequence number;

[0118] If not, execute step S208; if yes, execute step S210;

[0119] Step S208: Use the other directory snapshots with the largest global sequence number to align the own directory snapshot, so as to generate a global directory snapshot in the local node.

[0120] Step S210: Generate its own global directory snapshot based on its own directory snapshot;

[0121] Step S212: each node of the distributed database sends its own global directory snapshot to the coordinating node, and sends first preset request information to the coordinating node to obtain the global directory snapshots of other nodes from the coordinating node;

[0122] Step S214: each node of the distributed database generates a new global directory snapshot according to the global directory snapshot of each node;

[0123] Step S216: Each node of the distributed database generates a consistency snapshot and a consistency point of the distributed database on its own node according to its own global directory snapshot.

[0124] In this embodiment, a parallel method is adopted, where each node of the distributed database generates a consistent snapshot and a consistent point of the distributed database at its own node. Fig.14 As shown, each node obtains the directory snapshot of other nodes to solve the dependency problem of the directory snapshot of each node, and each node updates its own global directory snapshot to solve the transaction dependency problem in the distributed database.

[0125] The flow chart provided by the present embodiment is not intended to indicate that the operation of the method will be performed in any particular order, or that all operations of the method are included in all every case. In addition, each of the above methods may include additional operations. Within the scope of the technical thinking provided by the present embodiment method, additional changes may be made to the above method.

[0126] It should be understood that in some embodiments, each part can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.

[0127] This embodiment also provides a computer program product 10 , a computer readable storage medium 20 , and a computer device 30 . Fig.15 is a schematic diagram of a computer program product 10 according to an embodiment of the present invention, Fig.16 is a schematic diagram of a computer-readable storage medium 20 according to an embodiment of the present invention, Fig.17 is a schematic diagram of a computer device 30 according to an embodiment of the present invention. The computer program product 10 includes a computer program 11, which implements the steps of the method for generating a consistent snapshot of a distributed database described above when the computer program 11 is executed by a processor 32. The computer-readable storage medium 20 stores the computer program 11, which implements the steps of the method for generating a consistent snapshot of a distributed database described above in any of the embodiments when the computer program 11 is executed by the processor 32. The computer device 30 may include a memory 31, a processor 32, and a computer program 11 stored in the memory 31 and running on the processor 32.

[0128] The computer program 11 for performing the operation of the present invention may be an assembly instruction, an instruction set architecture (ISA) instruction, a machine instruction, a machine-related instruction, a microcode, a firmware instruction, a state setting data, a configuration data of an integrated circuit, or a source code or an object code written in any combination of one or more programming languages ​​and process programming languages. The computer program 11 may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider via the Internet). In some embodiments, in order to perform various aspects of the present invention, an electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA) may execute computer-readable program instructions by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit.

[0129] In the description of this embodiment, the computer program product 10 is a related product including the computer program 11 .

[0130] For the purpose of the description of the present embodiment, the computer readable storage medium 20 is a tangible device capable of retaining and storing the computer program 11, which can be any device that can contain, store, communicate, propagate or use the computer program 11 for an instruction execution system, device or apparatus or in conjunction with these instruction execution systems, devices or apparatuses. More specific examples (a non-exhaustive list) of the computer readable storage medium 20 include the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, and any suitable combination of the above.

[0131] The computer device 30 may be, for example, a server, a desktop computer, a notebook computer, a tablet computer, or a smart phone. In some examples, the computer device 30 may be a cloud computing node. The computer device 30 may be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, a program module may include routines, programs, target programs, components, logic, data structures, etc. that perform specific tasks or implement specific abstract data types. The computer device 30 may be implemented in a distributed cloud computing environment where remote processing devices linked via a communication network perform tasks. In a distributed cloud computing environment, program modules may be located on a local or remote computing system storage medium including a storage device.

[0132] The computer device 30 may include a processor 32 adapted to execute stored instructions, and a memory 31 providing temporary storage space for the operation of the instructions during operation. The processor 32 may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. The memory 31 may include a random access memory (RAM), a read-only memory, a flash memory, or any other suitable storage system.

[0133] The computer device 30 may also include a network adapter / interface and an input / output (I / O) interface. The I / O interface allows data to be input and output with external devices that may be connected to the computer device. The network adapter / interface may provide communication between the computer device and a network, which is typically shown as a communication network.

[0134] At this point, those skilled in the art should recognize that, although multiple exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications that conform to the principles of the present invention can still be directly determined or derived based on the content disclosed in the present invention without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be understood and recognized as covering all these other variations or modifications.

Claims

1. A method for generating a consistent snapshot of a distributed database, characterized in that: include: Each node of the distributed database obtains its own directory snapshot and other directory snapshots of other nodes in the distributed database; Each of the nodes generates a global directory snapshot of the distributed database according to each of the own directory snapshots and the other directory snapshots, and generates a consistent snapshot of the distributed database in the node according to the global directory snapshot.

2. The generation method according to claim 1, characterized in that: The step of each of the nodes generating a global directory snapshot of the distributed database according to each of the own directory snapshots and the other directory snapshots comprises: Each of the nodes obtains the global sequence number of the write-ahead log of other nodes respectively, and determines whether the global sequence number of its own write-ahead log is the maximum global sequence number; If not, the self-directory snapshot is aligned with another directory snapshot having the largest global sequence number to generate the global directory snapshot.

3. The generation method according to claim 1, characterized in that: After the step of each of the nodes generating a global directory snapshot of the distributed database according to each of the own directory snapshots and the other directory snapshots, the method further includes: Each of the nodes obtains the global directory snapshots of other nodes respectively, and generates a new global directory snapshot respectively according to each of the global directory snapshots.

4. The generation method according to claim 3, characterized in that: After the step of each of the nodes generating a global directory snapshot of the distributed database according to each of the own directory snapshots and the other directory snapshots, the method further includes: Each of the nodes sends its own global directory snapshot to the coordination node of the distributed database; The step of each of the nodes obtaining a global directory snapshot of other nodes respectively includes: Each of the nodes sends first preset request information to the coordinating node respectively, so as to obtain global directory snapshots of other nodes from the coordinating node.

5. The generation method according to claim 1, characterized in that: The step of each node of the distributed database obtaining its own directory snapshot comprises: Each of the nodes obtains its own write-ahead log respectively, and generates its own directory snapshot respectively according to its own write-ahead log.

6. The generation method according to claim 1, characterized in that: The step of each node of the distributed database obtaining its own directory snapshot and other directory snapshots of other nodes in the distributed database includes: After acquiring the snapshot of its own directory, each of the nodes sends the snapshot of its own directory to the coordination node of the distributed database; and Each of the nodes sends second preset request information to the coordinating node respectively, so as to obtain the other directory snapshots from the coordinating node.

7. The generation method according to claim 1, characterized in that: After the step of each of the nodes generating a global directory snapshot of the distributed database according to each of the own directory snapshots and the other directory snapshots, the method further includes: Each of the nodes generates a consistency point of the distributed database at the node according to its own global directory snapshot.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the generation method according to any one of claims 1 to 7 are implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the generation method according to any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the generation method according to any one of claims 1 to 7.

Images