A database server for implementing a distributed database

Abstract

A database server for implementing a distributed database, which can access a data storage device that stores a first copy of a database managed by the server. The server has a processor and one or more communication interfaces, which are used to communicate with a client and with a second database server that manages a second copy of the database. The database server implements changes to the database by the following steps: transmitting the changes to the second database server; storing a temporary record of the changes in the first copy of the database; when receiving a confirmation from the second database server that the second database server has stored the record of the changes in the second copy of the database: (i) instructing the second database server to store a permanent record of the changes in the second copy of the database, and (ii) storing the permanent record of the changes in the first copy of the database.

Description

Technical Field

[0001] The present invention relates to updating information stored in a database. Background Art

[0002] A standard database transaction is a sequence of database statements (operations) that indicate changes to the state of the database, and a commit that permanently applies all changes in the transaction to the database. Figure 1 A state diagram showing this functionality is shown.

[0003] It is common for databases to be distributed across multiple servers. Each server maintains its own copy of the database. This has the following advantages: clients can access any server to query the database; if one of the servers fails, the database remains available and the failed server is restored when appropriate. Clients communicate with the servers to make queries to the database. Figure 2 The architecture of such a system is shown.

[0004] To ensure that it is clear which copy of the database is authoritative, one of the servers is usually made the primary server and the other or each other server is made a backup server. The copy of the database held by the primary server is considered the final copy. The protocol that the servers use to perform database updates can give the primary server a special role in performing database updates.

[0005] Figure 3 A process for making updates to a distributed database is shown. In this example, there is a primary server, a standby server, and a client with write access to the database. The client sends a message to the primary server requesting that a change be made to the database. The primary server sends a message to the standby server requesting that the change be made. The standby server makes the change in its copy of the database and sends a confirmation to the primary server. When the primary server receives confirmation from the standby server that the change has been made in the standby server's copy of the database, the primary server knows that it is safe to make the change in its own copy of the database without risking compromising the integrity of the system. The primary server makes the change in its own copy of the database and then sends a confirmation to the client that the change has been made. One problem with this approach is that determining that a "final" change has been made to the database typically requires writing to long-term storage, which can involve relatively long delays. For example, typical latency for writing to disk storage can be as high as 30ms. In contrast, network latency can be in the range of 1ms or less.

[0006] Figure 4 A possible solution to this problem is shown in Figure 2In an approach where the primary server determines that changes to its copy of the database have been (finally) committed before it has received confirmation from the standby that the standby has made the changes. One problem with this approach is that if the primary fails after it has committed the changes (e.g., by writing to its log), but the message is still queued in the network and has not yet reached the standby, the state of the standby will be inconsistent with that of the primary, and the changes may be lost if the system is subsequently restored from a standby that has not yet committed the changes.

[0007] A more robust method is needed to make changes in the database with the potential to reduce latency. Summary of the invention

[0008] Said object is achieved by the embodiments of the invention described in the attached independent claims. Advantageous implementations of the embodiments of the invention are further defined in the dependent claims.

[0009] A first aspect of the present invention provides a database server for implementing a distributed database, the server having access to a data storage device storing a first copy of the database managed by the database server, and having a processor and one or more communication interfaces, the one or more communication interfaces being used to communicate with a client and with a second database server managing a second copy of the database, the database server being used to implement changes to the database transmitted from the client by: transmitting the changes to the second database server; storing a temporary record of the changes in the first copy of the database; and upon receiving confirmation from the second database server that the second database server has stored a record of the changes in the second copy of the database: (i) instructing the second database server to store a permanent record of the changes in the second copy of the database, and (ii) storing a permanent record of the changes in the first copy of the database.

[0010] For example, each standby server is capable of being promoted to function as a primary server if the previous primary server becomes corrupted or unavailable. If a server becomes corrupted or unavailable (regardless of whether it was previously serving as a primary or standby server), the server can recover its local copy of the database by communicating with the primary server. This provides a robust method of making changes to the database in a manner that reduces latency.

[0011] In an implementation of the first aspect, the database server is configured to implement the change by: receiving an indication of the change from the client, the change involving modification of one or more elements of the database; locking the elements involved in the change in the first copy of the database after the receiving step but before the step of transmitting the change to the second database server; and unlocking the elements involved in the change in the first copy of the database after the step of receiving a confirmation from the second database server that the second database server has stored a record of the change in the second copy of the database.

[0012] In an implementation of the first aspect, the database server is further configured to implement the change by: receiving an indication of the change from the client, the change involving modification of one or more elements of the database; creating a backup of the one or more elements involved in the change that existed before the change; and deleting the backup after the step of receiving confirmation from the second database server that the second database server has stored a record of the change in the second copy of the database.

[0013] In an implementation of the first aspect, the database server is further configured to transmit the change to the second database server by sending a message to the second database server, wherein the only change to the second database indicated by the message is the change transmitted from the client.

[0014] In an implementation of the first aspect, the database server is further configured to: after receiving a confirmation from the second database server that the second database server has stored a plurality of changed records in the second copy of the database, instruct the second database server to store a permanent record of the plurality of changes by sending a single message or a group of associated messages to the second database server.

[0015] In an implementation of the first aspect, the database server is further configured to: after receiving a confirmation from the second database server that the second database server has stored the changed record in the second copy of the database, send a confirmation of the change to the client.

[0016] In an implementation of the first aspect, the database server is further used to: when the server has been used as a primary server and is subsequently downgraded to be used as a standby server, roll back in the first copy of the database changes that are only designated as temporary records, but do not roll back changes that are designated as permanent records.

[0017] In an implementation of the first aspect, the database server is further configured to: when the server has been used as a primary server and fails without a backup server available, and is subsequently restored to be the primary server, convert the temporary record into a permanent record.

[0018] In an implementation of the first aspect, the database server has a recovery operation mode, in which, when the integrity of the first copy of the database is compromised, the server automatically stores a permanent record of all changes in the first copy of the database or its replacement version, and only a temporary record of all changes is stored in the first copy of the database.

[0019] In an implementation manner of the first aspect, the database server is a primary database server, and the second database server is a standby database server.

[0020] In an implementation of the first aspect, the database server is capable of operating as a standby database server, and is configured to, when operating as a standby database server, implement changes to the database transmitted from a primary database server by: receiving the changes from the primary database server; storing a temporary record of the changes in the first copy of the database; and sending a confirmation of the changes to the primary database server.

[0021] In an implementation of the first aspect, the database server is further configured, when used as a standby database server, to: upon receiving an instruction from the primary database server to store a permanent record of one or more changes in the second copy of the database, store the permanent record of the one or more changes in the first copy of the database.

[0022] In an implementation of the first aspect, the server is further used, when used as a standby database server: after receiving the changes from the primary database server and before storing a temporary record of the changes, to lock elements of the database related to the changes; and after storing the temporary record of the changes, to unlock these elements.

[0023] In an implementation of the first aspect, the server is further configured to transmit the change to the second database server before storing a temporary record of the change in the first copy of the database.

[0024] A second aspect of the present invention provides a method for implementing changes to a database through a database server, wherein the database server can access a data storage device that stores a first copy of the database, the method comprising: receiving an indication of changes to the database from a client; transmitting the changes to a second database server; storing a temporary record of the changes in the first copy of the database; receiving a confirmation from the second database server that the second database server has stored a record of the changes in the second copy of the database; after the receiving step: (i) instructing the second database server to store a permanent record of the changes in the second copy of the database, and (ii) storing a permanent record of the changes in the first copy of the database.

[0025] In an implementation of the second aspect, the method further comprises, after the receiving step, before instructing the second database server to store a permanent record of the change in the second copy of the database, sending a confirmation of the change to the client. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The present invention will now be described by way of example with reference to the accompanying drawings. In the accompanying drawings:

[0027] Figure 1 shows a state diagram for making changes to a database;

[0028] Figure 2 shows the general structure of a distributed database;

[0029] Figure 3 shows the message flow for making changes in a distributed database;

[0030] Figure 4 A second message flow for making changes in a distributed database is shown;

[0031] Figure 5 shows the architecture of the database server;

[0032] Figure 6 A third message flow for making changes in a distributed database is shown;

[0033] Figure 7 Shows the implementation Figure 6 The state diagram of the system of the process;

[0034] Figure 8 Shows the implementation Figure 6 The common functions of the system of processes;

[0035] Fig. 9 is realized Figure 6Flowchart of the master server of the process;

[0036] Fig.10 is realized Figure 6 Flowchart of the standby server of the process;

[0037] Fig.11 illustrates the functionality in a multi-tier database architecture;

[0038] Fig.12 A fourth message flow for making changes in a distributed database is shown. DETAILED DESCRIPTION

[0039] Figure 2 The architecture of a distributed database system is shown. The system includes multiple servers, labeled "primary" and "standby". The servers can communicate with each other through a network. The network can be a private network or a publicly accessible network such as the Internet, or it can include parts of both types. Clients can access the servers to query the database and retrieve data from it. At least some clients can also request the server to make changes in the database. Each server has its own copy of the database. The client can contact any server to query the database. In the first case, the client can contact any server to request changes in the database, but any such request is preferentially forwarded to the primary server for processing. This allows the protocol to operate in the manner described below to avoid possible corruption of the database. In order to provide recovery capabilities, for example, if the previous primary server is damaged or unavailable, each standby server can be upgraded to serve as a primary server. If a server is damaged or unavailable (regardless of whether it was previously used as a primary server or a standby server), the server can restore its local copy of the database by communicating with the primary server. The mechanisms of these processes will be described below.

[0040] Figure 5 The architecture of the server 20 is shown. Each server includes at least one processor 21 and a local memory 22. The local memory 22 stores program code for causing the processor to perform the functions described herein in a non-transient form. Each server can access a corresponding data memory 23, 24. The data memory can be a local memory of the server or a remote memory of the server. The server stores and maintains its own copy of the database in its data memory. The data memory can include one or both of a relatively low-latency memory 23 (such as a random access memory implemented on an integrated circuit) and a relatively high-latency memory 24 (such as a hard disk system using a rotating disk). The server has a communication interface 25 for communicating with clients and other servers.

[0041] The mechanisms by which the server can perform operations will now be described.

[0042] When a change is to be made to a database, the format of the data to be changed will depend on the format of the database. In one example, a database may store multiple data blocks called rows, and each row may include data fields corresponding to multiple columns, each column corresponding to a data category. For example, if the database is related to train operations, there may be columns corresponding to arrival times and departure times, and multiple rows: one row corresponds to each train service in many train services. In another example, the data may be stored in conceptually discrete blocks or spots. Changes to the database may be changes to existing data (e.g., update operations) or additions of new data (e.g., insert operations). Changes may be transmitted to the database system by passing new data to be stored (e.g., in the form of data tuples) to the database system and / or by passing logic that can be applied to currently stored data to cause changes to the database system. An example of the latter case is that the logic is passed to the database to indicate the records to be changed (e.g., all rows with arrival times after 12:34), or to indicate the changes to be made (e.g., increase the arrival time by 10 minutes). The database may implement SQL or any other suitable database protocol.

[0043] Figure 6 The mechanism and signal flow for making changes to the database are shown.

[0044] exist Figure 6 In the example above, there is one client, one primary server, and one standby server. In reality, there can be multiple clients, but only one client will initiate any particular database change, and there can be multiple standby servers. The primary and standby servers can actually have similar hardware and software, but can simply be used as primary and standby servers respectively.

[0045] The change process is as follows:

[0046] 1. A message requesting a database change, or commit, is sent from the client to the master server. A commit message indicates the changes to be made to the database.

[0047] 2. The master locks the records (tuples) that will be affected by the change in its copy of the database. Tuples are locked to prevent them from being affected by other changes that may occur in parallel with the current tuple.

[0048] 3. The primary server sends an interim or lightweight commit message to the or each standby server. A lightweight commit message indicates the changes to be made to the database. This is the same change indicated by a commit message from the client.

[0049] 4. The master server makes the changes to its local copy of the database. The master server stores an indication that the changes are temporary at this stage. This indication can be made by a flag associated with the changed data or by a record in a log maintained by the master server.

[0050] 5. After receiving the lightweight commit message from the primary, the or each standby locks the records (tuples) in its copy of the database that will be affected by the changes. The or each standby then makes the changes to its local copy of the database. The or each standby stores an indication that the changes are temporary at this stage. This indication may be made by a flag associated with the changed data or by a record in a log maintained by the primary. Upon completion of these operations, the or each standby returns a confirmation message to the primary.

[0051] 6. After receiving the confirmation message from the standby server or each standby server, the primary server sends a confirmation message of the change to the client. After receiving the confirmation, the client can consider the change to be complete, although the database considers the change to be temporary at this stage.

[0052] 7. When the master has sent an acknowledgment message to the client, the master unlocks the tuple that was locked for change in its local copy of the database.

[0053] 8. When the standby server has sent its acknowledgment message to the primary server, the standby server unlocks the tuple that was locked for change in its local copy of the database.

[0054] 9. The primary server sends a final commit message about the changes to the or each standby server. The final commit message indicates which change it relates to.

[0055] 10. The master updates its copy of the database to mark the change as complete. After this, the change is no longer considered a temporary change in its copy of the database.

[0056] 11. In response to the final commit message, the or each standby server updates its copy of the database to mark the change as completed. Thereafter, the change is no longer considered a temporary change in the database copy. After completing this operation, the standby server sends a confirmation message to the primary server.

[0057] It should be understood that at various points in the process of the primary server and the backup server performing operations in parallel, such as Figure 6 As shown, the order of the steps between the primary and standby servers is not important.

[0058] refer to Figure 6 The method described can be compared Figure 3The approach shown implements changes faster. One measure of the time it takes to implement a change is the delay between a change request from a client reaching the master server and the master server sending a reliable acknowledgment of the change to the client. Figure 6 In the method described in, a lightweight commit can be made in a memory having a lower latency than the memory to which the final commit is made. From the client's perspective, this can reduce the latency of the change operation. For example, each lightweight commit can be made in a log file or other data structure stored in a memory that has a lower latency for changes made by a processor of a corresponding server than the memory to which the final commit is made. The memory to which the lightweight commit is made can be a random access memory (RAM). The memory can be implemented on an integrated circuit. The memory to which the final commit is made can be a disk storage system, for example, a disk storage system having a rotating magnetic disk that stores data. Either system can be implemented by one or more solid-state disk drives or non-volatile memory. If a lightweight commit is implemented in a log file or other temporary storage before the final commit is made, the database state indicated by the temporary storage will be different from the state indicated in the memory to which the final commit is made. To accommodate this situation, preferably, when the client queries the database to request data from the database, the server gives priority to the database state indicated in the temporary storage rather than the state indicated in the memory to which the final commit is made.

[0059] refer to Figure 6 The described approach divides the commit process into two states: lightweight commit and final commit. Lightweight commit applies database changes / transactions durably, but the changes can still be undone by a rollback operation of the server, for example, in case the server recovers from a failure. This is described further below. Once changes have been applied in the lightweight commit phase, users cannot roll back the changes, and queries to the database will show the changes as visible as if they were made by a standard commit. Final commit makes the transaction permanent. After that, the final commit cannot be undone, even in case of failures. The sequence of final commits after lightweight commits leaves the database in the same state as if a standard commit had been implemented.

[0060] exist Figure 6 In this method, the client can use the same protocol as traditional mutation operations. The lightweight commit status and final commit status are not visible to the client.

[0061] Figure 7 Shows Figure 6 A state diagram of the transactions of a system.

[0062] Figure 8 Shows Figure 6 The logical functions of the primary server and the backup server in the method.

[0063] Fig. 9 and Fig.10 Shows the primary and backup servers for implementation Figure 6 Flowchart of operations performed by a method.

[0064] The system of the present invention can respond to failures in a variety of ways.

[0065] -If a standby server fails, it can be rebuilt based on the state of the primary server, as shown in the light commit and final commit.

[0066] -If the primary server fails, the standby server can take over the primary server. All interim (lightweight) commits in the new primary server are marked as final commits, and then its copy of the database is considered the database reference copy. The previous primary server can then be rebuilt based on the new primary server and used as a standby server. In this case, there may be multiple candidate standby servers to take over the original primary server. The states of the databases of these candidate standby servers can be compared to determine the server that has implemented the latest changes from the primary server, which can be selected as the new primary server. For ease of calculation, the standby server with the least lightweight commits waiting to be completed can be selected as the new primary server.

[0067] The serialization order saved in the master's copy of the database may be exactly the sorted list of database operations that would have been created by executing concurrent transactions against a given master database state. If the list were replayed continuously (i.e., non-concurrently) on another database that was originally in the same state as the master before the operations were implemented, the resulting state of the second database would be equivalent to the state of the master's copy of the database.

[0068] The primary server can send a final commit message individually for each change to the database. Alternatively, it may be advantageous to batch the changes together and send one final commit message to the standby server for multiple changes. This can reduce the amount of messaging between the primary and standby servers.

[0069] Some aspects of correcting node failures will now be discussed.

[0070] 1. If the client has an open transacted session, and the master node fails before the client sends its commit message, the transaction is uncommitted and is simply rolled back as is known in the art (eg, using a timeout).

[0071] 2. If the client sends a commit message to the master, but the master has not replicated the lightweight commit message, the transaction is simply rolled back.

[0072] 3. If the master has sent a lightweight commit message but has not yet received an acknowledgment of the lightweight commit message, the following situations may occur:

[0073] 3.1. If the primary server fails after recording the light commit locally, but no standby server has received the light commit, the transaction will not be committed, and the primary server will undo the lightly committed transaction as part of the recovery and resynchronization process.

[0074] 3.2. If the primary server fails after one or more standby servers have received the light commit, the most advanced standby server will be promoted to the new primary server and eventually complete the commit. During the resynchronization period, the failed primary server will receive the commit of the transaction.

[0075] 3.3. If the primary fails after recording a lightweight commit, and one or more standbys receive the lightweight commit, the most advanced standby will become the new primary and eventually complete the commit. During resynchronization, the failed primary undoes the lightweight committed transaction and will (again) receive the commit of the transaction.

[0076] 4. If the primary server confirms the commit, all standby servers have light commits and the transaction will be committed. The failed primary server undoes the lightly committed transaction and will receive the commit of the transaction (again).

[0077] When the primary server stores lightly committed changes in the log, they can be stored in binary format. The primary server can transmit the changes to the standby server in the same format.

[0078] Figure 6 The method can be embedded in a two-layer replication scheme. (See Fig.11 The first layer replicates changes synchronously, and the second layer replicates changes asynchronously. When replication is complete in the first layer, replication to the second layer is initiated.

[0079] If there are multiple standby servers, you can choose the number of change acknowledgments that the primary must receive from different standby servers before the primary implements the final commit. The number can be a number, such as 1, 2, 3, or more; a relative value, such as more than half; or a condition, such as at least one acknowledgment from each availability zone. The number can be equal to the total number of standby servers. This parameter can be set per transaction.

[0080] Fig.12The message flow in a multi-tier architecture is shown. The primary server sends a lightweight commit message to at least one standby server in tier 1 and sends an acknowledgment to the client when at least one of these standby servers has confirmed the lightweight commit in its copy of the database. The primary server then causes the second tier to make the changes. Here, the replication on tier 1 is synchronous with the client's commit request, and the replication on tier 2 is asynchronous. For clarity, Fig.12 The lock / unlock is omitted from the Figure 6 A similar method in .

[0081] One or more standby servers may not perform a hard write to the log. This will result in lower latency for operations, but in the event of a total system failure, the database may need to be restored from the last functioning primary replica.

[0082] One or more standby servers may use a consensus protocol that is not a full quorum. In the prior art, there are many examples of such protocols, such as Paxos or Raft. Depending on the properties of the protocol, the confidence in the integrity of the database may vary.

[0083] Applicants hereby disclose separately each individual feature described herein and any combination of two or more such features. With the common knowledge of those skilled in the art, such features or combinations can be implemented as a whole according to this specification, without considering whether such features or combinations of features can solve any problem disclosed herein; and without limiting the scope of the claims. This application shows that various aspects of the present invention can be composed of any such individual features or combinations of features. In view of the above description, it is obvious to those skilled in the art that various modifications can be made within the scope of the present invention.

Claims

1. A database server for implementing a distributed database, characterized in that: The server has access to a data storage storing a first copy of the database managed by the database server, and has a processor and one or more communication interfaces for communicating with a client and with a second database server managing a second copy of the database, the database server being configured to implement changes to the database transmitted from the client by: transmitting the changes to the second database server; storing a temporary record of the change in the first copy of the database; upon receiving confirmation from the second database server that the second database server has stored a record of the change in the second copy of the database: (i) instructing the second database server to store a permanent record of the change in the second copy of the database, (ii) storing a permanent record of the change in the first copy of the database; The server is used to: when the server has been used as a primary server and is subsequently downgraded to be used as a standby server, roll back in the first copy of the database changes that are only designated as temporary records, but not roll back changes that are designated as permanent records.

2. The database server according to claim 1, characterized in that: The server is used to implement the change by following the steps: receiving an indication of the change from the client, the change involving modification of one or more elements of the database; after said receiving step but before said step of transmitting said changes to said second database server, locking said elements involved in said changes in said first copy of said database; After the step of receiving a confirmation from the second database server that the second database server has stored a record of the change in the second copy of the database, the element involved in the change is unlocked in the first copy of the database.

3. The database server according to claim 1, characterized in that: The server is used to implement the change by following the steps: receiving an indication of the change from the client, the change involving modification of one or more elements of the database; creating a backup of the one or more elements involved in the change that existed before the change; After said step of receiving confirmation from said second database server that said second database server has stored a record of said change in said second copy of said database, said backup is deleted.

4. The database server according to any one of claims 1 to 3, characterized in that: The server is configured to transmit the changes to the second database server by sending a message to the second database server, the only changes to the second database indicated by the message being the changes transmitted from the client.

5. The database server according to any one of claims 1 to 3, characterized in that: The server is configured to, after receiving confirmation from the second database server that the second database server has stored a plurality of changed records in the second copy of the database, instruct the second database server to store a permanent record of these plurality of changes by sending a single message or a group of associated messages to the second database server.

6. The database server according to any one of claims 1 to 3, characterized in that: The server is configured to send a confirmation of the change to the client after receiving a confirmation from the second database server that the second database server has stored the changed record in the second copy of the database.

7. The database server according to any one of claims 1 to 3, characterized in that: The server is used to convert the temporary record into a permanent record when the server has been used as a primary server and fails when no backup server is available, and then recovers to be a primary server.

8. The database server according to any one of claims 1 to 3, characterized in that: The server has a recovery mode of operation in which, when the integrity of the first copy of the database is compromised, the server automatically stores a permanent record of all changes in the first copy of the database or an alternative version thereof, with only a temporary record of all changes being stored in the first copy of the database.

9. The database server according to any one of claims 1 to 3, characterized in that: The server is configured to store the temporary record and the permanent record in a log of the first copy of the database.

10. The database server according to any one of claims 1 to 3, characterized in that: The server is a primary database server, and the second database server is a standby database server.

11. The database server according to claim 1, characterized in that: The server is capable of operating as a standby database server and is used, when operating as a standby database server, to implement changes to the database transmitted from the primary database server by the following steps: receiving the changes from the primary database server; storing a temporary record of the change in the first copy of the database; A confirmation of the change is sent to the primary database server.

12. The database server according to claim 11, characterized in that: The server is used when used as a standby database server: Upon receiving an instruction from the primary database server to store a permanent record of one or more changes in the second copy of the database, storing the permanent record of the one or more changes in the first copy of the database.

13. The database server according to claim 11 or 12, characterized in that: The server is used when used as a standby database server: After receiving the changes from the primary database server and before storing a temporary record of the changes, locking elements of the database associated with the changes; After storing a temporary record of the changes, the elements are unlocked.

14. The database server according to any one of the preceding claims, characterized in that: The server is configured to transmit the changes to the second database server before storing a temporary record of the changes in the first copy of the database.

15. A method for implementing changes to a database by means of a database server, the database server having access to a data storage storing a first copy of the database, characterized in that: The method comprises: receiving an indication of a change to the database from a client; transmitting the changes to a second database server; storing a temporary record of the change in the first copy of the database; receiving from the second database server a confirmation that the second database server has stored a record of the change in a second copy of the database; After the receiving step: (i) instructing the second database server to store a permanent record of the change in the second copy of the database, (ii) storing a permanent record of the change in the first copy of the database; The server is used to: when the server has been used as a primary server and is subsequently downgraded to be used as a standby server, roll back in the first copy of the database changes that are only designated as temporary records, but not roll back changes that are designated as permanent records.

16. The method according to claim 15, characterized in that The method further comprises, after the receiving step, sending a confirmation of the change to the client before instructing the second database server to store a permanent record of the change in the second copy of the database.

Images