Reducing commit latency in distributed multi-version databases by reading clocks earlier

Abstract

In a distributed system, the client's transaction commit call occurs outside the transaction's lock holding period. The calculation of the transaction's timestamp information is moved to the client database, ensuring no conflicting reads or writes occur between the calculation and the acquisition of all locks for the transaction. Transactions are committed in phases, with each phase initiated by the client database. Timestamp information is added to the locks to ensure that timestamps are generated during the lock holding period. Thus, the increase in the number of network messages overlaps with the commit wait period, during which writes to the distributed database are delayed to ensure concurrency within the database.

Description

[0001] This application is a divisional application of patent application No. 201880032281.5 (PCT / US2018 / 015007), filed on January 24, 2018, entitled "Reducing Commit Waiting in Distributed Multi-Version Databases by Reading the Clock Earlier".

[0002] Cross-references to related applications

[0003] This application is a continuation of U.S. Patent Application No. 15 / 649,920, filed July 14, 2017, which claims the benefit of U.S. Provisional Patent Application No. 62 / 506,177, filed May 15, 2017, the disclosures of which are incorporated herein by reference. Background Technology

[0004] In database systems, the term "concurrency control" refers to algorithms used to ensure reasonable semantics in a database, even in the presence of concurrency. Multi-version concurrency control (MVCC) algorithms store multiple versions of a given data row (one at a time) to enable greater concurrency. Ideally, a distributed database is semantically equivalent to a single-machine database; concurrency control algorithms that guarantee such semantics are called "externally consistent." In an externally consistent MVCC distributed database, past consistent reads can be completed across the entire database without maintaining locks. Current reads and writes, such as those within read-write transactions, use local synchronization between reads and writes of each object in the database; strict two-phase locking and commit wait (where transactions delay committing until the transaction's timestamp has passed) guarantee external consistency.

[0005] Some distributed databases provide implementations of a virtual global clock that monotonically increases and reflects real-time. Because the clock value inherently contains observational uncertainty, and because events in a program are not instantaneous, clients cannot read the precise value of the clock at any given point in time. Instead, clients can be provided with functions that return an open interval of time, where the lower bound of the interval is below the value of the global clock at the instant the function completes execution, and the upper bound is above the value of the global clock at the instant the function begins execution. The function should guarantee that the interval is non-empty. Because time uncertainty fluctuates over time, the width of the returned time interval will also fluctuate. A commit waits for the upper bound of such an interval and waits until that time has elapsed.

[0006] Existing solutions perform time-based synchronization while maintaining user-level locking on the server. That is, when an object is written to the database, the server commits and waits, while maintaining implementation-level locking that protects exclusive access to the object being written. Summary of the Invention

[0007] When committing transactions in a distributed system, commit wait time can be reduced by using an earlier timestamp. Specifically, the timestamp is calculated by the client library before acquiring the lock used to communicate the transaction. The timestamp is calculated based on the range of uncertainty between clocks in the distributed system.

[0008] One aspect of this disclosure provides a method in which a client in a distributed system receives a transaction to be committed, calculates a current time plus a provisional value corresponding to a boundary of uncertainty of the clock in the distributed system, the clock including at least a client clock on the client and a server clock on the server, and initiates a commit of the transaction based on the calculated provisional value, wherein initiating the commit of the transaction occurs before acquiring all locks. The method may further include acquiring locks for the transaction, performing writes to the transaction, committing waits, and releasing the locks. Furthermore, the method may include adding timestamp information to the locks, wherein the timestamp information is used to ensure that the timestamp of the transaction is within the lock holding interval of the transaction. The lock holding interval is a period of time during which the server holds all the locks for the transaction, preventing reads and / or writes to a portion of the database during that time span (depending on how the locks are held). The added timestamp information may include a maximum read timestamp of the most recently committed read and a maximum write timestamp of the most recently committed write. The writes of the transaction may be delayed based on at least one of the maximum read timestamp or the maximum write timestamp.

[0009] The submitted transaction can be any of various types of transactions, such as read, write, read-modify-write, etc. In some examples, the transaction can be received by the client and temporarily stored in the client's memory.

[0010] If the transaction is a write command buffered in the client's memory, the method may further include ensuring that there are no conflicting reads or writes between the calculation of the provisional timestamp and the time when the transaction completes acquiring its lock. In some instances, the calculated provisional timestamp is used as the timestamp of the transaction.

[0011] Another aspect of this disclosure provides a system including a client device with one or more processors, the processors being programmed to receive a transaction to be submitted to a server with which the client communicates, and to initiate a commit of the transaction before the server acquires a lock. The client device initiates the commit and calculates a provisional timestamp for the transaction, such as the current time plus the value of a variable corresponding to a boundary of clock uncertainty in the distributed system, the clock including at least a client clock at the client and a server clock at the server. This calculation is performed outside the lock holding period. The processors may be further programmed to acquire a lock on the transaction, perform a write operation on the transaction using the calculated provisional timestamp from the client, commit wait, and release the lock. The commit wait may include waiting for a period of time until the calculated provisional timestamp has elapsed. Furthermore, the processors may also be programmed to add timestamp information to the lock, wherein the timestamp information of the lock is used to ensure that the timestamp of the transaction is within the lock holding period of the transaction. The added timestamp information of the lock may include the maximum read timestamp of the most recently committed read and the maximum write timestamp of the most recently committed write. The write operation of the transaction can be delayed based on at least one of the maximum read timestamp or the maximum write timestamp. Wherein, the transaction is a write buffered in the client's memory, and the one or more processors can be further programmed to ensure that there are no conflicting reads or writes between the calculation of the provisional value and the time when the transaction completes acquiring its lock. The server can be programmed to postpone the acquisition of the write lock or the execution of the write until the client calls commit.

[0012] Another aspect of this disclosure provides a non-transitory computer-readable medium storing instructions executable by one or more processors for performing a method. Such a method includes receiving a transaction to be committed to the distributed system at a client in a distributed system, and calculating a provisional timestamp of the transaction in a client-side library at the client. The calculated timestamp may be calculated as the current time plus a variable corresponding to a boundary of uncertainty of clocks in the distributed system, including at least a client clock at the client and a server clock at the server, and initiating the commit of the transaction outside a lockout period.

[0013] On the other hand, means are provided for receiving a transaction to be committed to a distributed database, and means are provided for calculating a provisional timestamp of the transaction in a client-side database on the client. The provisional timestamp may correspond to the current time plus the value of a variable corresponding to a boundary of clock uncertainty in the distributed system, the clock including at least the client-side clock on the client and the server-side clock on the server. Further, means are provided for initiating the commit of the transaction outside the lock-holding interval of the transaction, wherein the calculated provisional timestamp value can be used to commit the transaction. Attached Figure Description

[0014] Figure 1 This is a block diagram illustrating an example system according to aspects of this disclosure.

[0015] Figure 2 This is a schematic diagram illustrating the distribution of the database according to aspects of this disclosure.

[0016] Figure 3 This is a block diagram illustrating the hierarchical relationships between servers in a distributed database, according to aspects of this disclosure.

[0017] Figure 4 This is a block diagram illustrating an example of moving the calculation of timestamp information to a client in the system, according to aspects of this disclosure.

[0018] Figure 5 It is a diagram illustrating various operations of a transaction at different stages and states, based on aspects of this disclosure.

[0019] Figure 6 This is a diagram illustrating an example of the relative timing of events between devices in a distributed database, according to aspects of this disclosure.

[0020] Figure 7 This is another block diagram illustrating an example system according to aspects of this disclosure.

[0021] Figure 8 This is a flowchart illustrating an example method according to aspects of this disclosure. Detailed Implementation

[0022] Overview

[0023] The technique generally involves optimizing transaction commits in a distributed system. Before the server acquires the lock, the client library determines a tentative timestamp for the transaction. The calculated tentative timestamp can be equal to (current time + ε), where ε corresponds to a bound on the uncertainty of the local clock in the distributed system. As another example, the tentative timestamp can be calculated by selecting the latest timestamp from the bound on the uncertainty of global time, such that the selected timestamp will be later than any concept of global time. As yet another example, the calculated tentative timestamp can be the latest timestamp plus a variable, such as 1µs, 1ms, or any other value that the system guarantees all clocks are within that range.

[0024] Depending on the protocol used in the distributed database, the value of (current time + ε) can be determined in different ways. For example, the first protocol implements strict two-phase locking for transactions, where locks are acquired in the first phase ("lock acquisition") and held in the second phase ("lock holding interval"), which continues until the end of the transaction. If a transaction acquires all its locks before initiating a commit, evaluating (current time + ε) in the client database immediately after requesting a commit will produce the same result as evaluating it on the server.

[0025] This disclosure focuses more on scenarios involving a second protocol, where not all transaction locks have been acquired before the client's commit call, meaning the commit is not within the transaction's lock holding period. Therefore, evaluating (current time + ε) in the client library requires additional operations. Specifically, timestamp information is added to the locks to ensure that timestamps are generated during the lock holding period. For example, the timestamp of the latest commit read of variable v (max_read) is associated with the variable, and the timestamp of the latest commit write of variable v (max_write) is associated with the variable. The max_read and max_write timestamps are used in the commit protocol to ensure consistency, and the commit protocol includes a series of functions related to the commit. Such functions include, in turn, DoWrites, PrepareLocks, and DoCommit_Delay, which will be further described below.

[0026] Using the added timestamp information and functions, transaction T is committed in the following phases: start, write, and commit (or abort). Each phase is initiated by the client library. During the start phase, the client library assigns a unique identifier to transaction T. During the write phase, the client library sends one or more read or write commands, including the unique identifier, to the server. The AcquireWriteWW_Delay function is executed by the client library, which calculates max_read for all values ​​of v, but postpones acquiring the write lock and performing the write until the client calls commit. For example, for a read in a read-write transaction, the client library sends a read to the server, where the read acquires the lock. The server performs the read and returns the result to the client via the client library. For a write in a read-write transaction, the write is buffered on the client and sent to the server later. The next function in the write phase, AcquireReadWW, also acquires the read lock while attempting to avoid deadlock. For example, if a higher-priority transaction needs a lock held by another lower-priority transaction, the lower-priority transaction releases its lock and tries again. Furthermore, the Assign function is performed in the write phase, which assigns the value of expression E to variable v. The Assign function can also keep track of all variables and the tables of assigned values.

[0027] Then, the client library initiates the commit phase, and the server commits the transaction. During this commit phase, the client library executes the DoWrites, PrepareLocks, DoCommit_Delay, WaitUntil(ts), and ReleaseLocks functions. DoWrites acquires all locks and performs the write, which is saved as the result of the call to AcquireWriteWW.

[0028] PrepareLocks records all locks, ensuring they are held persistently. DoCommit_Delay sends a message to the server to commit the lock and also updates max_read and max_write for all values ​​of variable v. WaitUntil causes the client to wait for a specified period before releasing the lock via the ReleaseLocks function. For example, during this waiting period, the client can periodically call an updated current time, which, in response, returns a value from the global clock. The waiting period ends and the lock is released when the returned updated current time is greater than the previously calculated (current time + ε) value.

[0029] In this regard, the (current time + ε) of a transaction can be evaluated in the client library, while ensuring that no conflicting reads or writes occur between the evaluation of (current time + ε) and the acquisition of all locks for the transaction. By choosing a provisional timestamp in the client library, the final transaction timestamp can be earlier (approximately the amount of time it took to send the commit message to the server). Therefore, transactions can spend less time waiting for commit and can be committed earlier, resulting in increased efficiency and speed in the computing environment while maintaining consistency across several computing devices distributed across various geographical locations.

[0030] Example System

[0031] Figure 1 An example system including a distributed database is illustrated. Multiple servers 160, 170, and 180 can communicate with each other, for example, via network 150. Servers 160, 170, and 180 can further communicate with multiple client devices, such as clients 110 and 120. Servers 160-180 can control the storage of data in one or more databases. For example, as shown, each server 160-180 is associated with data centers 162, 172, and 182. Each data center 162, 172, and 182 can include multiple computing devices for storing data. The distributed database can implement protocols such as Paxos to provide consistency between systems. In some current systems, consistency between data centers 162, 172, and 182 is maintained by servers 160, 170, and 180, which wait for a period of time (e.g., commit wait) before publishing a write transaction. To mitigate the negative impact of this wait time, such as latency and reduced throughput, some network messages between clients and servers can overlap with the commit wait period. For example, by determining the timestamp of a transaction on the client device, messages can be sent from the client to the server during the commit wait period to influence write and commit messages.

[0032] Although only a few servers are shown, it should be understood that a distributed database can include any number of servers. Similarly, while each server 160, 170, 180 is shown as being associated with its own data center, it should be understood that in other examples, a server may be associated with one or more smaller databases. For example, a database may include multiple servers. Examples of distributed systems are further described in U.S. Patent Application No. 13 / 905,637, which is incorporated herein by reference in its entirety.

[0033] Each of clients 110 and 120 is shown to have applications 112 and 122 and client libraries 114 and 124, but it should be understood that additional features of the client devices may also exist. Any of clients 110 and 120 can write data to the distributed database by sending data to one of servers 160, 170, and 180 via network 150. Although only a few clients are shown, it should be understood that a large number of client devices can communicate with the distributed database via network 150.

[0034] Data centers 162, 172, and 182 can be positioned at considerable distances from each other. For example, as combined... Figure 2 Further described, data centers can be located in various countries around the world. Each data center 162, 172, 182 may include several storage devices, such as hard disk drives, random access memory, disks, disk arrays, tape drives, or any other type of storage device. Data centers 162, 172, 182 can implement any of several architectures and technologies, including, but not limited to, direct-attached storage (DAS), network-attached storage (NAS), storage area network (SAN), Fibre Channel (FC), Fibre Channel over Ethernet (FCoE), hybrid architecture networks, etc. In addition to storage devices, data centers may also include several other devices, such as cabling, routers, etc. Furthermore, in some examples, data centers 162, 172, 182 may be virtualized environments.

[0035] Each server has a local clock 164, 174, and 184. Each local clock 164, 174, and 184 can derive its time from the atomic time master 190. The atomic time master 190 can be a reference clock, for example, used for communication with one or more servers in a distributed database. (See the following for more details.) Figure 3 As further described, the atomic time master device 190 can derive its time from another source such as GPS.

[0036] Each of servers 160, 170, and 180 can have a different value for the current time. For example, clock 164 can differ from clock 174, and clock 174 can differ from clock 184. These differences can be based on factors such as the parent server and factors like drift, oscillator frequency uncertainty, and calibration intervals. The range of this uncertainty among devices in the distributed database is represented by the value ε. Each clock in the database can maintain a time interval ε, which represents the uncertainty boundary. For example, in the following combination... Figure 3During the described calibration period, a time interval ε is determined based on oscillator frequency uncertainty, calibration phase uncertainty, etc. To maintain external consistency between devices, client 110 and / or server 160 can enforce a commit wait time based on ε before accessing objects recently written to the distributed database. This time period corresponds to the uncertainty limit of the local clock.

[0037] Figure 2 This is a geographic map of data centers 210, 220, 230, 240, 250, 260, and 270 located at various locations on Earth. According to some examples, each data center may include an atomic time master device. Each atomic time master device can be connected to a receiver, such as a GPS receiver, for receiving time signals. The GPS receiver may include, for example, roof-mounted antennas 215, 225, 235, 245, 255, 265, and 275, which may be located on the roof above data centers 210, 220, 230, 240, 250, 260, and 270. Host servers can be housed in server racks located within data centers 210, 220, 230, 240, 250, 260, and 270. Conduits can then be installed to route antenna cables from the host servers to the rooftop antennas. Sharing a single antenna among multiple receivers is possible. For example, this can be achieved using an antenna splitter.

[0038] Figure 3 An example of a time platform 300 providing tightly synchronized global clocks across data centers is shown. In this example, platform 300 is constructed as a three-tiered hierarchy of servers, each including its own clock, where child servers calibrate their clocks based on the clocks of their parents. Applications run on hosts 360, 370, and 380.

[0039] The arrows point from the server calibrating their clocks to a known server with a better clock for calibration. For example, as shown, host 360 calibrates its clock based on atomic master device 392. Atomic master device 392 calibrates its clock based on GPS time masters 302 and 304. Host 370 calibrates its clock based on atomic master devices 394 and 396. Atomic master device 394 calibrates its clock based on GPS master device 304. Host 380 calibrates its clock based on atomic master device 396, and atomic master device 396 calibrates its clock based on GPS master device 306. In some examples, child servers may determine which parent servers to use for calibration based on, for example, geographic location, signal strength, or any other marker. In other examples, child / parent pairings may be predetermined. Although Figure 3Hosts 360, 370, and 380 are shown calibrated to atomic masters 392, 394, and 396, but it should be understood that in other examples, hosts 360, 370, and 380 may additionally or alternatively be directly calibrated to GPS time masters 302, 304, and 306.

[0040] At each level of the hierarchy, calibration consists of polling the parent server and intersecting with one or more time intervals received from the parent and extended with the network latency of the host involved in the calibration. Each server can have an associated value (ε) representing the maximum time difference between the time reflected on the server's local clock and the time reflected by the clocks of other servers in the database. The value of ε for each server is derived from the ε of its parent and adjusted for uncertainty based on the product of oscillator frequency uncertainty, effective calibration interval, and the network round-trip time (RTT) from the server to the parent. Thus, in some examples, different values ​​of ε can be maintained on the local clock of each server. In other examples, ε can be globally consistent across devices in the system. Furthermore, in some examples, ε can vary over time because parameters such as oscillator frequency uncertainty, effective calibration interval, and RTT change over time.

[0041] Oscillator frequency uncertainty can be modeled as consisting of frequency instability and oscillator aging. Frequency instability refers to how much the oscillator drifts over a short time period, while oscillator aging refers to how much the oscillator's drift changes over a long time period. The effective calibration interval can be determined by the larger of two values: the calibration interval itself, such as the time between server calibrations, and the time the server must disconnect from its parent.

[0042] Regarding server-to-parent network RTT, the farther the host is from its parent, the greater the phase uncertainty introduced. This uncertainty can also be modeled as two components: calibration phase uncertainty and calibration frequency uncertainty. Calibration phase uncertainty corresponds to the level of uncertainty in calculating the oscillator's phase alignment. Calibration frequency uncertainty corresponds to the level of frequency uncertainty due to the uncertainty in the duration of the calibration period.

[0043] Figure 4An example is shown where a client device determines a time interval from which a transaction can be committed and provides this information to the server, enabling the exchange of additional network messages between the client and server during the commit waiting period. In this example, client 110 receives the transaction, for example, through a user interface or from another client device. As an example, the transaction could be writing an object to a distributed database. Client 110 determines a time interval corresponding to the current time plus ε. For example, as combined with the above... Figure 3 The current time discussed here can be the time reflected by the client's local clock during the calculation, and ε represents the bound of the clock's uncertainty.

[0044] When client 110 sends a write request to server 160 via network 150, server 160 commits the object to the database, and client 110 includes information related to the calculated time interval. In some examples, the information includes a timestamp of a lock-holding interval during which writing or reading is not permitted. The lock-holding interval is specific to each transaction, begins by acquiring locks for all transactions, and ends by releasing locks for the transaction. The timestamp information provided by client 110 can indicate when locks are acquired and when they are released. In other examples, the timestamp information is used to force server 160 to commit the transaction at a specific time, such as a timestamp within an interval corresponding to the current time plus ε.

[0045] Figure 5 An example protocol for moving the evaluation of (now + ε) to the client library is shown, which is the opposite of performing such a calculation on the server after the call is committed. This optimization can result in an earlier timestamp, thus reducing the commit wait time.

[0046] Because the client's call commit is not within the lock holding interval, there is a challenge in the client library regarding the timestamp of the transaction to guarantee that it will fall within the lock holding interval when evaluating (now + ε). Therefore, timestamp information is added to the lock to preserve the invariance that the transaction's timestamp falls within its lock holding interval. Additionally, for buffered writes, conflicting reads or writes are not allowed between the time the client evaluates (now + ε) and the time the transaction completes acquiring its lock. To ensure that the timestamp falls within the lock holding interval and to ensure that there are no conflicting reads or writes between evaluating (now + ε) and acquiring the lock, metadata is associated with each variable v, representing the target of the transaction. Such metadata includes max_readts(v) and max_writets(v). max_readts(v) is the timestamp of the most recent committed read of variable v, and max_writets(v) is the timestamp of the most recent committed write of variable v. For example, if the target is a field in the database to be changed, such as the total in a bank account, the timestamps of the most recent read and write of that field are used as metadata for future transactions. In many instances, such times will be recorded.

[0047] like Figure 5 As shown, the protocol includes several functions. Some of these functions can be computed based on metadata such as the last committed read timestamp and the last committed write timestamp. As an example, NewTS_Delay(T) increments the metadata's max_read and max_write timestamps. This call evaluates the result for a read transaction as result = max(result, ++max_writets(v)) and the result for a write transaction as result = max(result, ++max_readts(v), ++max_writets(v)).

[0048] Figure 5 Transaction-based phases categorize the functions performed during transaction execution into different columns. For example, a transaction can be divided into a start phase, a commit phase, abort phase, and write phase. The abort phase is an alternative to the commit phase. While some operations can be performed by the server, each of these phases can be initiated by the client device. The protocol can be further divided into different rows based on different computation states.

[0049] The database state is the computational state at the "current" time, which logically is the point in time immediately following the last committed transaction in the history. For each ongoing transaction T, the database maintains a read set and a write set. The read set is a list of database variables read by the transaction so far, along with their values. The write set is a list of database variables written by the transaction so far, along with the values ​​to be installed for those variables if T commits.

[0050] Proof states can be established to demonstrate that external consistency is caused by the execution of algorithms within the database state. For example, a proof state could include several annotations that are not maintained by the actual database. Such annotations could include an event timeline and an event dependency graph of all committed transactions, as well as the state required to extend those data structures when any ongoing transaction commits (or aborts). The event dependency graph shows the order in which transactions are committed and indicates which transactions depend on others. As a simplified example, if the first transaction consists of adding 5 to a number in the database, and the second transaction consists of doubling a number in the database, switching the order of the committed transactions will produce different results. Therefore, the second transaction can be considered dependent on the first transaction. To update the event dependency graph, a set of potential event-dependent edges are added as transactions are executed, if it is possible to generate a transaction commit (or abort). For example, the scheduler could maintain a global state for each variable in the database regarding READ and WRITE events, and a transaction-specific state for each transaction regarding READ and WRITE events. The global state for each variable can include the last committed WRITE event, for which any subsequent READ event receives a write-read dependency, and any subsequent write event receives a write-write dependency. The global state can also include a set of all committed READ events that read the value of the last committed write. Any subsequent WRITE event receives a read-write dependency from each event in the set. For each ongoing transaction, potential changes to the last committed read and write events are tracked as "pending".

[0051] In the historical and database state, for a new transaction T that may have already been received by the client, the transaction begins in the start phase. Next, in the write phase, `AcquireWriteWW_Delay(v,T)` acquires write locks in a way that avoids deadlocks. For example, if a higher-priority transaction needs a lock held by another transaction, the higher-priority transaction forces the other transaction to release the lock and try again. Other techniques for avoiding deadlocks can be used alternatively or separately. However, for `AcquireWriteWW_Delay(v,T)`, write locks are not acquired immediately. Instead, the function evaluates the result, but postpones acquiring write locks and performing the write until the client calls commit. This also reorders the created dependencies. `AcquireRead(E,T)` acquires all locks required by the transaction to evaluate expression E. Some locks will not be needed because they are already held by the transaction. Furthermore, `Assign(v,E,T)` adds the result of the expression evaluation to the transaction's write set. For each variable v in expression E that is not in the union of the read and write sets of T... i A value is assigned and that value is stored in the table in association with the variable.

[0052] During the commit phase, the commit is initiated by the client outside the lock-holding interval. For example, a timestamp for the transaction is allocated based on the function `NewTS_Delay()`. Furthermore, transactions are committed during the lock-holding interval using the functions `DoWrites(T)`, `PrepareLocks(T)`, `DoCommit_Delay(T)`, `WaitUntil(ts(T))`, and `ReleaseLocks(T)`. As mentioned above, `NewTS_Delay(T)` increments the `max_read` and `max_write` timestamps of the metadata. This call evaluates the result for a read transaction: `result = max(result, ++max_writets(v))`, and evaluates the result for a write transaction: `result = max(result, ++max_readts(v), ++max_writets(v))`. `DoWrites(T)` acquires all locks and then performs all writes, which are saved as the result of the call to `AcquireWriteWW_Delay(T)`. `PrepareLocks(T)` records the locks held for the transaction. This helps ensure consistency by providing other transactions with information about the locks being held, thus preventing deadlocks as discussed above in conjunction with the function AcquireWriteWW_Delay(v,T). Furthermore, DoCommit_Delay(T) adds the data to be written in the transaction to the database, and appropriately updates max_readts(v) and max_writets(v) for all values ​​of v in the union of the read and write sets of T. ReleaseLocks(T) releases the locks on each variable in the transaction.

[0053] In some instances, a transaction may be aborted. For example, if a higher-priority transaction requires a lock held by transaction T, transaction T may need to be aborted and retried. During the abort phase, the result of NewTS_Delay(T) can be logged and the locks released. Afterward, the transaction can return to the start phase to try again.

[0054] To maintain the proof state, potential additions to the event dependency graph are prepared during the write phase. `AddNewEdges(v,E,T)` updates the event dependency graph based on new READ and WRITE events added during the processing of `Assign(v,E,T)`. For example, e w You can specify the WRITE event added by Assign(v,E,T), and for each READ event to be processed... r Add edge (e) r ,e wSince Assign(v,E,T) is executed, UpdatePending(v,E,T) updates the proof state. If the transaction is aborted, the prepared potential edges are removed. However, if an update is committed, UpdateCommitted(T) updates the proof state if transaction T commits during the commit phase.

[0055] Figure 6 An example time graph is shown for executing a function that involves committing read-write transactions, such as the one described above. Figure 5 The function described is shown in the graph, which is divided into client-side and server-side operations. The X-axis represents time, with the lock-hold interval 690 extending into segments of that time. Although the operations are shown in a specific order, it should be understood that the order of operations can be modified, and some operations can be performed concurrently with other operations. Furthermore, operations can be added or omitted. For example, an operation illustrated as being performed by the server can, in some instances, be performed by the client.

[0056] The client receives transaction 602, for example, via user input or from another device, and assigns a unique identifier to the transaction at 604. At 606, the client initiates a write operation but postpones the write and acquisition of the lock (608) until a commit is invoked. At 612, the client initiates a commit, at which point the server can begin a commit wait period (620). At 614, the client calculates a provisional timestamp for the transaction. For example, the client determines a provisional value corresponding to the interval between the current time reflected by the client's local clock and an uncertainty ε. At 616, the client sends a message to the server to commit the write, providing the server with the provisional timestamp value. At 622, the server acquires the lock, and, provided there is no conflict, at 624, the server begins executing the write using the provisional timestamp provided by the client. For example, the server can check whether any other transaction holds a lock during the period including the provisional timestamp. If another transaction holds such a lock but has a lower priority than the transaction received at 602, the other transaction may be forced to relinquish its lock and restart. If the conflicting transaction has a higher priority, the calculated provisional timestamp can be updated, for example, based on the maximum read timestamp and the maximum write timestamp. For instance, the calculated provisional timestamp can be moved to a later time to avoid a conflict. At 626, the transaction is committed. Meanwhile, at 618, the client waits until the calculated provisional timestamp has passed. Once the transaction is committed, at 627, the server sends a notification to the client indicating that the transaction has been committed. At 628, the lock is released.

[0057] Figure 7This is a diagram of system 700 used to reduce commit wait times in distributed databases. As shown, an example of system 700 may include several servers 710 and 770 coupled to network 750. The system may also include a client 760 capable of communicating with servers 710 and 770 via network 750.

[0058] Server 710 may include processor 720, memory 730, and other components typically found in general-purpose computers. Memory 730 may store information accessible to processor 720, including instructions 732 executable by processor 720. Memory may also include data 734 that can be retrieved, manipulated, or stored by processor 720. Memory 730 may be a non-transitory computer-readable medium capable of storing information accessible to processor 720, such as a hard disk drive, solid-state drive, tape drive, optical storage, memory card, ROM, RAM, DVD, CD-ROM, or a write-only memory. Processor 720 may be a well-known processor or other lesser-known types of processor. Alternatively, processor 720 may be a dedicated controller such as an ASIC.

[0059] Instruction 732 can be a set of instructions, such as machine code, that are directly executed by processor 720, or a set of instructions, such as a script, that are indirectly executed by processor 720. For this purpose, the terms "instruction," "step," and "program" can be used interchangeably herein. Instruction 732 can be stored in object code format for direct processing by processor 720, or stored in other types of computer languages ​​including scripts or sets of independent source code modules that are interpreted on demand or pre-compiled. The functionality, methods, and routines of the instructions are explained in more detail in the examples above and the example methods below.

[0060] Data 734 can be retrieved, stored, or modified by processor 720 according to instruction 732. For example, although the system and method are not limited to a specific data structure, data 734 can be stored in a computer register or a relational database as a table or XML document with multiple different fields and records. Data 734 can also be formatted in a computer-readable format, such as, but not limited to, binary values, ASCII, or Unicode. Furthermore, data 734 can include information sufficient to identify relevant information, such as numbers, descriptive text, proprietary code, pointers, references to data stored in other memory, including other network locations, or information used by functions to compute relevant data. For example, data 734 can include time data, which can be encoded based on instruction 732 in a time format used to describe a moment in time, such as Coordinated Universal Time, Unix epoch, and unambiguous International Atomic Timeepoch.

[0061] although Figure 7 While processor 720 and memory 730 are functionally shown as being within the same block, processor 720 and memory 730 may actually include multiple processors and memories, which may or may not be stored in the same physical housing. For example, some of instructions 732 and data 734 may be stored on a removable CD-ROM, and others may be stored within a read-only computer chip. Some or all of the instructions and data may be stored in a location physically distant from processor 720 but still accessible to processor 720. Similarly, processor 720 may actually include a collection of processors that may or may not operate in parallel.

[0062] Servers 710 and 770 can be nodes in network 750 and can communicate directly and indirectly with other nodes in network 750. For example, servers 710 and 770 may include web servers that can communicate with client device 760 via network 750, allowing the web servers to use network 750 to send information to client applications. Servers 710 and 770 may also include several computers, such as a load-balanced server farm, that exchange information with different nodes in network 750 for receiving, processing, and sending data to client devices. In this example, client computers will typically still be located at different nodes in network 750 compared to the computers that make up servers 710 and 770. Figure 7Only a few servers 710 and 770 are shown, but it should be understood that a typical system can include a large number of connected servers, each on a different node in the network 750.

[0063] Similar to servers 710 and 770, each client 760 may be configured with a processor 762, memory 763, instructions 764, and data 767. Each client 760 may be a personal computer intended for personal use, having all the internal components typically found in a personal computer, such as a central processing unit (CPU), CD-ROM, hard disk drive, and display device 765, speakers, modem and / or network interface devices, user input 766, and all components for connecting these elements to each other. The display device 765 may be, for example, a monitor with a screen, projector, touchscreen, small LCD screen, television, or another device such as an electronic device that can be operated to display information processed by the processor 762. The user input 766 may be, for example, a mouse, keyboard, touchscreen, or microphone. Furthermore, the computer according to the systems and methods described herein may include devices capable of processing instructions and transmitting and receiving data to and from people and other computers, including general-purpose computers, PDAs, tablets, mobile phones, smartwatches, network computers lacking local storage, television set-top boxes, and other networked devices.

[0064] Client 760 may include application interface module 769. The application interface module can be used to access services provided by servers such as servers 710 and 770. For example, the application interface module may include subroutines, data structures, object classes, and other types of software components that allow the server and client to communicate with each other. In one aspect, application interface module 769 may be a software module that can operate in conjunction with several types of operating systems known in the art. For example, client 760 may be connected to a Structured Query Language (SQL) database server, which can operate in conjunction with application interface module 769 for storing and retrieving information data. Memory 763 coupled to client 760 may store data 767 accessed by application module 769. Data 767 may also be stored on removable media such as disks, tapes, SD cards, or CD-ROMs, which may be connected to client 760.

[0065] Servers 710 and 770, and client 760, can communicate directly and indirectly, such as through network 750. For example, using Internet sockets, client 760 can connect to services operating on remote servers 710 and 770 via Internet Protocol (IP) protocols. Servers 710 and 770 can establish listening sockets that can accept initiating connections for sending and receiving information. Network 750 and intermediate nodes can include various configurations and protocols, including the Internet, World Wide Web, Intranet, Virtual Private Network, Wide Area Network, Local Area Network, Private Network using proprietary communication protocols of one or more companies, Ethernet, WiFi (e.g., 702.71, 702.71b, g, n, or other such standards), and HTTP, as well as various combinations thereof. Devices capable of transmitting data to and from other computers can facilitate such communication; these devices include modems (e.g., dial-up, cable, or fiber optic) and wireless interfaces.

[0066] although Figure 7 Computing devices 710 and 760 are shown as separate blocks, each containing its own processor and memory; however, the operations described herein can involve a single computing device or many computing devices, for example, in the “cloud.” For instance, the various operations described herein that involve a single computing device (e.g., a single central processing unit (CPU) in a single server) can involve multiple computing devices (e.g., multiple processors in a load-balanced server farm). Similarly, memory components in different locations can store different portions of instruction 732 and together form a medium for storing instructions. In some examples, device 760 can be used as a thin client, where device 710 performs all or almost all operations that are not directly related to receiving and providing information to the user via user input component 766 and display 765. The various operations described herein that are performed by computing devices can be performed by virtual machines. For example, instruction 732 can be specific to a first type of server, but the associated operations can be performed by a second type of server running a hypervisor that emulates the first type of server. Operations can also be performed by containers, for example, computing environments that are not dependent on an operating system bound to a specific type of hardware.

[0067] Example Method

[0068] Figure 8 An example method for reducing commit latency by reading the clock on the client side earlier is shown. It should be understood that the following operations do not necessarily have to be performed in the exact order described below. Instead, the steps can be processed in different orders or simultaneously. Steps may be added or omitted unless otherwise stated.

[0069] In box 810, the client receives a first transaction, for example, from a user or from another client device. The first transaction may include a request to write data to the distributed database. For example, the first transaction may be a read-modify-write (RMW) command or any other type of command.

[0070] In box 815, the client begins executing a transaction. This may include assigning a unique identifier to the transaction. Such an identifier may be included in the message exchanged between the client and the server to commit the transaction.

[0071] In box 820, the client initiates a transaction commit. The commit is initiated by the client outside the lock holding interval that holds read and / or write locks. Upon initiating a commit, the client may send one or more messages to the server. These messages may include, for example, metadata related to the most recently committed transaction, updates to previously provided information, and other information for allocating timestamps to the lock holding interval.

[0072] In box 830, the client calculates a provisional timestamp for the transaction. This provisional timestamp can be a value corresponding to the current time plus ε, such as the time reflected on the client's local clock when the value is calculated. ε corresponds to a boundary of time uncertainty in the database. For example, ε can be calculated as the difference between the latest time reflected by the clock in the database and an earlier time reflected by the clock in the database. Due to server calibration based on the parent server, oscillator frequency uncertainty of each device, and / or other factors, each device can have its own value of ε. For example, the value of ε for the first server in the database can be different from the value of ε for the second server in the database. In some examples, each device can store information that allows the device to quickly calculate its own value of ε.

[0073] In box 835, the client sends a message to the server to begin committing the transaction using a calculated provisional timestamp value.

[0074] In box 840, the server acquires a lock for the first transaction, such as a write lock. Once the lock is acquired, no further writes to the database are allowed until the lock is released.

[0075] In box 850, the server executes a write transaction, and the server writes the data associated with the transaction to the database. For example, if the transaction is a read-modify-write transaction that updates a field in the database with new data, the server will update the field by inputting the new data.

[0076] In box 860, the server waits for the time corresponding to the calculated value to elapse. In box 870, the server releases the lock on the transaction. Therefore, regardless of any uncertainty in the database regarding the current time and when the transaction will be committed, other devices in the database will consistently read the database.

[0077] Although the various operations in the examples above have been described in relation to the server or client, it should be understood that such operations may be performed alternatively or additionally by other devices.

[0078] As described in the example above, moving the evaluation of the current time plus ε to the client library effectively reduces commit wait time, thereby lowering latency and consequently increasing throughput. For example, by assigning timestamps to lock-hold intervals, the length of lock-hold intervals can be reduced while ensuring transactions are committed within the interval. Furthermore, such methods and systems can be implemented cost-effectively without requiring the replacement of all hardware in an existing system.

[0079] Unless otherwise stated, the above alternative examples are not mutually exclusive, but can be implemented in various combinations to achieve unique advantages. Because these and other variations and combinations of the features discussed above can be utilized without departing from the subject matter defined by the claims, the above description of the embodiments should be interpreted by way of illustrating the subject matter defined by the claims rather than by way of limiting it. Furthermore, the provision of examples described herein and clauses expressed as "such as," "comprising," etc., should not be construed as limiting the subject matter of the claims to the specific examples; rather, the examples are intended only to illustrate one embodiment among many possible embodiments. Moreover, the same reference numerals in different figures can identify the same or similar elements.

Claims

1. A method, characterized in that, include: In a distributed system, the server receives transactions from the client that are to be submitted to the distributed system. The server identifies a provisional timestamp for the transaction, wherein the provisional timestamp is calculated by adding the value of a variable corresponding to a limit of the uncertainty of the clock in the distributed system to the current time, the clock including at least the client clock on the client and the server clock on the server. The transaction is initiated by the server receiving a commit from the client outside the lock-holding interval of the transaction; The transaction is executed based on the provisional timestamp, and the lock holding interval begins when the lock for the transaction has been acquired and ends when the lock for the transaction is released.

2. The method according to claim 1, characterized in that, The transaction includes a write, and further includes delaying the write until the client initiates the commit.

3. The method according to claim 1, characterized in that, This further includes delaying the acquisition of the lock until the client initiates the submission.

4. The method according to claim 1, characterized in that, This further includes examining conflict-related matters.

5. The method according to claim 4, characterized in that, Checking for conflicting transactions includes checking whether any other transactions remain locked during the time period including the provisional timestamp.

6. The method according to claim 4, characterized in that, Further includes: Detect conflicting transactions; as well as Determine whether the conflicting transaction has a higher priority.

7. The method according to claim 6, characterized in that, Furthermore, when the conflicting transaction has a lower priority, it is forced to relinquish its lock and restart.

8. The method according to claim 6, characterized in that, Furthermore, when the conflicting transaction has a higher priority, the provisional timestamp is updated.

9. The method according to claim 8, characterized in that, Updating the provisional timestamp includes moving it to a later time to avoid the conflict.

10. The method according to claim 1, characterized in that, Further includes acquiring a lock in response to receiving the submission to initiate the process.

11. A system, characterized in that, include: One or more memory units; A server in a distributed system that communicates with the one or more memories, the server being configured to: Receive the transaction to be submitted to the distributed system from the client; Identify a provisional timestamp for the transaction, wherein the provisional timestamp is calculated using the current time plus the value of a variable corresponding to a limit of the uncertainty of the clock in the distributed system, the clock including at least the client clock on the client and the server clock on the server; Receive the commit initiation of the transaction from the client outside the lock-holding interval of the transaction; and The transaction is executed based on the provisional timestamp.

12. The system according to claim 11, characterized in that, The transaction includes a write, wherein the server is further configured to delay the write until the client initiates the commit.

13. The system according to claim 11, characterized in that, The server is further configured to delay acquiring the lock until the client initiates the submission.

14. The system according to claim 11, characterized in that, The server is further configured to check for conflicting transactions, wherein checking for conflicting transactions includes checking whether any other transactions remain locked during the time period including the provisional timestamp.

15. The system according to claim 14, characterized in that, The server is further configured to: Detecting conflicting transactions; and Determine whether the conflicting transaction has a higher priority.

16. The system according to claim 15, characterized in that, The server is further configured to force the conflicting transaction to relinquish its lock and restart when the conflicting transaction has a lower priority.

17. The system according to claim 15, characterized in that, The server is further configured to update the provisional timestamp when the conflicting transaction has a higher priority.

18. The system according to claim 17, characterized in that, Updating the provisional timestamp includes moving it to a later time to avoid the conflict.

19. The system according to claim 11, characterized in that, The server is further configured to acquire a lock in response to receiving the submission to start.

20. A computer-readable medium, characterized in that, The storage includes instructions executable by one or more processors, the instructions for performing a method comprising: In a distributed system, the server receives transactions from the client that are to be submitted to the distributed system. The server identifies a provisional timestamp for the transaction, wherein the provisional timestamp is calculated by adding the value of a variable corresponding to a limit of the uncertainty of the clock in the distributed system to the current time, the clock including at least the client clock on the client and the server clock on the server. The server receives the transaction commit from the client outside the lock-holding interval of the transaction; and The transaction is executed based on the provisional timestamp.

Images