High throughput algorithm for multi-version concurrency control with global synchronous time

Abstract

This disclosure relates to a high-throughput algorithm for multi-version concurrency control with globally synchronized time. Throughput is preserved in the distributed system while concurrency is maintained by pushing commit wait periods to the client commit path and future readers. Instead of the server performing commit waits, the server assigns timestamps, which are used to ensure that causality is preserved. When the server executes a transaction that writes data to the distributed database, the server acquires a user-level lock and assigns the transaction a timestamp equal to the current time plus an interval corresponding to the range of clock uncertainty in the distributed system. After assigning the timestamp, the server releases the user-level lock. Any client device must wait until the assigned timestamp has elapsed before performing a read of the written data.

Description

[0001] Case Analysis

[0002] This application is a divisional application of Chinese invention patent application 201710884380.3, filed on September 26, 2017. Technical Field

[0003] This application relates to a high-throughput algorithm for multi-version concurrency control with global synchronization time. Background Technology

[0004] In database systems, the term concurrency control refers to the algorithm used to ensure consistent database behavior even in the presence of concurrency. Multi-version concurrency control (MVCC) algorithms store multiple versions of a given data slice (one version for each write) to achieve greater concurrency. Systems that provide a global concept of absolute time can be integrated with MVCC in distributed databases. The resulting distributed database is semantically equivalent to a single-machine database because consistent reads can be performed across the entire database. Consistent reads are ensured using local synchronization between reads and writes at each object in the database, along with commit waits, where writes need to be delayed in a timely manner to ensure appropriate semantics. This local synchronization requires expensive global synchronization within the database.

[0005] The problem with existing solutions is that they perform commit waits while simultaneously holding user-level locks on the server. In other words, while writing objects to the database, the server performs commit waits, maintaining the locks the database uses to protect access to the objects being written. This means the throughput for writing to any object is limited to at most (1 / commit wait time). Summary of the Invention

[0006] One aspect of this disclosure provides a system comprising a server adapted to communicate with other servers and clients in a distributed computing environment. The server includes a processor configured to receive requests to write data, write data to memory in the distributed computing environment, and, while the data is being committed to memory, release locks on the server and impose a commit wait time on at least one of: a client library, another server, or a client. The commit wait time is a mechanism to ensure that a client cannot see the effects of a transaction before its timestamp. For example, a client waits for a time interval (referred to as the commit wait time) to elapse before seeing the effects of the transaction. In this aspect, various clients will obtain consistent reads of the distributed computing environment and be able to make further modifications accordingly.

[0007] In the example where commit wait is imposed on both the client library and other servers, the commit wait is executed by the client library before it notifies the client that a write operation has been committed. The other servers execute the commit wait when a transaction read is performed.

[0008] In other examples where the commit time is imposed only on the client, the processor sends the client a timestamp of when data was written. This timestamp may have been assigned the local time at the server where the data was written, plus a time limit that restricts the difference between all clocks in the system. This time should be in the future, after the time of any write that has been committed. Any client waits until the assigned timestamp has passed before returning the data to the client. For example, the assigned timestamp has passed when the local clock at the client reading the data reflects a current time later than the assigned timestamp. The time limit that restricts the difference between all clocks in the system can be maintained by the local clock at the server as an interval ε. The processor can also be configured to receive a second request to write second data and write the data to memory in the distributed computing environment without waiting for the commit wait time to terminate.

[0009] Another aspect of this disclosure provides a method comprising: receiving a request from one or more clients at a first computing device to write data to a distributed computing environment; acquiring a write lock at the first computing device; writing the data to a memory in the distributed computing environment via the first computing device; and releasing the write lock via the first computing device without waiting for a commit wait time to terminate, such that the commit wait time is imposed on at least one of: a client library, another server, or a client.

[0010] To shift the commit wait to the client, the processor is also configured to assign a timestamp to the data being written, the timestamp being equal to the first time plus the interval, and to release the lock after assigning the timestamp. Before reading the data, any client waits until the assigned timestamp has elapsed. Attached Figure Description

[0011] Figure 1 This is a block diagram illustrating an example system according to various aspects of this disclosure.

[0012] Figure 2 This is a schematic diagram illustrating the distribution of the database according to various aspects of this disclosure.

[0013] Figure 3 It is a block diagram illustrating the hierarchical relationships among servers in a distributed database according to various aspects of this disclosure.

[0014] Figure 4This is a block diagram illustrating an example of shifting the submission wait time to other devices in the system according to various aspects of this disclosure.

[0015] Figure 5 This is a diagram illustrating an example of the relative timing of events in a device within a distributed database according to various aspects of this disclosure.

[0016] Figure 6 This is a block diagram illustrating another example of shifting the submission wait time to other devices in the system according to various aspects of this disclosure.

[0017] Figure 7 This is a diagram illustrating another example of the relative timing of events in a device within a distributed database according to various aspects of this disclosure.

[0018] Figure 8 This is another block diagram illustrating an example system according to various aspects of this disclosure.

[0019] Figure 9 This is a flowchart illustrating example methods according to various aspects of this disclosure. Detailed Implementation

[0020] Overview

[0021] This disclosure relates to a method for reducing the time period during which devices in a distributed system must wait for write transactions to be committed while maintaining consistency throughout the distributed system. This time period (referred to as the commit wait time) is moved outside of a second time period that maintains user-level locking. Server-side code releases the commit wait time, and conversely, the commit wait time is imposed on the client library. For example, a timestamp associated with the write transaction is recorded, and the user-level lock is released, and any device in the distributed database that wants to read the committed write transaction must wait until that timestamp is guaranteed to have passed.

[0022] In the first example, the commit wait does not occur at the device performing the write transaction; instead, it occurs in two places. The commit wait occurs in the client library before the client notifies the client that it has committed the write. The commit wait also occurs at the server because any device attempting to read the written data finds that it must commit at the server. Therefore, the commit wait is moved from the "writer" to the "reader." This does not change the observable commit order of the transaction. Furthermore, it does not change the commit order of data-related transactions.

[0023] In the second example, a commit wait is pushed to the client commit path and future readers. For example, instead of the server executing a commit wait, the server assigns a timestamp, which is used to ensure causality is preserved. For example, when the server executes a transaction that writes data to a distributed database, the server can acquire a user-level lock and assign the transaction a timestamp equal to (current time + ε), where ε will be a measure of the uncertainty of clocks in the distributed system. For example, ε could represent the difference between the latest time reported by all clocks in the system and the earliest time reported by all clocks in the system. In some implementations of a global clock, ε can typically be, for example, less than 3ms. After assigning the timestamp, the server releases the user-level lock. The database must guarantee that no client device has read the data before the time of the assigned timestamp. For example, client library code will enforce this invariant. In other words, the client must wait for a period of time dependent on ε before performing a read of the data written by the server.

[0024] In the second example above, commit waits are completed in parallel across all clients. Because the server must communicate with the clients, overlapping the commit wait time with the communication time between the server and clients reduces the overall wait time. If it takes a long time for a message from the server to reach the client, the client will have to commit wait for a shorter period. Each client can have a different local time. Therefore, clients with a later local time may experience shorter commit wait times because the assigned timestamp will arrive more quickly. Conversely, clients with an earlier local time may experience longer commit wait times. By releasing user-level locks on the server during the commit wait time, throughput at the server is increased. Meanwhile, the concurrency and reads maintained by the distributed database will be accurate and consistent across all clients, reflecting committed data regardless of differences in local clock times.

[0025] Example System

[0026] Figure 1The illustration depicts an example system including a distributed database. Multiple servers 160, 170, and 180 can communicate with each other, for example, via network 150. Servers 160, 170, and 180 can also communicate with multiple client devices, such as clients 110 and 120. Servers 160-180 can control the storage of data in one or more databases. For example, as shown, each server 160-180 is associated with data centers 162, 172, and 182. Each data center 162, 172, and 182 can include multiple computing devices for storing data. While some devices may be replicas of other devices, some devices may be causally dependent on other devices. For example, bits of data written in data center 162 can affect data stored in data center 172. The distributed database can implement protocols such as Paxos to provide consistency across the system. In some current systems, consistency across data centers 162, 172, and 182 is maintained by servers 160, 170, and 180, which wait for a period of time (e.g., commit wait) before publishing a write transaction. However, this wait period imposed by the server during writes reduces throughput. Therefore, this wait period can instead be imposed on one or more other devices, or shifted in time to different actions of the server. For example, the wait period can be shifted to a client library on the client that writes data to the server, and other servers may need to perform a commit wait on that server before executing a read transaction. In other examples, the commit wait is shifted to a client library running on the client device seeking to read the written data.

[0027] Although only a few servers are shown, it should be understood that any number of servers can be included in a distributed database. Similarly, although each server 160, 170, 180 is shown as associated with its own data center, it should be understood that in other examples, servers may be associated with one or more smaller databases. For example, a database may include multiple servers. Examples of distributed systems are also described in U.S. Patent Application No. 13 / 905,637, which is incorporated herein by reference in its entirety.

[0028] Each of clients 110 and 120 is shown as having applications 112 and 122 and client libraries 114 and 124; however, it should be understood that additional features of the client devices may also exist. Any of clients 110 and 120 can write data to the distributed database by sending data to one of servers 160, 170, and 180 via network 150. Although only a few clients are shown, it should be understood that a large number of client devices can communicate with the distributed database via network 150.

[0029] Data centers 162, 172, and 182 can be located at considerable distances from each other. For example, as combined... Figure 2 Further described, data centers can be located in various countries around the world. Each data center 162, 172, 182 can include numerous storage devices, such as hard disk drives, random access memory, disks, disk arrays, tape drives, or any other type of storage device. Data centers 162, 172, 182 can implement any of many architectures and technologies, including but not limited to Direct Attached Storage (DAS), Network Attached Storage (NAS), Storage Area Network (SAN), Fibre Channel (FC), Fibre Channel over Ethernet (FCoE), hybrid architecture networks, etc. In addition to storage devices, data centers can include many other devices, such as cabling, routers, etc. Furthermore, in some examples, data centers 162, 172, 182 can be virtualized environments.

[0030] Each server has a local clock 164, 174, and 184. Each local clock 164, 174, and 184 can derive its time from the atomic time master 190. The atomic time master 190 can be, for example, a reference clock communicating with one or more servers in a distributed database. (As in combination...) Figure 3 As described further below, the atomic time master 190 can derive its time from another source, such as GPS.

[0031] Figure 2 This is a geographic map of data centers 210, 220, 230, 240, 250, 260, and 270 located at various locations on Earth. According to some examples, each data center may include an atomic time master. Each atomic time master can be connected to a receiver (such as a GPS receiver) to receive time signals. The GPS receiver may include, for example, roof-mounted antennas 215, 225, 235, 245, 255, 265, and 275, which can be located on the roof above data centers 210, 220, 230, 240, 250, 260, and 270. Host servers can be housed in server racks located in data centers 210, 220, 230, 240, 250, 260, and 270. Conduits can then be installed to route antenna cables from the host servers to the roof antennas. Sharing a single antenna across several receivers is possible. This can be achieved, for example, using an antenna splitter.

[0032] Figure 3The illustration shows an example of a time platform 300 that provides tightly synchronized global clocks across data centers. In this example, platform 300 is structured as a three-tiered hierarchy of servers, each including its own clock, where child servers calibrate their clocks based on their parent's clock. The application runs on hosts 360, 370, and 380.

[0033] The arrows point from the server calibrating its clock to a well-known server with a better clock from which it calibrates. For example, as shown, host 360 calibrates its clock based on atomic master 392. Atomic master 392 calibrates its clock based on GPS time masters 302 and 304. Host 370 calibrates its clock based on atomic masters 394 and 396. Atomic master 394 calibrates its clock based on GPS master 304. Host 380 calibrates its clock based on atomic master 396, which in turn calibrates its clock based on GPS master 306. In some examples, child servers may determine which parent servers are used for calibration based on, for example, geographic location, signal strength, or any other markers. In other examples, child / parent pairings may be pre-defined. Although Figure 3 The host computers 360, 370, and 380 are shown calibrating the atomic master controllers 392, 394, and 396, but it should be understood that in other examples, the host computers 360, 370, and 380 may additionally or alternatively calibrate the GPS time master controllers 302, 304, and 306 directly.

[0034] At each level of the hierarchy, calibration involves polling the parent server and intersecting with one or more time intervals received from that parent, where these time intervals are extended by the network latency of the calibration from the involved hosts. Each server can have an associated value (ε), representing the maximum time difference between the time reflected on the server's local clock and the time reflected by the clocks of other servers in the database. Each server's ε value is derived from its parent's ε and the server-to-parent network round-trip time (RTT), with the ε for its parent adjusted for the uncertainty arising from the product of oscillator frequency uncertainty and the effective calibration interval. Thus, in some examples, the local clock at each server can maintain a different ε value. In other examples, ε can be globally consistent across devices in the system. Furthermore, in some examples, ε can vary over time because parameters such as oscillator frequency uncertainty, effective calibration interval, and RTT change over time.

[0035] Oscillator frequency uncertainty can be modeled as including frequency instability (such as how much the oscillator drifts during a short time scale) and oscillator aging (such as how much the oscillator drift changes during a long time scale). The effective calibration interval can be determined by the larger of two values: the calibration interval itself (such as the time interval between server calibrations) and how long the server may need to disconnect from its parent.

[0036] Regarding server-to-parent network RTT, the farther the host is from its parent, the greater the phase uncertainty. This uncertainty can also be modeled as two components: calibration phase uncertainty and calibration frequency uncertainty. Calibration phase uncertainty corresponds to the level of uncertainty in the oscillator's calculated phase alignment. Calibration frequency uncertainty corresponds to the level of frequency uncertainty attributable to the duration of the calibration period.

[0037] Figure 4 The illustration shows an example where a commit wait time is pushed to the client database and any server seeking to read the written data. In this example, client 110 performs a transaction (such as writing data to a distributed database). Specifically, client 110 writes object O. The write request is sent from client 110 to server 160 via network 150, which commits object O to the database. However, each of servers 160, 170, and 180 can have a different representation of the current time. For example, clock 164 can be different from clock 174, and clock 174 can be different from clock 184. As discussed above, these differences can be based on, for example, the parent server and factors such as drift, oscillator frequency uncertainty, calibration interval, etc. This range of uncertainty among the devices in the distributed database is represented by the value ε. Each clock in the database can maintain a time interval ε, which represents the range of uncertainty. Based on oscillator frequency uncertainty, calibration phase uncertainty, etc., the time interval ε is, for example, combined with... Figure 3 The calibration period described above is determined. Therefore, to maintain consistency between devices, client library 114 of client 110, which sends a write request, can enforce a wait time based on ε before accessing the write object O. Furthermore, any server attempting to read object O also waits. For example, if server 170 has requested to read data including object O, server 170 will need to perform a commit wait and wait for a period of time to elapse. This period corresponds to the uncertainty range of the local clock.

[0038] Figure 5 This illustration demonstrates how and when actions are performed by specific devices in a distributed database to maintain database state consistency by shifting commit wait times to the client database and the server that wants to read updated data. In this example, a remote single-site read-modify-write (RMW) transaction increments a counter x.

[0039] As shown, client 1 initiates a transaction, and the server acquires a read lock. When the transaction is used to increment the counter x and has just begun, the server reads x = 0. The server reads data x and writes it at time 3312. The read time and any other time can be represented using any of various units (such as seconds of a day, milliseconds, etc.). The server commits and waits until the time reflected by its local clock ("now") minus ε is greater than or equal to 3312. Client 1 increments the counter x and requests that x = 1 be written to the database. The server upgrades to a write lock for the transaction. In other examples, instead of first acquiring a read lock and then upgrading to a write lock, the server can begin by acquiring a write lock. The server assigns a timestamp of the current time reflected by its local clock ("now") plus ε (clock uncertainty). In this example, the resulting timestamp is 4225. The server commits data x = 1, applies x = 1, and releases the lock. Then, client 1's client library enforces the commit wait time. Client 1 will be notified that data x = 1 is committed when the current time ("now") reflected by Client 1's local clock minus the uncertainty (ε) is greater than or equal to the timestamp (4225) assigned by the server.

[0040] Client 2 initiates a subsequent transaction (Transaction 2) to increment the counter to x = 2. The server again acquires a read lock, reads at time 4225 x = 1, and commits, waiting for at least ε time, for example, until the time reflected by the server's local clock ("now") minus ε is greater than or equal to the read time (4225). Client 2 then increments the counter to x = 2. The server upgrades to a write lock and assigns a timestamp to Transaction 2 equal to the current time of its local clock plus ε, which here totals 5132. The server proposes a commit for Transaction 2, applies x = 2 to the database, and releases the write lock. Client 2 executes a commit wait through its client database until its local clock reflects a time greater than or equal to the assigned timestamp of 5132, and thereafter Client 2 is notified that Transaction 2 has committed.

[0041] In the examples above, the latency for each transaction 1 and transaction 2 is 2RTT + max(2ε,RTT). The throughput of executing transactions under high contention will be limited by the write lock holding time, which is the time to record the transaction plus the commit wait = RTT + max(2ε,RTT). The resulting throughput limit will be 1 / (RTT + max(2ε,RTT)). It is known that an RMW transaction running under high contention can acquire write mode locks for appropriate read operations to avoid aborts because multiple readers attempting to escalate their locks deadlock. The resulting throughput will then be approximately 1 / (2RTT + max(2ε,RTT)).

[0042] exist Figure 5 In the example, commit wait occurs in two places: in the client library before it notifies the client that it has committed a write, and on the server when it is performing a transactional read. Instead of having the writer (server) execute a commit wait loop (which would logically be equivalent to executing the following code snippet while(now()+ε<=T){}, where T is the commit time on the transaction), the loop instead executes in two places: in the client library by the writer before any point in the client library that indicates the transaction has been committed to the client; and on the server after it has acquired a lock on the data it is reading. In this respect, commit wait is moved from the writer to the causally related reader. In other examples, a similar result is produced by each previous read with commit wait.

[0043] Moving the commit wait to the leave path at the client side results in an observable commit order for the transaction that is the same as if the commit wait were executed at the server side. Furthermore, moving the commit wait from the writer to the reader side does not change the commit order of data-related transactions. Moving the commit wait across servers is also possible. The commit wait ensures that the timestamp T assigned to the transaction precedes the event E_release in which all locks are released. Therefore, the assigned timestamp T occurs within the interval between when a lock is acquired and when it is released. Whether the commit wait is moved to the leave path at the client side or moved across servers, the fact that ε can now be derived from different oscillators is irrelevant, because timestamp T will have elapsed after the commit wait has been executed. Moving the commit wait to the client side ensures that the reader sees no effect of the transaction only after its timestamp has elapsed.

[0044] Figure 6 The illustration shows an example where a commit wait is moved to the client commit path and a future reader. In this example, when client 110 writes object O to the distributed database, server 160, receiving the write request, assigns a timestamp to the write transaction. The assigned timestamp is equal to the current time reflected by the server's local clock 164 plus ε. After assigning the timestamp, server 160 releases the write lock on the transaction. However, any client device attempting to execute a read transaction on the database must wait for the assigned timestamp to elapse. For example, if client 120 wants to execute a read transaction, client 120 will execute a commit wait. Once the local clock at client 120 reflects a time greater than or equal to the assigned timestamp, the client will be notified of a successful commit.

[0045] Figure 7 The illustration shows an example of how and when actions are performed by a specific device when the submission wait time is moved to the client. Figure 5Compared to the previous example, the server does not perform any commit waits for reads within a transaction. Instead, all commit waits occur at the client level. For example, with Figure 5 Compared to the examples, Figure 7 The example uses the transaction pipeline to push commits, waiting until further along. It is pushed to... Figure 5 Any commits of the write side in a transaction's read are pushed further by the reads at the server, and then again through... Figure 7 The path from the client to the write to the future reader. Pushed from the writer to... Figure 5 The example shows the read's commit wait being contained within the client's own write commit wait of the transaction. Figure 7 In the process, it must be at a timestamp that is later than those timestamps being read.

[0046] As in Figure 7 As shown, client 1 initiates a transaction, which can increment the variable x again, and the server acquires a read lock for the transaction and reads data. When the transaction is used to increment the counter x and has just begun, the server reads x = 0. In this example, the server reads that x is written at time 3312. Client 1 increments the counter x and requests that x = 1 be written to the database. The server upgrades to a write lock for the transaction and assigns a timestamp of the current time ("now") reflected by its local clock plus ε (commit wait time). For example, the timestamp could be equal to max(now + ε, previous_timestamp++). In this example, the resulting timestamp is 4225. The server proposes a commit for data x = 1, applies x = 1, and releases the lock. Then, client 1's client library enforces the commit wait time. Client 1 will be notified that data x = 1 is committed when the current time ("now") reflected by client 1's local clock minus the uncertain time interval (ε) is greater than or equal to the timestamp assigned by the server (4225).

[0047] Client 2 initiates a subsequent transaction (Transaction 2) to increment the counter to x = 2. The server again acquires the read lock, and the read occurs at time 4225, where x = 1. Client 2 then increments the counter to x = 2. The server upgrades to a write lock and assigns a timestamp to Transaction 2 equal to the current time on its local clock plus ε, which here totals 5132. The server proposes a commit for Transaction 2, applies x = 2 to the database, and releases the write lock. Client 2 executes a commit wait through its client database until its local clock reflects a time greater than or equal to the assigned timestamp of 5132, after which Client 2 is notified by Transaction 2.

[0048] In some examples, the server can receive transaction 2 before client 1 is notified that transaction 1 has been committed. The server can then act on transaction 2 regardless of the execution of client 1's commit wait.

[0049] Instead of performing commit waits, the server assigns timestamps and ensures that data dependencies are properly serialized. Because the server does not perform commit waits, its throughput is unaffected by ε. Clients perform commit waits, thus ensuring that causality is preserved. Moreover, because commit waits are completed in parallel across all clients, the throughput of each client is affected only by changes in ε, not by the value of ε.

[0050] Although it was described as having only a few clients and one server Figure 7 This is an example, but it should be understood that the described characteristics can be applied to multi-site transactions. For example, additional servers located in different geographical regions and serving additional clients can similarly assign timestamps and move commit waits to the clients. In such an example, commit waits should not exist, while maintaining user-level locking.

[0051] In some examples, timestamp assignments can also be pushed to the client. This will result in a latency of RTT + max(2ε, 2RTT). If the client acquires a write lock instead of a read lock, such as to avoid aborts under high contention, the throughput becomes approximately 1 / (3RTT), which is the theoretical maximum throughput of the cell without commit wait.

[0052] Figure 8 This is a diagram of a system 800 used to maintain consistency in a distributed database while maintaining high throughput. As shown, an example of system 800 may include a number of servers 810 and 870 coupled to network 850. The system may also include a client 860 capable of communicating with servers 810 and 870 via network 850.

[0053] Server 810 may include processor 820, memory 830, and other components typically found in general-purpose computers. Memory 830 is capable of storing information accessible to processor 820, including instructions 832 executable by processor 820. Memory may also include data 834 retrievable, manipulated, or stored by processor 820. Memory 830 may be a type of non-transitory computer-readable medium capable of storing information accessible to processor 820, such as hard disk drives, solid-state drives, magnetic tape drives, optical storage devices, memory cards, ROM, RAM, DVDs, CD-ROMs, writable memory, and read-only memory. Processor 820 may be a well-known processor or other lesser-known types of processors. Alternatively, processor 820 may be a special-purpose controller (such as an ASIC).

[0054] Instruction 832 can be a set of instructions (such as machine code) executed directly by processor 820 or a set of instructions (such as a script) executed indirectly. In this regard, the terms "instruction," "step," and "program" can be used interchangeably herein. Instruction 832 can be stored in other types of computer languages, either in object code format for direct processing by processor 820 or in scripts or sets of independent source code modules that are interpreted on demand or compiled in advance. The functionality, methods, and routines of the instructions are explained in more detail in the foregoing examples and in the example methods below.

[0055] Data 834 can be retrieved, stored, or modified by processor 820 according to instructions 832. For example, while the system and method are not limited to a specific data structure, data 834 can be stored in computer registers, as a table in a relational database with multiple distinct fields and records, or as an XML document. Data 834 can also be formatted in a computer-readable format, such as, but not limited to, binary values, ASCII, or Unicode. Moreover, database 834 can include information sufficient to identify relevant information, such as numbers, descriptive text, proprietary code, pointers, references to data stored in other memory (including other network locations), or information used to compute relevant data through functions. For example, data 834 can include time data that can be encoded based on instructions 832 in a time format used to describe time (such as Coordinated Universal Time, Unix epochs, and unambiguous International Atomic Time epochs).

[0056] Although Figure 8 Functionally, processor 820 and memory 830 are illustrated as being within the same block; however, processor 820 and memory 830 may actually comprise multiple processors and memories that may or may not be stored in the same physical housing. For example, some of instructions 832 and data 834 may be stored on a removable CD-ROM and others may be stored within a read-only computer chip. Some or all of the instructions and data may be stored in a physically distant location but still accessible to processor 820. Similarly, processor 820 may actually comprise a collection of processors that may or may not operate in parallel.

[0057] Servers 810 and 870 can be located at a node of network 850 and can communicate directly and indirectly with other nodes of network 850. For example, servers 810 and 870 can include a web server, which may be able to communicate with client device 860 via network 850, enabling it to use network 850 to transmit information to client applications. Servers 810 and 870 can also include a number of computers, such as a load-balancing server cluster, which exchange information with different nodes of network 850 for the purpose of receiving, processing, and transmitting data to client devices. In this example, the client computer will typically still be located at a different node of network 850 from the computers that make up servers 810 and 870. Figure 8 Only a few servers 810 and 870 are described, but it should be understood that a typical system can include a large number of connected servers, each located at different nodes on the network 850.

[0058] Each client 860 may be configured similarly to servers 810 and 870, including a processor 862, memory 863, instruction set 864, and data set 867. Each client 860 may be a personal computer intended for human use, having all the internal components typically found in a personal computer, such as a central processing unit (CPU), CD-ROM, hard disk drive, and display device 865 (e.g., a monitor with a screen, projector, touchscreen, small LCD screen, television, or another device (such as an electrical device operable to display information processed by processor 862)), speakers, modem and / or network interface device, user input 866 (such as a mouse, keyboard, touchscreen, or microphone), and all components used to connect these elements to each other. Furthermore, the computer according to the systems and methods described herein may include devices capable of processing instructions and transferring data to and from people and other computers, including general-purpose computers, PDAs, tablets, mobile phones, smartwatches, network computers lacking local storage, television set-top boxes, and other networked devices.

[0059] Client 860 may include application interface module 868. The application interface module can be used to access services available by servers (such as servers 810 and 870). For example, the application interface module may include subroutines, data structures, object classes, and other types of software components used to allow the server and client to communicate with each other. In one aspect, application interface module 868 may be a software module operable in conjunction with several types of operating systems known in the art. For example, client 860 may be connected to a Structured Query Language (SQL) database server, which may operate in conjunction with application interface module 868 for storing and retrieving informational data. Memory 863 coupled to client 860 may store data 867 accessed by application module 868. Data 867 may also be stored on removable media (such as disks, tapes, SD cards, or CD-ROMs) that can be connected to client 860.

[0060] Servers 810 and 870, and client 860, can communicate directly and indirectly (e.g., via network 850). For example, using an Internet port, client 860 can connect to services operating on remote servers 810 and 870 via Internet Protocol (IP) protocols. Servers 810 and 870 can establish listening sockets that can accept initiating connections for sending and receiving information. Network 850 and intermediate nodes can include various configurations and protocols, including the Internet, World Wide Web, intranet, virtual private network, wide area network, local area network, private network using communication protocols proprietary to one or more companies, Ethernet, WiFi (e.g., 802.81, 802.81b, g, n, or other such standards), and HTTP, as well as various combinations thereof. Such communication can be facilitated by devices capable of transmitting data to and from other computers (such as modems (e.g., dial-up, cable, or fiber optic) and wireless interfaces).

[0061] Although Figure 8Server 810 and client 860 are shown as separate blocks, each containing its own processor and memory; however, the operations described herein can involve a single server or client, or many servers or clients (e.g., in the “cloud”). For example, various operations described herein as involving a single server (e.g., a single central processing unit (CPU) in a single server) can involve multiple servers (e.g., multiple processors in a load-balancing server cluster). Similarly, memory components in different locations can store different portions of instruction 832 and collectively form a medium for storing instructions. In some examples, client 860 can be used as a thin client, where server 810 performs all or almost all operations that do not directly involve receiving and providing information to a user via user input component 866 and display 865. The various operations described herein, as performed by the server or client, can be performed by a virtual machine. By way of example, instruction 832 can be specific to a first type of server, but related operations can be performed by a second type of server running a hypervisor simulating a first type of server. Operations can also be performed by containers (e.g., computing environments that do not depend on an operating system bound to a specific type of hardware).

[0062] Example Method

[0063] Figure 9 The illustration shows an example method for moving a submission wait from the server to another device so that it is not held up for further processing by the server. It should be understood that the following operations do not necessarily need to be performed in the exact order described below. Instead, the steps can be processed in a different order or simultaneously. Unless otherwise stated, steps may also be added or omitted.

[0064] In block 910, the server receives the first transaction from the client. The first transaction includes a request to write data to the distributed database. For example, the first transaction could be an RMW command, or any other type of command.

[0065] In block 920, the server acquires a lock for the first transaction. This lock can be a write lock. However, in other examples, such as where the first transaction is an RMW command, the server could first acquire a read lock and then escalate it to a write lock.

[0066] In block 930, the server assigns a timestamp T to the first transaction. The timestamp T corresponds to the current time (such as the time reflected on the server's local clock when the timestamp is assigned) plus ε. ε corresponds to the range of time uncertainty in the database. For example, ε can be calculated as half the difference between the latest time reflected by the clock in the database and the earliest time reflected by the clock in the database. Due to server calibration based on the parent server, oscillator frequency uncertainty for each device, and / or other factors, each device can have its own ε value. For example, the ε value at the first server in the database can differ from the ε value at the second server in the database. In some examples, each device can store information that allows the device to quickly calculate its own ε value.

[0067] In block 940, the server submits a commit for the data to be written. For example, the server writes data to a distributed database.

[0068] In block 950, the server releases the lock for the first transaction. The server does not execute a commit wait. Instead, in block 960, the commit wait is pushed to the reader or client.

[0069] The aforementioned example methods and systems are advantageous because they provide increased server throughput while maintaining consistency in the distributed database. Moreover, such methods and systems can be implemented cost-effectively without requiring the replacement of all hardware in an existing system.

[0070] Unless otherwise stated, the foregoing alternative examples are not mutually exclusive, but can be implemented in various combinations to achieve unique advantages. The foregoing description of embodiments should be understood through illustration rather than through limitation by the subject matter defined by the claims, when these and other variations and combinations of the features discussed above can be utilized without departing from the subject matter defined by the claims. Furthermore, the provision of examples described herein and phrases such as “such as,” “comprising,” etc., should not be interpreted as limiting the subject matter of the claims to specific examples; rather, the examples are intended to illustrate only one of many possible embodiments. Further, the same reference numerals in different figures can identify the same or similar elements.

Claims

1. A system for concurrency control, comprising: a server device comprising at least one input / output to communicate with other servers and clients in a distributed computing environment, the server device comprising: one or more processors configured to: receive a request to write first data; write the first data to a memory in the distributed computing environment; while the first data is being committed to the memory, effects of the first data are not visible to at least one of a client library, the other servers, or the clients for a time interval without preventing second data from being written to the memory during the time interval.

2. The system of claim 1, wherein, the effects of the first data are not visible to both the client library and the other servers during the time interval.

3. The system of claim 2, wherein, the client library notifies one of the clients that the first data was committed after the time interval.

4. The system of claim 2, wherein, the other servers perform transaction reads during the time interval.

5. The system of claim 1, wherein, the time interval is imposed on the clients, and wherein the one or more processors are further configured to: assign a timestamp to the first data, the timestamp equal to a first time plus the interval; and release a lock after assigning the timestamp.

6. The system of claim 5, wherein, any of the clients must wait until the assigned timestamp has passed before reading the first data.

7. The system of claim 6, wherein, the assigned timestamp has passed when a local clock at the client reading the first data reflects a current time later than the assigned timestamp.

8. The system of claim 5, wherein, the first time is a current time reflected by a local clock at the server when the server assigns the timestamp to the first data.

9. The system of claim 8, wherein, the interval corresponds to a time difference between a latest time reported by all clocks in the distributed computing environment and an earliest time reported by all clocks in the distributed computing environment.

10. The system of claim 9, wherein, the interval is maintained by the local clock at the server.

11. A method for concurrency control, comprising: receiving, at a first computing device from one or more clients, a request to write first data to a distributed computing environment; obtaining, at the first computing device, a write lock; and writing, by the first computing device, the first data to a memory in the distributed computing environment; wherein while the first data is being committed to the memory, effects of the first data are not visible to at least one of a client library, other servers, or clients for a time interval without preventing second data from being written to the memory during the time interval. the effects of the first data are not visible to both the client library and the other servers during the time interval.

12. The method of claim 11, wherein, the client library notifies one of the clients that the first data was committed after the time interval.

13. The method of claim 12, wherein, the other servers perform transaction reads during the time interval.

14. The method of claim 12, wherein, the time interval is imposed on the clients, and further comprising:

15. The method of claim 11, wherein, ​ assigning a timestamp to the first data, the timestamp being equal to a first time plus the interval; releasing the lock after assigning the timestamp.

16. The method of claim 15, wherein, Any of the clients must wait until the assigned timestamp has passed before reading the data.

17. The method of claim 16, wherein, The assigned timestamp has passed when a local clock at the client reading the data reflects a current time later than the assigned timestamp.

18. The method of claim 15, wherein, The first time is a current time reflected by a local clock at the server when the server assigns the timestamp to the first data, and wherein the interval corresponds to a time difference between a latest time reported by all clocks in the distributed computing environment and an earliest time reported by all clocks in the distributed computing environment.

Images