Separate memory based single-writer b-tree architecture

Abstract

A method (500) for a split-memory based single-writer B-tree architecture, the method comprising receiving a write request (20) for a distributed database (152), the write request requesting a data processing hardware (144) to update the distributed database. The distributed database is indexed using a B-tree (154) stored on a plurality of servers (158). Each server of the plurality of servers stores a portion (156) of the B-tree. The method comprises modifying a portion (212) of a fixed-size buffer pool (210) using the write request. The fixed-size buffer pool is stored at a local memory (159) of a master server of the plurality of servers and corresponds to a portion of the B-tree. The method comprises, in response to modifying the portion of the fixed-size buffer pool, writing the modified portion of the fixed-size buffer pool to a respective server of the plurality of servers that stores the corresponding portion of the B-tree.

Description

Technical Field

[0001] This disclosure relates to a single-writer B-tree architecture for decoupled memory. Background Technology

[0002] Following the recent success of separate storage and compute in cloud database systems, there has been a growing interest in memory-separated architectures. Tree-like data structures, such as B-trees, are important data structures for traditional database indexes. However, these traditional designs cannot be translated into high-performance distributed setups, such as those required by many applications using separate memory architectures. Summary of the Invention

[0003] One aspect of this disclosure provides a method for a single-writer B-tree architecture based on discrete memory. The computer-implemented method causes data processing hardware to perform operations when executed by the data processing hardware. These operations include receiving a write request for a distributed database, the write request requesting the data processing hardware to update the distributed database. The distributed database is indexed using a B-tree stored on multiple servers. Each of the multiple servers stores a portion of the B-tree. These operations include modifying a portion of a fixed-size buffer pool using the write request. The fixed-size buffer pool is stored in the local memory of the master server among the multiple servers and corresponds to a portion of the B-tree. These operations include: in response to modifying that portion of the fixed-size buffer pool, writing the modified portion of the fixed-size buffer pool to the corresponding server among the multiple servers that stores the corresponding portion of the B-tree.

[0004] Implementations of this disclosure may include one or more of the following optional features. In some implementations, writing the modified portion of the fixed-size buffer pool includes using an in-memory cluster-level file system. In some of these implementations, each write request is mapped to a single transaction of the in-memory cluster-level file system. Optionally, writing the modified portion of the fixed-size buffer pool includes: determining, before writing the modified portion of the fixed-size buffer pool, that another pending write based on a second write request has modified the same portion of the fixed-size buffer pool; and, based on determining that the other pending write has modified the same portion of the fixed-size buffer pool, writing the modified portion of the fixed-size buffer pool to the respective server of the plurality of servers in an order based on the order in which the write request and the second write request were received.

[0005] In some examples, writing the modified portion of the fixed-size buffer pool includes: determining, before writing the modified portion of the fixed-size buffer pool, that another pending write based on a second write request has modified a different portion of the fixed-size buffer pool; and, based on determining that the other pending write has modified the different portion of the fixed-size buffer pool, writing the modified portion of the fixed-size buffer pool to the respective servers of the plurality of servers in an order not based on the order in which the write request and the second write request were received. Optionally, writing the modified portion of the fixed-size buffer pool includes: determining, before writing the modified portion of the fixed-size buffer pool, that another pending write based on a second write request has modified the same portion of the fixed-size buffer pool; and, in response to determining that the other pending write has modified the same portion of the fixed-size buffer pool, simultaneously writing the modified portion based on the write request and the modified portion based on the second write request to the respective server.

[0006] In some examples, the modified portion of writing to the fixed-size buffer pool includes enqueuing (pushing) writes for that modified portion of the fixed-size buffer pool into a first-in, first-out (FIFO) data structure. In some of these examples, the modified portion of writing to the fixed-size buffer pool further includes generating a dependency graph using each write stored in the FIFO. The modified portion of writing to the fixed-size buffer pool may further include batching multiple writes together based on the dependency graph.

[0007] These operations further include receiving a read request for the distributed database, the read request requesting the data processing hardware to read data from the distributed database. These operations may also include: based on receiving the read request, retrieving one or more portions of the B-tree from a second subset of the plurality of servers, and storing the one or more portions of the B-tree in a second fixed-size buffer pool for the B-tree. The second fixed-size buffer pool is stored in the local memory of a secondary server among the plurality of servers, and this secondary server is different from the primary server. These operations may also include retrieving the data using the second fixed-size buffer pool.

[0008] Another aspect of this disclosure provides a system for a single-writer B-tree architecture based on discrete memory. The system includes data processing hardware and memory hardware in communication with the data processing hardware. The memory hardware stores instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations. These operations include receiving a write request for a distributed database, the write request requesting the data processing hardware to update the distributed database; indexing the distributed database using a B-tree stored on multiple servers; each of the multiple servers storing a portion of the B-tree; modifying a portion of a fixed-size buffer pool using the write request; the fixed-size buffer pool being stored in the local memory of a master server among the multiple servers and corresponding to a portion of the B-tree; and writing the modified portion of the fixed-size buffer pool to the corresponding server among the multiple servers storing the corresponding portion of the B-tree in response to the modification of that portion of the fixed-size buffer pool.

[0009] This aspect may include one or more of the following optional features. In some implementations, writing the modified portion of the fixed-size buffer pool includes using an in-memory cluster-level file system. In some of these implementations, each write request is mapped to a single transaction of the in-memory cluster-level file system. Optionally, writing the modified portion of the fixed-size buffer pool includes: determining, before writing the modified portion of the fixed-size buffer pool, that another pending write based on a second write request has modified the same portion of the fixed-size buffer pool; and, based on determining that the other pending write has modified the same portion of the fixed-size buffer pool, writing the modified portion of the fixed-size buffer pool to the respective server of the plurality of servers in an order based on the order in which the write request and the second write request were received.

[0010] In some examples, writing the modified portion of the fixed-size buffer pool includes: determining, before writing the modified portion of the fixed-size buffer pool, that another pending write based on a second write request has modified a different portion of the fixed-size buffer pool; and, based on determining that the other pending write has modified the different portion of the fixed-size buffer pool, writing the modified portion of the fixed-size buffer pool to the respective servers of the plurality of servers in an order not based on the order in which the write request and the second write request were received. Optionally, writing the modified portion of the fixed-size buffer pool includes: determining, before writing the modified portion of the fixed-size buffer pool, that another pending write based on a second write request has modified the same portion of the fixed-size buffer pool; and, in response to determining that the other pending write has modified the same portion of the fixed-size buffer pool, simultaneously writing the modified portion based on the write request and the modified portion based on the second write request to the respective server.

[0011] In some examples, the modified portion of writing to the fixed-size buffer pool includes enqueuing (pushing) writes for that modified portion of the fixed-size buffer pool into a first-in, first-out (FIFO) data structure. In some of these examples, the modified portion of writing to the fixed-size buffer pool further includes generating a dependency graph using each write stored in the FIFO. The modified portion of writing to the fixed-size buffer pool may further include batching multiple writes together based on the dependency graph.

[0012] These operations further include receiving a read request for the distributed database, the read request requesting the data processing hardware to read data from the distributed database. These operations may also include: based on receiving the read request, retrieving one or more portions of the B-tree from a second subset of the plurality of servers, and storing the one or more portions of the B-tree in a second fixed-size buffer pool for the B-tree. The second fixed-size buffer pool is stored in the local memory of a secondary server among the plurality of servers, and this secondary server is different from the primary server. These operations may also include retrieving the data using the second fixed-size buffer pool.

[0013] Details of one or more implementations of this disclosure are set forth in the accompanying drawings and the following description. Other aspects, features, and advantages will become apparent from the specification, drawings, and claims. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of an example system based on a single-writer B-tree architecture with separate memory.

[0015] Figure 2 yes Figure 1 A schematic diagram of the B-tree writer of the system.

[0016] Figures 3A to 3C This is a diagram illustrating operations on nodes of a B-tree.

[0017] Figure 4 yes Figure 1 A schematic diagram of the B-tree reader of the system.

[0018] Figure 5 This is a flowchart of an example arrangement of operations for a method based on a single-writer B-tree architecture with decoupled memory.

[0019] Figure 6 This is a schematic diagram of an example computing device that can be used to implement the systems and methods described herein.

[0020] In the various figures, the same reference numerals indicate the same elements. Detailed Implementation

[0021] Building on recent successes with decoupled storage and compute in cloud database systems, there has been a growing interest in decoupled memory architectures. Decoupled memory refers to a paradigm that separates compute nodes from memory nodes to improve memory utilization and scalability. These cloud database systems use indexes to provide an efficient way to access records in the database. Tree data structures such as B-trees are important data structures for traditional database indexes. However, these traditional designs cannot be translated into high-performance distributed setups, such as those required by many applications using decoupled memory architectures.

[0022] The implementation described in this paper refers to a B-tree controller that incorporates a high-performance single-write / multiple-reader design on discrete memory. The B-tree controller can be integrated with cluster-level file systems to modify B-trees stored across multiple computers or servers (i.e., discrete memory) without relying on performance-degrading global locking tables. The B-tree controller can be used, for example, as a database index or a general-purpose distributed in-memory key-value store.

[0023] refer to Figure 1In some implementations, the cloud database system 100 includes a remote system 140 that communicates with one or more user devices 10 via a network 112. The remote system 140 can be a single computer, multiple computers, or a distributed system (e.g., a cloud environment) with scalable / elastic resources 142, including computing resources 144 (e.g., data processing hardware) and / or storage resources 146 (e.g., memory hardware). A data repository 150 (i.e., a remote storage device) can be placed on the storage resource 146 to allow clients (e.g., user devices 10) or one or more of the computing resources 144 to scalably utilize the storage resource 146. The data repository 150 can be configured to store one or more databases 152 or tables (i.e., cloud databases).

[0024] Remote system 140 is configured to receive database queries 20 (i.e., requests 20) from user devices 10 (each user device is associated with a corresponding user 12) via, for example, network 112. Each user device 10 may correspond to any computing device (such as a desktop workstation, laptop workstation, or mobile device (i.e., smartphone)). User device 10 includes computing resources 18 (e.g., data processing hardware) and / or storage resources 16 (e.g., memory hardware). User 12 may construct database queries 20 (also referred to herein as write requests 20) using Structured Query Language (SQL) interface 14, but may also use other interfaces. Database query 20 requests remote system 140 to query one or more databases 152 or to interact with one or more databases. For example, query 20 may request the database to conditionally return data from database 152, add additional data to database 152, and / or modify data in database 152. Any number of users 12 or user devices 10 may query database 152 simultaneously or in parallel. For example, database 152 is a distributed cloud database that simultaneously serves hundreds or thousands (or more) of users. Other entities (e.g., applications running on remote system 140 or other remote servers) may interact with the database additionally or alternatively.

[0025] In some implementations, database 152 has a database index 154. Database index 154 is a self-balancing tree data structure or a B-tree 154. A B-tree is a tree that starts at a root node and ends at leaf nodes. Each node includes one or more keys, and these keys act as separator values ​​for partitioning subtrees. Database index 154 improves the speed of retrieval operations on database 152. Database index 154 is divided into multiple parts 156, 156a-n (e.g., pages, blocks, or any other partition of database index 154). Parts 156 are distributed across multiple servers 158, 158a-n, or other compute / memory nodes. Each server 158 includes local memory 159 that stores one or more parts 156 of database index 154 (i.e., B-tree 154). Server 158 may be part of a remote system 140 to provide a distributed memory architecture for user 12.

[0026] Remote system 140 executes B-tree controller 160. B-tree controller 160 receives write requests 20 for distributed database 152. B-tree controller 160 includes a buffer pool 210. Buffer pool 210 maintains a copy of at least a portion of the B-tree in the local memory or other computing resources of one of the servers 158 (i.e., the master server 158 or the writer server 158). The master server 158 may have different computing resources (e.g., more resources) than other servers 158 that are not the master server 158. Buffer pool 210 includes multiple buffer sections 212, each buffer section 212 corresponding to a corresponding B-tree section 156. That is, buffer pool 210 maintains a local copy or "cache" of at least a portion of the B-tree 152 in the local memory (e.g., local RAM) of the master server 158.

[0027] In some examples, buffer pool 210 is a fixed-size buffer pool that uses an eviction policy to maintain buffer portions 212 corresponding to the most relevant (e.g., most frequently or recently accessed) B-tree portions 156. In some implementations, the fixed-size buffer pool is smaller than the database index 154 and therefore can only store a portion of the database index 154. When the fixed-size buffer pool 210 is full and a write request 20 references a B-tree portion 156 that is not currently in the buffer pool 210, the B-tree controller 160 can evict the existing buffer portion 212 to make room for a new buffer portion 212. In some examples, the B-tree controller 160 evicts buffer portions 212 that have not been accessed for a long time (via reads and / or via writes).

[0028] B-tree controller 160 uses write request 20 to modify buffer pool 210. That is, B-tree controller 160 updates the buffer pool based on the data and / or location of write request 20 to maintain database index 154. For example, B-tree controller 160 updates one or more buffer portions 212 to reflect the required updates to database index 154 due to write request 20 (i.e., due to a write to database 152). B-tree controller 160 updates the buffer portion 212 corresponding to the respective B-tree portion 156 associated with write request 20.

[0029] B-tree controller 160 includes B-tree writer 220. As discussed in more detail below, B-tree writer 220, in response to a modification to buffer pool 210, writes the modified buffer portion 212 to the corresponding server 158 storing the corresponding B-tree portion 156 of database index 154. Optionally, B-tree writer 220 writes the modified buffer portion 212 to server 158 as quickly as possible (bandwidth and system resources allow). This contrasts with conventional B-trees, where updates are typically flushed only when the buffer overflows or based on some other opportunistic event. In some examples, B-tree writer 220 performs (e.g., as a background process) to asynchronously write the buffer portion 212 to the corresponding server 158. Optionally, B-tree writer 220 uses a dedicated thread pool and does not need to block the completion of the write. The B-tree controller 160 can use write-ahead logging (WAL) technology to update or maintain logs for tracking updates to database 152, enabling recovery and replay purposes.

[0030] Now for reference Figure 2 In some implementations, the B-tree writer 220 writes the modified buffer portion 212 to the server 158 via the cluster-level file system 230. The cluster-level file system 230 can utilize principles from Remote Memory Access (RMA). Optionally, the cluster-level file system 230 is based on one-sided RMA (i.e., no computation is required on the remote end / server 158). The cluster-level file system 230 provides native transactions with transactional writes. Optionally, each write request 20 is mapped to a single transaction on the cluster-level file system 230. That is, regardless of how many nodes are modified by the write, each B-tree update request is mapped to a single transaction on the cluster-level file system 230. The cluster-level file system 230 can provide an abstraction layer for decoupled memory (i.e., the splitting of the B-tree portion 156 in the server 158) according to the B-tree controller 160.

[0031] In some implementations, B-tree writer 220 enqueues each pending write to cluster-level file system 230 (i.e., writes to modify / update B-tree portion 156) into queue 222. Optionally, queue 222 is a first-in-first-out (FIFO) queue to maintain the order of writes. B-tree writer 220 can enqueue or flush these writes to queue 222 immediately after receiving them from B-tree controller 160, instead of waiting for buffer pool 210 to overflow. Optionally, B-tree writer 220 is an asynchronous background process that writes to cluster-level file system 230 without blocking.

[0032] In some examples, based on writes within queue 222, B-tree writer 220 generates a dependency graph 224. Dependency graph 224 defines the dependencies between writes within queue 222 (i.e., describes changes to B-tree 154 resulting from one or more pending writes in queue 222 that have not yet been written to server 158). Dependencies between writes indicate that the order in which writes occur is important. Writes without any dependencies can be written to server 158 in any order relative to each other (i.e., arbitrary order). In some examples, the B-tree writer is multi-threaded (i.e., two or more threads are simultaneously preparing writes to the cluster-level file system 230). Dependency graph 224 explicitly captures the order relationships among writes within queue 222 to ensure that dependencies are followed when flushing the queue. For example, B-tree writer 220 uses locking algorithms (such as node-level read-write locks) to ensure that two threads cannot simultaneously attempt to modify the same node in B-tree 154. Dependency diagram 224 can represent a flush buffer with a configurable maximum size, which can exert backpressure on incoming write requests 20. B-tree writer 220 can empty queue 222 as quickly as allowed by the bandwidth / throughput of server 158 via cluster-level file system 230.

[0033] Now for reference Figures 3A to 3CB-tree writer 220 can ensure that dependent writes are written to server 158 in the order defined by dependency diagram 224. For example, B-tree writer 220 determines a first pending write 310 based on a first write request 20 (i.e., a write or B-tree operation enqueued in queue 222 originating from a write request 20 that modifies one or more nodes 312 of B-tree 154) and another second pending write 310 based on a second write request 20, each modifying the same portion 212 of fixed-size buffer pool 210 (e.g., modifying or affecting the same node 312 of B-tree 154). Based on the determination that the writes affect the same portion 212, B-tree writer 220 writes the modified portion 212 of fixed-size buffer pool 210 to the corresponding server 158 in the order in which the first write request 20 and the second write request 20 are received. In another example, B-tree writer 220 determines that a first pending write 310 based on a first write request 20 and another second pending write 310 based on a second write request 20 do not modify the same portion 212 of a fixed-size buffer pool 210 (e.g., modify or affect different nodes 312 of the B-tree 154). Based on the determination that the writes do not affect the same portion 212, B-tree writer 220 writes the modified portion 212 of the fixed-size buffer pool 210 to the corresponding server 158 in an order not based on the order in which the first and second write requests 20 are received.

[0034] In some examples, for each B-tree operation (i.e., for each write to queue 222), B-tree writer 220 determines an increment set for that operation, which defines the changes to each node of B-tree 154 for that operation. For example, when the operation does not include node splits (which may be the case for most operations), the increment set contains only leaf node changes. Optionally, B-tree writer 220 batches operations affecting the same node in the flush buffer or pending operations in queue 222 together (i.e., puts the operations into the same transaction for cluster-level file system 230). Each increment set can always be included in the same transaction of cluster-level file system 230 (i.e., the changes represented by the increment set never split across multiple transactions) to keep B-tree 154 always valid and simplify recovery.

[0035] In some examples, B-tree writer 220 periodically batches or combines writes 310 from queue 222 to improve throughput through cluster-level file system 230. For example, B-tree writer 220 determines that a first pending write 310 based on a first write request 20 and a second pending write 310 based on a second write request 20 each modify the same portion of buffer pool 210 (e.g., modifying overlapping nodes 312 of B-tree 154). In response to determining that the pending writes modified the same portion of buffer pool 210, B-tree writer 220 simultaneously writes the modified portions 156 from both writes 310 to the corresponding servers 158.

[0036] B-tree writer 220 can use dependency graph 224 and / or increment set to coordinate batching and flushing or write order. That is, B-tree writer 220 can batch writes 310 together based on dependency graph 224 and / or increment set. For example, if (e.g., in queue 222) two pending B-tree 154 updates overlap in the set of modified nodes 312, these two updates are merged or batched together into a single transaction to save round trips through cluster-level file system 230. Accordingly, reads and writes through cluster-level file system 230 do not conflict with each other to improve performance. In some implementations, B-tree controller 160 prevents node 312 from being evicted from B-tree buffer 210 while node 312 that needs to be flushed (i.e., committed to B-tree 154 on server 158) still has changes. Therefore, such a node 312 will never trigger a buffer pool cache miss, and writes and reads caused by cache misses will always be disjoint sets of nodes and will correspondingly not conflict. Furthermore, since all writes are coordinated according to dependency graph 224, there will be no conflicts between reads and writes.

[0037] exist Figure 3A and Figure 3B In the example, schematic diagram 300A ( Figure 3A This includes a first write 310, 310A (i.e., the first operation or write originating from write request 20 and placed in queue 222), and schematic diagram 300B ( Figure 3B This includes the second writes 310 and 310B. The first write 310A and the second write 310B are two insertion operations affecting the same parent node 312 and 312a. More specifically, the first write 310A splits the second node 312 and 312b of the first node 312a into the second node 312b and the third node 312 and 312c. The second write 310B splits the fourth node 312 and 312d of the first node 312a into the fourth node 312d and the fifth node 312 and 312e. Figure 3CAs shown in schematic diagram 300B, B-tree writer 220 can batch two write operations 310A, 310B into a single batched write 310, 310C that performs two inserts simultaneously (i.e., a batched transaction). B-tree writer 220 can also periodically batch many other combinations of writes 310. As a simple example, B-tree writer 220 can batch multiple consecutive inserts that modify the same leaf node 312 together. In this example, B-tree writer 220 can only write the last or most recent write value to node 312. More specifically, if write 310 modifies a specific node 312 to have a <key, value> pair with <key 1, value 5> and then a subsequent write 310 modifies the same specific node 312 to have a <key, value> pair with <key 2, value 7>, then the B-tree writer 220 can either discard the first write 310 and only flush or commit the second write, or perform both modifications within a single transaction of the cluster-level file system 230. In some examples, the B-tree writer 220 limits the number of writes 310 that are batched together based on a configurable maximum batch threshold.

[0038] Now for reference Figure 4 In some implementations, the B-tree controller 160 includes one or more B-tree readers 410. Each B-tree reader 410 can execute in the local memory of the secondary server 158 to serve read requests for the B-tree 154. In some examples, the primary server 158 (i.e., the server hosting the B-tree writer 220) is also a secondary server 158 (i.e., the server hosting the B-tree reader 410) in addition to one or more other servers 158. The primary server 158 always has the latest B-tree 154, without any obsolescence.

[0039] B-tree controller 160 may include any number of B-tree readers 410 (i.e., a single-writer, multiple-reader architecture) executing on any number of servers 158. For example, in addition to the master server 158, one or more servers 158 may also host B-tree readers 410 to prevent the master server 158 from becoming overloaded. B-tree readers 410 receive read requests 402 for distributed database 152, which request data processing to read data from distributed database 152. Based on the received read request 402, B-tree readers 410 retrieve one or more portions 156 of B-tree 154 from one or more other servers 158 (i.e., a subset of servers 158). For example, B-tree readers 410 maintain a B-tree reader buffer 420 (i.e., a second fixed-size buffer pool) in local memory and update the B-tree reader buffer 420 based on B-tree portions 156 received via, for example, a cluster-level file system 230. In other words, B-tree reader 410 stores one or more portions 156 of B-tree 154 in a second fixed-size buffer 420. B-tree reader 410 uses B-tree reader buffer 4230 to retrieve read data requested by read request 402 (or directs another module or system to the location of read data for retrieval).

[0040] B-tree reader buffer 420 serves as a local cache for a portion of B-tree 154 for B-tree reader 410. B-tree reader buffer 420 may include an eviction policy similar to that of B-tree buffer pool 210. Before relying on B-tree reader buffer 420, the B-tree reader can determine whether B-tree reader buffer 420 is up-to-date (i.e., whether B-tree writer 220 has performed any relevant updates to B-tree 154, causing B-tree reader buffer 420 to become invalid or obsolete). To improve performance and invalidate poor traversals caused by concurrent writes, in some implementations, B-tree reader 410 determines the predicted position of leaf node 312 to be read from B-tree 154 based on B-tree reader buffer 420. B-tree reader 410 then determines whether the predicted position of leaf node 312 is accurate by reading it from B-tree 154 using cluster-level file system 230. When the prediction is correct, B-tree reader 410 can determine that B-tree reader buffer 420 is new enough and no additional reads from cluster-level file system 230 are required. When the prediction is incorrect, B-tree reader 410 can determine that B-tree reader buffer 420 is outdated and retrieve the path (i.e., node 312 from parent node 312 to leaf node 312) via cluster-level file system 230 to update B-tree reader buffer 420.

[0041] In some examples, each node 312 includes a fence key that defines the range of keys that node 312 is responsible for, even if those keys do not exist in the tree. Based on the fence key (i.e., fence key verification), the B-tree reader 410 can quickly traverse the B-tree reader buffer 420 to predict the location of the corresponding leaf node 312. Optionally, the B-tree controller 160 exposes a watermark in the queue 222 or the flush buffer to allow the B-tree reader to determine the freshness of a read. For example, a large number of operations in the queue 222 or the flush buffer relative to a smaller number of operations in the queue 222 may indicate that the read is older. The B-tree reader 410 can schedule read operations based on the watermark (e.g., delaying reads until the watermark is sufficiently low).

[0042] Therefore, the implementation described in this paper includes a single-writer B-tree architecture based on distributed memory. The B-tree 154 can be used as a primary or secondary index for a database (such as a cloud database 152) (i.e., for storing the mapping between primary keys and row positions). The B-tree 154 can be used as a general-purpose, high-performance distributed data structure (such as for high-performance in-memory key-value stores that can scale to larger sizes than local memory while still providing excellent durability). The B-tree reader 410 reads data directly from local memory in a lock-free manner without impacting write performance because the load on each server 158 is typically very low (i.e., because the B-tree 154 is distributed across a sufficient number of servers 158). The single-writer B-tree writer 220 provides high performance by maximizing the throughput of the cluster-level file system 230 by utilizing immediate and opportunistic batch processing. In some examples, the database can be sharded such that it is split into multiple partitions. In these examples, each shard or partition can have a single writer (i.e., B-tree writer 220). The implementation described in this paper is compatible with replicated remote storage for disaster recovery, as any replication scheme can be used for remote memory storage. Furthermore, the B-tree controller 160 utilizes one-sided remote direct memory access (RDMA), allowing remote memory hosts (i.e., servers 158) to perform no computations when reading their corresponding B-tree portions 156.

[0043] Figure 5This is a flowchart illustrating an example arrangement of operations for method 500, which uses a single-writer B-tree architecture based on discrete memory. The computer-implemented method 500 causes data processing hardware 144 to perform operations when executed by data processing hardware 144. At operation 502, method 500 includes receiving a write request 20 for a distributed database 152. The write request requests data processing hardware 144 to update the distributed database 152. The distributed database 152 is indexed using a B-tree 154 stored on multiple servers 158. Each of the multiple servers 158 stores a portion 156 of the B-tree 154. At operation 504, method 500 includes modifying a portion 212 of a fixed-size buffer pool 210 using the write request 20. The fixed-size buffer pool 210 is stored in the local memory 159 of the master server 158 among the multiple servers 158. The portion 212 of the fixed-size buffer pool 210 corresponds to the portion 156 of the B-tree 154. At operation 506, in response to modifying portion 212 of a fixed-size buffer pool 210, method 500 includes writing the modified portion 212 of the fixed-size buffer pool 210 to the corresponding portion 156 of a storage B-tree 154 in a plurality of servers 158.

[0044] Figure 6 This is a schematic diagram of an example computing device 600 that can be used to implement the systems and methods described in this document. The computing device 600 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The components shown herein, their connections and relationships, and their functions are intended to be exemplary only and are not intended to limit the implementations of the invention described and / or claimed in this document.

[0045] Computing device 600 includes a processor 610, memory 620, storage device 630, a high-speed interface / controller 640 connected to memory 620 and high-speed expansion port 650, and a low-speed interface / controller 660 connected to low-speed bus 670 and storage device 630. Each of components 610, 620, 630, 640, 650, and 660 is interconnected using various buses and can be mounted on a common motherboard or otherwise. Processor 610 can process instructions for execution within computing device 600, including instructions stored in memory 620 or storage device 630, to display graphical information of a graphical user interface (GUI) on an external input / output device such as a display 680 coupled to high-speed interface 640. In other implementations, multiple processors and / or multiple buses, as well as multiple memories and various types of memory, can be used as appropriate. Furthermore, multiple computing devices 600 can be connected, with each device providing a portion of the necessary operation (e.g., as a server group, blade server cluster, or multiprocessor system).

[0046] Memory 620 stores information non-temporarily within computing device 600. Memory 620 may be a computer-readable medium, a volatile memory cell, or a non-volatile memory cell. Non-temporary memory 620 may be a physical means for temporarily or permanently storing programs (e.g., instruction sequences) or data (e.g., program state information) for use by computing device 600. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electronically erasable programmable read-only memory (EEPROM) (e.g., commonly used in firmware, such as boot programs). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase-change memory (PCM), and magnetic disks or magnetic tapes.

[0047] Storage device 630 provides mass storage for computing device 600. In some implementations, storage device 630 is a computer-readable medium. In various implementations, storage device 630 may be a floppy disk device, hard disk device, optical disk device, magnetic tape device, flash memory or other similar solid-state memory device, or device array (including devices arranged as a storage area network or other configuration). In additional implementations, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer or machine-readable medium, such as memory 620, storage device 630, or memory on processor 610.

[0048] High-speed controller 640 manages bandwidth-intensive operations of computing device 600, while low-speed controller 660 manages lower bandwidth-intensive operations. This assignment of responsibilities is merely exemplary. In some implementations, high-speed controller 640 is coupled to memory 620, display 680 (e.g., via a graphics processor or accelerator), and high-speed expansion port 650 which can accept various expansion cards (not shown). In some implementations, low-speed controller 660 is coupled to storage device 630 and low-speed expansion port 690. Low-speed expansion port 690, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, Wireless Ethernet), may be coupled to one or more input / output devices, such as keyboards, pointing devices, scanners, or networking devices, such as switches or routers, for example, via a network adapter.

[0049] The computing device 600 can be implemented in a variety of different forms, as shown in the figure. For example, it can be implemented as a standard server 600a or multiple implementations as a group of such servers 600a, as a laptop computer 600b, or as part of a rack server system 600c.

[0050] Various implementations of the systems and techniques described herein can be implemented in digital electronic and / or optical circuit systems, integrated circuit systems, specially designed ASICs (Application-Specific Integrated Circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system, which includes at least one programmable processor, which may be dedicated or general-purpose, and is coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to transfer data and instructions to the storage system, at least one input device, and at least one output device.

[0051] A software application (i.e., a software resource) can refer to computer software that instructs a computing device to perform a task. In some examples, a software application may be referred to as an "application," "app," or "program." Example applications include, but are not limited to, system diagnostic applications, system management applications, system maintenance applications, word processing applications, spreadsheet applications, messaging applications, media streaming applications, social networking applications, and game applications.

[0052] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, non-transitory computer-readable medium, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0053] The processes and logic flows described in this specification can be executed by one or more programmable processors, also known as data processing hardware, which execute one or more computer programs to perform functions by manipulating input data and generating output. The processes and logic flows can also be executed by special-purpose logic circuit systems, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). For example, processors suitable for executing computer programs include both general-purpose microprocessors and special-purpose microprocessors, as well as any one or more processors of any type of digital computer. Typically, the processor receives instructions and data from read-only memory or random access memory, or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or operatively coupled to receive data from or transfer data to said mass storage device, or both. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry systems.

[0054] To provide interaction with the user, one or more aspects of this disclosure can be implemented on a computer having a display device for displaying information to the user (e.g., a CRT (cathode ray tube), LCD (liquid crystal display) monitor, or touchscreen) and possibly a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including sound, speech, or tactile input. Additionally, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a webpage to a web browser on the user's client device in response to a request received from a web browser.

[0055] Various implementations have been described. However, it should be understood that various modifications can be made without departing from the spirit and scope of this disclosure. Therefore, other implementations are within the scope of the following claims.

Claims

1. A computer-implemented method (500) executed by data processing hardware (144), the computer-implemented method causing the data processing hardware (144) to perform operations, characterized in that, The operation includes: A write request (20) is received for a distributed database (152), the write request (20) requesting the data processing hardware (144) to update the distributed database (152), and to index the distributed database (152) using a B-tree (154), the B-tree (154) being stored on multiple servers (158), each of the multiple servers (158) storing a portion (156) of the B-tree (154); The write request (20) modifies a portion (212) of a fixed-size buffer pool (210), which is stored in the local memory (159) of the master server (158) among the plurality of servers (158), and the portion (212) of the fixed-size buffer pool (210) corresponds to a portion (156) of the B-tree (154); and In response to modifying the portion (212) of the fixed-size buffer pool (210), the modified portion (212) of the fixed-size buffer pool (210) is written to the corresponding server (158) of the plurality of servers (158) that stores the corresponding portion (156) of the B-tree (154).

2. The method (500) as described in claim 1, characterized in that, The modified portion (212) of writing to the fixed-size buffer pool (210) includes the use of an in-memory cluster-level file system (230).

3. The method (500) as described in claim 2, characterized in that, Each write request (20) is mapped to a single transaction in the in-memory cluster-level file system (230).

4. The method (500) according to any one of claims 1 to 3, characterized in that, The modified portion (212) written into the fixed-size buffer pool (210) includes: Before writing the modified portion (212) of the fixed-size buffer pool (210), it is determined that another pending write (310) based on the second write request (20) has modified the same portion (212) of the fixed-size buffer pool (210); and Based on the determination of the other pending write (310), the same portion (212) of the fixed-size buffer pool (210) is modified to write the modified portion (212) of the fixed-size buffer pool (210) to the respective server (158) of the plurality of servers (158) in the order in which the write request (20) and the second write request (20) are received.

5. The method (500) according to any one of claims 1 to 4, characterized in that, The modified portion (212) written into the fixed-size buffer pool (210) includes: Before writing the modified portion (212) of the fixed-size buffer pool (210), it is determined that another pending write (310) based on the second write request (20) has modified a different portion (212) of the fixed-size buffer pool (210); and Based on the determination of the other pending write (310), the different portions (212) of the fixed-size buffer pool (210) are modified to write the modified portions (212) of the fixed-size buffer pool (210) to the respective servers (158) of the plurality of servers (158) in an order not based on the order in which the write request (20) and the second write request (20) are received.

6. The method (500) according to any one of claims 1 to 5, characterized in that, The modified portion (212) written into the fixed-size buffer pool (210) includes: Before writing the modified portion (212) of the fixed-size buffer pool (210), it is determined that another pending write (310) based on the second write request (20) has modified the same portion (212) of the fixed-size buffer pool (210); and In response to determining that the other pending write (310) has modified the same portion (212) of the fixed-size buffer pool (210), the modified portion (212) based on the write request (20) and the modified portion (212) based on the second write request (20) are simultaneously written to the corresponding server (158).

7. The method (500) according to any one of claims 1 to 6, characterized in that, The modified portion (212) written to the fixed-size buffer pool (210) includes enqueuing the write (310) to the modified portion (212) of the fixed-size buffer pool (210) into a first-in-first-out (FIFO) data structure (222).

8. The method (500) as described in claim 7, characterized in that, The modified portion (212) of writing to the fixed-size buffer pool (210) further includes generating a dependency graph (224) using each write (310) stored in the FIFO (222).

9. The method (500) as described in claim 8, characterized in that, The modified portion (212) of writing to the fixed-size buffer pool (210) further includes batching multiple writes (310) together based on the dependency graph (224).

10. The method (500) according to any one of claims 1 to 9, characterized in that, The operation further includes: Receive a read request (402) for the distributed database (152), the read request (402) requesting the data processing hardware (144) to read data from the distributed database (152); Based on the received read request (402), one or more portions (156) of the B-tree (154) are retrieved from a second subset of the plurality of servers (158); One or more portions (156) of the B-tree (154) are stored in a second fixed-size buffer pool (420) for the B-tree (154), the second fixed-size buffer pool (420) being stored in the local memory (159) of a secondary server (158) among the plurality of servers (158), the second server (158) being different from the primary server (158); and The data is retrieved using the second fixed-size buffer pool (420).

11. A system (100), characterized in that, include: Data processing hardware (144); as well as A memory hardware (146) communicating with the data processing hardware (144), the memory hardware (146) storing instructions that, when executed on the data processing hardware (144), cause the data processing hardware (144) to perform operations, the operations including: A write request (20) is received for a distributed database (152), the write request (20) requesting the data processing hardware (144) to update the distributed database (152), and to index the distributed database (152) using a B-tree (154), the B-tree (154) being stored on multiple servers (158), each of the multiple servers (158) storing a portion (156) of the B-tree (154); The write request (20) modifies a portion (212) of a fixed-size buffer pool (210), which is stored in the local memory (159) of the master server (158) among the plurality of servers (158), and the portion (212) of the fixed-size buffer pool (210) corresponds to a portion (156) of the B-tree (154); and In response to modifying the portion (212) of the fixed-size buffer pool (210), the modified portion (212) of the fixed-size buffer pool (210) is written to the corresponding server (158) of the plurality of servers (158) that stores the corresponding portion (156) of the B-tree (154).

12. The system (100) as claimed in claim 11, characterized in that, The modified portion (212) of writing to the fixed-size buffer pool (210) includes the use of an in-memory cluster-level file system (230).

13. The system (100) as claimed in claim 12, characterized in that, Each write request (20) is mapped to a single transaction in the in-memory cluster-level file system (230).

14. The system (100) as claimed in any one of claims 11 to 13, characterized in that, The modified portion (212) written into the fixed-size buffer pool (210) includes: Before writing the modified portion (212) of the fixed-size buffer pool (210), it is determined that another pending write (310) based on the second write request (20) has modified the same portion (212) of the fixed-size buffer pool (210); and Based on the determination of the other pending write (310), the same portion (212) of the fixed-size buffer pool (210) is modified to write the modified portion (212) of the fixed-size buffer pool (210) to the respective server (158) of the plurality of servers (158) in the order in which the write request (20) and the second write request (20) are received.

15. The system (100) as claimed in any one of claims 11 to 14, characterized in that, The modified portion (212) written into the fixed-size buffer pool (210) includes: Before writing the modified portion (212) of the fixed-size buffer pool (210), it is determined that another pending write (310) based on the second write request (20) has modified a different portion (212) of the fixed-size buffer pool (210); and Based on the determination of the other pending write (310), the different portions (212) of the fixed-size buffer pool (210) are modified to write the modified portions (212) of the fixed-size buffer pool (210) to the respective servers (158) of the plurality of servers (158) in an order not based on the order in which the write request (20) and the second write request (20) are received.

16. The system (100) as claimed in any one of claims 11 to 15, characterized in that, The modified portion (212) written into the fixed-size buffer pool (210) includes: Before writing the modified portion (212) of the fixed-size buffer pool (210), it is determined that another pending write (310) based on the second write request (20) has modified the same portion (212) of the fixed-size buffer pool (210); and In response to determining that the other pending write (310) has modified the same portion (212) of the fixed-size buffer pool (210), the modified portion (212) based on the write request (20) and the modified portion (212) based on the second write request (20) are simultaneously written to the corresponding server (158).

17. The system (100) as claimed in any one of claims 11 to 16, characterized in that, The modified portion (212) written to the fixed-size buffer pool (210) includes enqueuing the write (310) to the modified portion (212) of the fixed-size buffer pool (210) into a first-in-first-out (FIFO) data structure (222).

18. The system (100) as claimed in claim 17, characterized in that, The modified portion (212) of writing to the fixed-size buffer pool (210) further includes generating a dependency graph (224) using each write (310) stored in the FIFO (222).

19. The system (100) as claimed in claim 18, characterized in that, The modified portion (212) of writing to the fixed-size buffer pool (210) further includes batching multiple writes (310) together based on the dependency graph (224).

20. The system (100) as claimed in any one of claims 11 to 19, characterized in that, The operation further includes: Receive a read request (402) for the distributed database (152), the read request (402) requesting the data processing hardware (144) to read data from the distributed database (152); Based on the received read request (402), one or more portions (156) of the B-tree (154) are retrieved from a second subset of the plurality of servers (158); One or more portions (156) of the B-tree (154) are stored in a second fixed-size buffer pool (420) for the B-tree (154), the second fixed-size buffer pool (420) being stored in the local memory (159) of a secondary server (158) among the plurality of servers (158), the second server (158) being different from the primary server (158); and The data is retrieved using the second fixed-size buffer pool (420).

Images