Versioning of items in data structures

Abstract

A system comprising: a memory storing a data structure comprising a plurality of items, each item comprising a key-value pair; a writer arranged to perform a plurality of write operations, each write operation writing a respective item, either a new item being added to the data structure or an existing item in the data structure being modified; and a reader configured to perform a group read operation to read any items from the data structure having keys within a specified range. The writer is configured to maintain a global write version and the reader is configured to maintain a global read version.

Description

Background Technology

[0001] Tree structures such as B-trees can be used as indexed data structures to store key-value pairs (each pair consists of a value, i.e., what is being stored, and a key, used to index that value). Each leaf node of the tree contains multiple items, which are key-value pairs. Moving upwards from the leaves, each internal node of the tree contains an indication of the range of key values ​​contained in each of its child nodes (which can be leaves or other internal nodes).

[0002] When a new item is added to the tree, the writer uses the tree structure to determine which leaf contains the key range of the new item. If the leaf is not full, the new item can be added to an existing leaf. However, if a leaf is full (leaves and internal nodes typically have a maximum byte size), then the leaf will be split—that is, a new leaf will be created. This also means updating the parent nodes of both the old and new leaves so that the key ranges of both leaves are correctly referenced. If updating a parent node would also cause the parent node to exceed its maximum size, then the parent node itself will be split, and the grandparent references will be updated, and so on. If an item is deleted, this may also involve merging leaves or internal nodes.

[0003] In a given application, a writer or reader may need to perform many operations. A reader can read a single item with a specific key, or it can perform a range scan to read from a range of keys. A writer can write new entries, modify existing entries, or delete entries. Writing here can also be called a form of update because it updates the tree (not necessarily existing items). Writing can be adding new items to the tree or modifying existing items in the tree. More generally, other forms of tree updates include splitting nodes and merging nodes.

[0004] Indexed data structures are used in a wide range of software systems, such as data storage and databases, file systems, and operating systems. Indexed data structures store key-value pairs and support operations such as lookup, scan, insertion, and deletion of key-value pairs. B-trees are one such indexed data structure. B-trees are sequentially indexed, meaning they also support scan operations. A scan operation returns all key-value pairs stored in the tree, where the keys are within a specified range. Summary of the Invention

[0005] If a reader attempts to read a set of items from a portion of the tree while a writer is writing to the same portion of the tree, there could be potential problems with concurrent readers and writers (for the purposes of this article, "concurrency" simply means time overlap).

[0006] For example, consider a scenario where a reader is performing a range scan while data is being written within the scan range. Imagine performing two writes to a given leaf: a new or modified item with key 30, and a new or modified item with key 90 (in reality, the key could be much larger, but this is just for illustration). The writes are performed in this order and take a finite amount of time. Now imagine the reader is also scanning items with keys 1-100 at the same time. Performing the scan in this order also takes a finite amount of time. Due to the direction of the scan (from 1 to 100 upwards), the reader might pick up the write to the later item with key 90 instead of the write to the earlier item with key 30, even though the write to the item with key 30 might complete earlier. For example, suppose both writes to items begin after the range scan starts, but the write to the item with key 30 doesn't complete until the scan passes key 30, while the write to the item with key 90 is completed when the scan reaches key 90 (also note that different write operations don't necessarily take the same amount of time to complete, for example, due to different CPU resource availability). In this scenario, the scan will capture new write values ​​for the item with key 90, but will not capture new write values ​​for the item with key 30, even though the write to the item with key 90 started later. Alternatively, as another example, suppose the write to the item with key 30 begins before the scan starts, but doesn't complete until after the scan passes key 30. If the scan simply ignores items being written to (e.g., because they are locked by the writer), then the scan can capture writes to items with key 90 but not key 30.

[0007] It would be desirable if the snapshots of a portion of the tree captured by the reader always matched the order in which the items were written into the tree. Similar issues can arise in other forms of data structures used to store key-value pairs, not just tree structures.

[0008] The requirement for consistency is sometimes expressed as "linearity." A set of operations can be said to be linear if the effects of each operation occur as if each operation were performed instantaneously in time (rather than spanning a finite amount of time). This instant must be between the time the operation is issued and the time its result is received.

[0009] A simple solution is to lock the entire tree for each operation. This is detrimental to performance, but guarantees strong consistency.

[0010] More complex systems may employ Multi-Version Concurrency Control (MVCC). In a traditional MVCC system, this consistency is achieved through versioning and having each writer place a lock on the memory location of the item being written to. The tree has a single global version number, and each item is assigned a separate version number. When a write operation wants to write to an item, it places a lock on the memory location of the item to be written to by setting its separate version number to an invalid value. The write operation then writes to the item in question, then obtains the read version number by reading the global version number, and then increments it. It then releases the lock by setting the item's version number to the global version number previously read in the same write operation (from before the increment). At the start of a read operation (e.g., a range scan), the reader reads the global version number and then begins searching for items with keys within a specified range. In doing so, it will read any unlocked items whose version number is less than or equal to the global version number it read at the start of the read operation. If the read operation encounters a locked item within the scan range, it must wait for the lock on that item to be released before it can complete. Whether an item is visible depends on its version number. A write operation acquires its write version at the end of its execution, so even if it starts before a read, it may acquire its write version after the reader acquires its read version. If a write operation within the scan range acquires its write version before a read operation begins, the item will then be read by the read operation because its version number is lower than the version number the reader read at the start of the read operation. If both writes begin but are not completed at the start of the range scan, the reader must wait for both locks to complete the range scan. On the other hand, if a write to an item with key 30 begins but is not completed before the range scan begins, and a write to an item with key 90 begins after the range scan begins, the scan will capture the write to the item with key 30 but not 90 because the version number of the item with key 90 is higher than the version number the reader read at the start of the scan operation (although the scan still must wait for both locks to be released to detect this). This is because the write increments its version at the end of its execution. If both writes to key 30 begin after the range scan has started but are still within the scan period, the scan will still see the update to key 30 but not the update to key 90. This is because the write to key 30 will be set to the same version captured by the scan operation (although the scan still has to wait for both locks to be released to detect this). Therefore, the scan may not capture all writes, but there is no scenario where it would capture the later write to key 90 but not the earlier write to key 30.

[0011] However, this traditional approach requires the reader to wait for the lock to be released before completing the read. Ideally, a versioning scheme should be provided to address the consistency issue without requiring the reader to wait for the lock.

[0012] According to one aspect disclosed herein, a system is provided comprising: a memory, a writer, and a reader. The memory stores a data structure comprising multiple items, each item comprising key-value pairs. The writer is configured to perform multiple write operations, each write operation writing a corresponding item, or adding a new item to the data structure, or modifying an existing item in the data structure. The reader is configured to perform group read operations to read any item from the data structure having keys within a specified range. The writer is configured to maintain a global write version, and the reader is configured to maintain a global read version. The writer is configured to perform each write operation by: i) reading and subsequently incrementing the global write version, ii) writing the corresponding item to the data structure, including versioning the corresponding item using a corresponding version number equal to the global write version read by the corresponding write operation before the increment, and iii) generating a release indicator indicating the corresponding version number. The writer is also configured to send release signals to the reader corresponding to at least some release indicators, each release signal signaling a corresponding version number, but not to send a release signal corresponding to any release indicator indicating a higher corresponding version number until release indicators indicating all lower version numbers are generated. The reader is configured to update the global read version to equal the corresponding version number indicated in the release signal upon receiving each release signal from the writer. The reader is configured to perform group read operations by reading the global read version at the start of the group read operation, and then selectively reading items from a specified range of keys in the data structure based on version numbers equal to or less than the global read version read at the start of the group read operation, skipping (i.e., ignoring) items in a specified range with higher version numbers.

[0013] Therefore, each individual item is assigned a local write version number, while the entire data structure is associated with a global write version number and a separate global read version number (unlike the single global version number in traditional schemes). When a write operation is to be performed, the writer reads and increments the global write version (ideally atomically), and uses the value read (from before the increment) as its write version. The write operation uses its write version to version the items it creates. Before completion, the write operation must release its corresponding update. Writes are released version-ordered: when a write operation waiting to complete becomes one with the lowest unreleased (uncommitted) write version, it updates the global read version with its own write version. Each read operation observes the global read version at the start of the read operation and uses it as its own read version. The reader only reads items whose version is less than or equal to its read version, ignoring the rest.

[0014] Consider the example of a range scan of keys 1-100 and two writes to keys 30 and 90, the former starting before the latter. The reader can start in one of three ways: I) before keys 30 and 90 are released, or II) before key 30 is released but before key 90 is released, or III) after both keys 30 and 90 are released. In case I), the reader will ignore keys 30 and 90 even if they are entered simultaneously, because their versions are greater than their read versions. In case II), the reader must see key 30, because key 30 is only released after it is fully inserted. If it observes key 90, it will ignore it, because key 90's version will be greater than the reader's read version. In case III), the reader will observe keys 30 and 90 simultaneously, because they must be entered before they can be released.

[0015] Because the read version can lag behind the latest write version, and because the reader is configured to ignore items with versions higher than its read version, the reader will not wait for the item the writer is currently updating. In the previous scheme, read locking was required because incrementing the global version number and releasing the write to the reader were the same thing—once a subsequent write operation incremented the global version number, the reader would immediately jump to attempting to read all items versioned with that latest version number. In the currently disclosed scheme, incrementing the write version and releasing the write to the reader are separate. By using two global version numbers, read operations only access items that have been committed, without having to wait for concurrent operations.

[0016] In this embodiment, a write operation can still place a write lock on the item it is writing to, preventing other write operations from attempting to write to the same item simultaneously. After resolving the write lock, the write is versioned. A write lock can be implemented by temporarily setting the item's version number to an invalid value (also referred to herein as an "unspecified" value), somewhat similar to existing techniques. However, in the currently disclosed scheme, the reader can safely ignore items with unspecified version numbers because it knows that the read version it is using is less than a single version number that will be versioned once the item is released.

[0017] In embodiments, the system can support multi-write atomic operations, i.e., performing two or more write operations within a single atomic operation of the processor. The currently disclosed scheme also ensures that, in this case, the write operations remain linearizable. Returning to the example above, if the two writes are merely concurrent, then they can be linearized in any order. If two writes are merely concurrent (but not atomic), then they can be linearized in any order. Just because the write to the item for key 30 starts before 90 doesn't mean it will be linearized before 90. If the scan sees 30 but not 90, then they can be linearized so that 30 comes first. If it sees 90 but not 30, then 90 can be linearized first and there will still be correct execution. However, if 30 and 90 are both part of the same multi-write atomic operation, then the reader needs to see both of them or neither. The above scheme also ensures this.

[0018] One disadvantage of using the currently disclosed scheme is that signaling between the writer and reader is required to increment the global read count in order to release items. This signaling overhead can be significant, especially if the signaling is via a bus such as a PCIe bus. Implementations can mitigate this by batching updates together for release. That is, instead of sending a release signal corresponding to each individual writer operation to increment the global read count by 1 each time, the writer can increment the global read count by sending a release signal only every few writes, in jumps of more than one at a time.

[0019] In other words, the writer can be configured not to send a release signal corresponding to each release index to the reader, but to send at least some release signals corresponding to release indices with non-sequential version numbers one after another, so as to batch the release of some items together.

[0020] Batch release of multiple writes reduces the signaling overhead of sending release signals to the reader. However, this is not necessary; in other embodiments, it may be sufficient to send a separate release signal for each write, thus incurring the signaling overhead; or in some systems, particularly if the writer and reader are closely connected, the signaling overhead of sending a single release signal may not be too bad.

[0021] An example protocol for batch release is as follows. This can be applied to situations where the writer includes multiple concurrent threads, and at least some write operations are performed by different threads. In such an embodiment, the system may include a circular buffer comprising N slots with indices i = 0…N-1. Each write operation places its release indicator in a slot i_r in a circular buffer with N slots, where i_r = v modulo N, and v is the corresponding version number. The writer is configured to, when a write operation places its corresponding release indicator in the circular buffer, check whether the slot with the immediately following index contains a release indicator indicating the next lowest version number, and…

[0022] If not, the corresponding release indicator is left in the circular buffer to allow the corresponding item to be released by another write operation, but

[0023] If so, a batch of release indicators is determined, which includes the corresponding release indicator and any release indicators occupying consecutive slots immediately preceding the corresponding release indicator, indicating the continuous operation of version numbers, and a single release signal is sent to the reader indicating the determined maximum version number in operation, causing the reader to update the global read version to that version number, thereby releasing the batch of items corresponding to that batch of release indicators.

[0024] One advantage of this particular protocol is that it leverages the nondeterminism of the writer to create batches, which is more flexible than simply batching n consecutive items together for each run. However, this is just one example; another option is indeed to batch n items together for a fixed run, or create a batch every t microseconds, or not use batching at all as mentioned above.

[0025] In embodiments, the data structure may include a tree structure, such as a B+ tree or other B-trees. In some such embodiments, each leaf node may include a first block and a second block, wherein items in the first block are stored in key-order sequence, while newly inserted items are added to the second block in chronological order (i.e., the order in which they were written). The first block may have a larger maximum size than the second block, so the first block may be referred to as the "large block" and the second block as the "small block". If a new write causes a small block to exceed its maximum size, the small block may be merged into the large block.

[0026] In this embodiment, the disclosed scheme for version control and release items may be applied only to items in the second block. Items in the first block may alternatively be associated with a single node version of the corresponding node.

[0027] However, this is not limiting, and in other implementations, the disclosed scheme can be applied to all items in a node, or other subdivisions of the node. Furthermore, the disclosed scheme is not limited to use in tree-based structures, but can also be applied to writing and reading items in other forms of data structures, such as table-based structures.

[0028] In this embodiment, the writer can be implemented in software, but the reader can be implemented in custom hardware, such as in a programmable gate array (PGA) or a field-programmable gate array (FPGA).

[0029] The synopsis is provided to introduce, in a simplified form, some concepts further described in the detailed description below. The synopsis is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. The claimed subject matter is also not limited to implementations that address any or all of the shortcomings pointed out herein. Attached Figure Description

[0030] To aid in understanding the embodiments disclosed herein and to illustrate how such embodiments can be implemented, reference is made to the following figures by way of example only, wherein:

[0031] Figure 1 This schematically illustrates a range scan of the form of an inclusive scan for an example key interval [14,22], where the interval formed by the keys of the returned items is the largest subset falling within the input interval;

[0032] Figure 2 A range scan in the form of a cover scan of the example key interval [14,22] is schematically shown, where the interval formed by the keys of the returned items is a minimal superset containing the input interval;

[0033] Figure 3 The range scan of the example key intervals (14, 22) is illustrated schematically in the form of a cover scan.

[0034] Figure 4 This is a schematic block diagram of a system according to embodiments disclosed herein;

[0035] Figure 5 The diagram illustrates a data structure including a tree structure.

[0036] Figure 6 This schematically illustrates nodes that include both small and large blocks;

[0037] Figure 7 This illustrates how to insert a new item into a small block;

[0038] Figure 8This illustration demonstrates a method for sorting small pieces by including sequence hints;

[0039] Figure 9 This illustrates a method for sorting small blocks based on sequence cues;

[0040] Figure 10 This is a schematic circuit diagram of a quaternary small piece sorting machine;

[0041] Figure 11 This illustrates how to merge large and small blocks during an insertion operation;

[0042] Figure 12 This schematically illustrates a method for splitting new content during insertion.

[0043] Figure 13 An example of a release ring with eight slots is illustrated schematically. Detailed Implementation

[0044] This disclosure provides a novel form of multi-version concurrency control (MVCC) that operates based on two global versions—a read version and a write version. This allows the writer to release all updates of an operation (atomic in the embodiment) without blocking the reader, allowing reads to be wait-free, rather than waiting for updates in the software.

[0045] An exemplary implementation uses a release ring data structure, see below. Figure 13 Let's discuss this in more detail. The exposed release ring data structure allows the writer to efficiently batch release operations to amortize the cost of communicating with the hardware. This data structure can be used in any scenario where sequential release operations are required and this involves expensive signaling operations.

[0046] The embodiment also provides a node layout in which each node consists of a first block and a second block. Preferably, the first block has a larger maximum size (in bytes) than the second block, so the first block and the second block can be referred to as the large block and the small block, respectively. The large block and the small block will be described below by way of example, but in principle, anywhere below this, "first block" and "second block" can be more generally used instead.

[0047] Large chunks are sorted, allowing for efficient node searching. In this case, sorting means ordering the items by key, such that the order in which the items are keyed is the same as the order in which they appear in the memory address space (depending on the implementation's virtual or physical address space). Small chunks, on the other hand, contain the most recent insertions and deletions of nodes and are arranged chronologically, allowing for fast updates. Another option is to keep the entire node unsorted, resulting in fast update performance but poor search performance; or to sort the entire node, resulting in fast search performance but poor update performance.

[0048] Optionally, keyboard shortcuts can be used in large blocks. This allows node searches to examine only a small portion of the entire node, resulting in better performance.

[0049] As an alternative or additional optimization, in embodiments, "back pointers" can be included in the smaller blocks. These allow ordering to be established between items in the smaller and larger blocks without comparing the keys. This results in efficient implementations (e.g., in hardware) because it eliminates the need to compare the entire key.

[0050] As another optional optimization, which can be used with or independently of keyboard shortcuts and / or back pointers, in an embodiment, "sequence hints" can be included within a chunk. Sequence hints within a chunk allow highly efficient hardware to establish the sorting order of items within the chunk without comparing keys.

[0051] Another optional optimization is to provide synchronization for complex updates. In this case, when a node is split or merged, the writer updates the tree by creating a copy of the subtree (e.g., a thread-private copy) and then swapping it with a single pointer update in the page table.

[0052] The disclosed techniques are particularly (but not exclusively) applicable to systems where the writer is implemented in software running on one or more processors, but the reader can be implemented in dedicated hardware. The hardware may include FPGAs or PGAs, or even dedicated (i.e., fixed-function) circuitry such as ASICs (Application-Specific Integrated Circuits).

[0053] Dedicated hardware for implementing indexed data structures is hardware with dedicated pipelines to perform data structure operations. This contrasts with a general-purpose CPU that implements an instruction set and executes programs that use that instruction set to perform data structure operations. Dedicated hardware has several advantages: it offers higher throughput per watt, greater predictability and lower latency, and can be cheaper to run the same functionality on a CPU as on software. The main disadvantages of using dedicated hardware are that it is more difficult to design, takes longer to build than software, and, in the case of dedicated hardware, cannot be changed after it is built, thus it is typically used only for functions with very wide deployment.

[0054] The embodiments disclosed herein provide a memory B-tree that benefits from the best of both worlds: all operations can be performed in software on the CPU, while lookup and scan operations can be performed on dedicated hardware we have designed. In many workloads, lookup and scan are the primary operations, meaning that most operations benefit from hardware offloading. Lookup and scan are simpler than update operations, meaning the entire system can be built and deployed much faster. In one example implementation, the memory could be DRAM-based host memory, but this is not limiting; in other examples, the B-tree could be stored in, for example, a storage NVM device, such as an SSD or a 3D-Xpoint drive.

[0055] One consideration when designing such hardware-software data structures is ensuring that the benefits of a pure hardware design carry over to the hybrid data structure. For example, one advantage of running operations in dedicated hardware is low and predictable latency. If hardware operations need to wait for software operations, their latency can be significantly impacted, thus losing most of the benefits of hardware specialization. Furthermore, ensuring consistency between software and hardware is challenging, and some implementations may decide to implement a weak consistency model for simplicity, but weaker consistency models can be more difficult to use. The techniques disclosed in this paper can be applied to solve one or more of these problems and / or other issues.

[0056] The embodiments disclosed herein provide a strongly consistent in-memory B-tree (which guarantees linearity), supports multi-bond atomic updates, and allows lookups and scans in hardware without waiting for software or retries. B-trees are widely used, therefore this idea has very broad applicability and can be guaranteed to be built on specialized hardware.

[0057] However, it should be noted that the scope of this disclosure is not limited to implementing writing in software and reading in hardware. In other possibilities, the writer may be implemented in some form of hardware such as an FPGA or ASIC, and / or the reader may be implemented in software.

[0058] Examples of the above techniques will be described in more detail in the reference figures.

[0059] The following description will be in the form of a B-tree, but more generally, the ideas disclosed in this paper, such as multi-version concurrency control, different types of read and write operations, the use of large and small blocks, shortcuts, back pointers, sequential hints, and / or synchronization of complex updates (splits and merges), can be applied to any tree structure or other form of data structure for storing key-value pairs. Tree structure

[0060] Key-value stores like B-trees store data as key-value pairs (also called items). The size of an item may be variable, but depending on the implementation, the data structure may limit the total allowed size of items, for example, 512 bytes, or in some implementations they may be fixed-size to simplify implementation. Each key is associated with a single corresponding value; multiple key-value pairs with the same key are not allowed.

[0061] Figure 1 , Figure 4 and Figure 5 Elements of a tree structure 103, such as a B-tree, are illustrated as an example. Tree 103 is a form of data structure implemented in memory, which can be used as a key-value store. Tree 103 includes multiple nodes 102, some of which are leaf nodes 102L and others are internal nodes 102I. Each node 102 has a corresponding node ID, which may also be referred to herein as a logical ID (LID). Each leaf node 102L includes a corresponding set of one or more items 101, each item 101 including key-value pairs (for example, ...). Figure 1 The diagram shows node 102 undergoing a range scan, but this is not a limitation and is merely an example of a read operation type. Key-value pairs consist of a key that acts as an index to the item, mapping to a value that is the item's content. The number shown within item 101 in the diagram is an example key (in practice, keys could run to much larger numbers, but this is for illustration only).

[0062] Each internal node 102I specifies a corresponding set of one or more child nodes of the internal node. Once the tree is built, at least one or more internal nodes will have multiple child nodes (although when there is only one leaf in the tree, such as when it is first created, the root node has only one child node). Each internal node 102I also indicates the range of keys contained in its corresponding child nodes. Figure 5 The specific scheme shown, where keys are sandwiched between pointers, will be discussed later; it is merely one possible example of this mapping.

[0063] By implementing the mapping in any way, the node IDs in internal node 102I thus define the edges between parent and child nodes, forming a tree structure. Each internal node 102I is the parent node of at least one corresponding child node. Once the tree is built, at least one or more internal nodes are each the parent node of multiple corresponding child nodes (although, as mentioned above, when there is only one leaf in the tree, the root will have a single child node). A child node of a given parent node can be either a leaf node 102L or another internal node 102I. A leaf node 102L is the only child node, not the parent node (i.e., a leaf is the bottom of the tree, or in other words, the end of a branch in the tree).

[0064] For a key to be "contained" by a child node, this means either that the child node is a leaf node 102 that contains an item (key-value pair) with that key; or that the child node is another internal node 102I that, through its node ID to key mapping, ultimately causes the leaf node 102I containing the item with that key to move down one or more levels (or "generations") down the tree hierarchy.

[0065] One of the internal nodes 102I at the top of the tree is the root node. If a writer or reader wants to write to or read an item with a specific key, it first queries the mapping from the node ID to the key range specified in the root node to find which child node of the root contains the desired key. If that child node is itself another internal node 102I, the writer or reader then queries the mapping specified in that node to find which child node of the next generation contains the desired key, and so on, until a leaf node 102L containing an item with the desired key is found.

[0066] In this embodiment, key comparisons may follow the semantics of the C memcmp function, meaning that integer keys should be stored in big-endian format to preserve integer comparison semantics. However, this is only one possible example.

[0067] In an embodiment, the B-tree can take the form of a B+ tree, where only leaf nodes 102L contain key-value pairs, and internal nodes 102I contain only the mapping from node ID to key range. Read / write operations

[0068] Tree structures such as B-trees can support read operation types: single lookup and / or range scan. They support write (update) operation types: insert, modify, and / or delete. Example semantics of such operations are described below.

[0069] Search: This takes a key as an argument and returns the value associated with that key if it exists in the tree. Otherwise, it returns a status "Not Found".

[0070] Scan: This takes two keys as input parameters and returns all key-value pairs from the tree, where the keys lie in the closed interval [low key, high key]. In this embodiment, two types of scans are supported: containment scan and overlay scan. Figure 1 , 2 3 illustrates the difference. (l, u) indicates that the lower bound l and the upper bound u are not included in the result, even if they are in the tree. [l, u] indicates that they are included if they are in the tree. Combinations [l, u) where l is included but u is not, and (l, u] where l is not included but u is also possible.

[0071] Use include scan ( Figure 1The interval formed by the keys of the returned items is the largest subset falling within the input interval. When serving database queries (such as "get all items with keys between 36 and 80"), a containment scan can be used. The results do not include items with key 35, even though there are no items with key 36. A covering scan can be used instead. Figure 2 and Figure 3 The interval formed by the keys of the returned items is a minimal superset containing the input intervals. Figure 2 In the case of returning the item with key 22, but... Figure 3 No. Covering scans are useful when querying storage metadata, such as "retriev the disk blocks containing data with storage offsets ranging from 550 to 800 from the beginning of the file". The results include the highest key before 550 and the lowest key after 800 (i.e., only the keys on the wither side of the specified range), meaning that all disk blocks containing data within the requested storage range are returned.

[0072] Insertion: This requires inserting a new key-value pair. If an item with the specified key does not exist in the tree, the key-value pair is inserted into the tree. Otherwise, the operation does not update the tree and returns the status "Already exists".

[0073] Modification: This requires updating existing key-value pairs. If an item with the specified key exists in the tree, the value will be updated to the value of the input parameter. Note that the size of this value can be different from before. Otherwise, the operation will not update the tree and will return a status "Not Found".

[0074] Deletion: This requires a key for the item to be deleted. If the item exists in the tree, it is deleted. Otherwise, the operation returns a status "Not Found". System Architecture

[0075] Figure 4 An example system according to an embodiment is shown. The system includes a writer 401, a reader 402, system memory 403, and optionally, local memory 404 for the reader 402. A tree structure 103 is stored in system memory 403. The writer 401 and reader 402 are operatively coupled to system memory 403. For example, the reader 402 is operatively coupled to the writer 401 via a bus 405, such as a PCIe bus; while the writer 401 is operatively coupled to system memory 403 via a separate, local, or dedicated connection 406 that does not require communication via a bus. The writer 401 may be arranged to access system memory 403 via local connection 406 to write to tree 103, while for the reader 402 to read from the tree, it may have to access memory 403 via bus 405. The reader 402 may be connected to its local memory 404 via its own local connection 407 and may be arranged to cache a portion of tree 103 in the reader's local memory 404.

[0076] Memory 403 can take any suitable form and may include one or more memory media embodied in one or more memory cells. For example, memory media may include silicon-based electronic media such as SRAM (Static Run Time Memory) or DRAM (Dynamic Run Time Memory), EEPROM (Electrically Erasable and Programmable ROM), flash memory, etc.; or magnetic media such as disks or magnetic tapes; or even more exotic forms such as rewritable optical media or synthetic biological memory; or any combination of these and / or other types. Memory cells embodying memory 403 may include on-chip memory cells located on the same chip as writer 401, or separate cells located on the same board as writer 401, or external cells such as SSDs (Solid State Drives) or hard disk drives, etc., which may be implemented in the same housing or rack as writer 401 or even in different locations in separate housings or racks; or any combination of these. In one particular embodiment, as shown, memory 403 includes DRAM that may be implemented on the same board or chip as writer 401. For example, in one particular example, the system uses a DRAM board inserted into the board where the CPU is located. However, this is not limiting.

[0077] The reader local memory 404, if used, may take any suitable form, including any suitable medium in any one or more cells. In a particular embodiment, as shown, the reader local memory 404 includes DRAM, which may be integrated into the reader 402 or implemented on the same chip or board as the reader 402.

[0078] In this embodiment, the writer 401 is implemented in software, which is stored in the system’s non-transitory computer-readable memory and configured to run on one or more processors of the system, such as a CPU; while the reader 402 is implemented in hardware, which may take the form of an FPGA, a PGA, or even a dedicated (fixed-function) hardware circuit, or a combination thereof.

[0079] The memory (and / or reader, in alternative embodiments implemented in software) used to store the software of the writer 401 may take any suitable form, such as any of those described above, or alternatively, a read-only memory (ROM), such as a silicon-based electronic ROM or an optical medium, such as a CD-ROM, etc. The one or more processors may include, for example, a general-purpose CPU (central processing unit); or a dedicated or accelerator processor, such as a GPU (graphics processing unit), a DSP (digital signal processor), a cryptographic processor, or an AI accelerator processor, etc.; or any combination of these and / or other kinds.

[0080] In one particular embodiment, as shown, the writer 401 is implemented in software running on the CPU and the reader 402 is implemented in the FPGA.

[0081] B-trees store data in system memory 403. In embodiments, this allows for storing trees as large as the system memory (typically hundreds of GB), but at the cost of a reader 402 (e.g., an FPGA) accessing the system memory via PCIe 405. The reader (e.g., an FPGA) 402 can cache the upper levels of the tree in its local memory (e.g., DRAM) 404 to speed up execution. In one example implementation, the reader 402 records the LID of the root node and the number of tree levels in its local register file.

[0082] B-trees can also be designed to allow data to be stored on secondary storage, enabling the storage of even larger trees. The emerging very low-latency NVMe (Non-Volatile Memory Fast) devices make this an attractive option, even for low-latency scenarios.

[0083] In this embodiment, FPGA 402 performs only lookup and scan operations, while update operations are performed by CPU 401. Synchronization between the CPU and FPGA is preferably lightweight. Supporting only read operations on the FPGA allows read-centric use cases to benefit from hardware offloading, eliminating the need to implement all operations in hardware. Data Structures

[0084] In this embodiment, B-tree 103 is a B+ tree, which means that key-value pairs 101 are stored only in leaves 102L, while internal nodes 102I store key indicators and pointers to child nodes.

[0085] Figure 5 An example of a two-level tree is shown, but the principles described here can be applied to trees with any number of levels (i.e., any number of parent and child levels).

[0086] The data structure including tree 103 also includes page table 501. Page table 501 maps node IDs (also called logical IDs, LIDs) to the actual memory addresses of each node 102 stored in system memory 403. In this way, writer 401 or reader 402 can look up the address based on the node ID in page table 501 and locate node 102 in memory 403 based on its node ID (LID).

[0087] As described above, each internal node 102I includes information indicating how each of its child node IDs is mapped to the key range contained in the respective child node. Figure 5 Here is an example illustrating how to achieve this.

[0088] In this example, each internal node 102I includes a set of node pointers 502 and key indicators 503. Each node pointer 502 is a pointer to a child node specified according to a node ID (logical ID, LID). Each key indicator 502 in this example specifies one of the keys at the boundary of the range contained in the child node. In each internal node 102I, a pointer 502 with a key l to the left of the pointer and a key u to the right of the pointer points to a subtree of keys in the storage range [l, u). Furthermore, each internal node 102I stores pointers 502 to subtrees whose keys are lower than the lowest key in the internal node. This leftmost pointer is stored in the node's header 603 (see, for example, [link to relevant documentation]). Figure 6 In other words, each key indicator is sandwiched between a pair of pointers, and the key indicator specifies the boundary between the ranges of the child nodes pointed to by the pointers on either side of the key indicator. In other words, the inner nodes store the leftmost child node. They essentially store X keys and X+1 pointers. The leftmost node contains X+1 pointers. Figure 5 In the specific example shown, a new item with the key [22, 65] will be inserted in the middle leaf; an item with the key less than 22 will be inserted in the leftmost leaf; and an item with the key greater than or equal to 65 will be inserted in the rightmost leaf.

[0089] Understandable, Figure 5 The scheme shown for mapping children to key ranges is just one example. Another example is, for instance, storing all boundary keys in a node first, and then using all LIDs to follow them.

[0090] In some embodiments, each leaf 102L may also store pointers to the leaves to its left and to the leaves to its right to simplify the implementation of range scanning.

[0091] B-tree 103 grows from the bottom (like any other B-tree). If, during insertion, a new item 101 cannot fit into the leaf 102 it maps to, the leaf is split in two. Items in the old leaf are split in half, and pointers to the new leaves are inserted into the parent node. If the parent node becomes full, it is also split. Splitting can continue recursively to the root of the tree. If the root is split, a new root is created, increasing the height of the tree. When a leaf becomes empty because all items have been deleted, it is removed from its parent node. Alternatively, another option is to merge leaves when they are less than half full to keep the B-tree's space constant. Node layout

[0092] In an embodiment, the size of B-tree nodes 102 may be 8KB to align them with DRAM page sizes. They may be allocated in fixed memory that the operating system cannot move so that readers 402 (e.g., FPGAs) can access nodes 102 by using their physical memory addresses; however, this is not required.

[0093] Internal nodes 102I preferably do not directly store the addresses of their child nodes. Instead, they store a logical identifier (LID) for each node. In one example, the LID size could be 6 bytes, limiting the maximum tree size to 2^61 bytes. The mapping from a node's LID to its virtual and / or physical address is maintained in page table 501. Page table 501 can be stored either in system memory 403 or in FPGA-attached memory 404. When a new node mapping is created or when a node mapping is changed, a writer (e.g., a CPU) updates table 501 in system memory 403. If a cache is maintained on the reader side, writer 401 also issues commands (e.g., via PCIe) to reader 402 (e.g., an FPGA) to update a copy of the table in reader-attached memory 404. Alternatively, the reader can poll the tree for updates.

[0094] Using LID to address nodes provides a degree of indirection, such as allowing nodes to be stored on NVMe devices. It also helps to synchronize access to the tree, because in some cases, update operations may perform copy-on-write operations on nodes.

[0095] In this embodiment, the layout of internal node 102I and leaf node 102L is identical. See, for example, [link to example]. Figure 6 The node header 603 can be stored, for example, in the first 32 bytes of node 102. Header 603 can contain: fields specifying the type of node 102 (internal or leaf), the number of bytes used in the node, lock bits used to synchronize access to the node, and the node version number, and / or several other fields discussed below. For example... Figure 5 In the example shown, the header 603 of the inner node 102I also stores the LID of the leftmost child node. The header of the leaf node 102L can store the LIDs of the left and right siblings. Keys and values ​​can be of variable size, so they can be stored as blobs. Each blob has a 2-byte header specifying its size. LIDs can be stored without a header because their size is known. Values ​​can be stored inline in the B-tree node 102L to improve sequential access performance. Values ​​larger than 512 bytes can be stored outside the B-tree 103, and a pointer to that value is stored in the leaf 102L.

[0096] Without considering the choice of specific implementation parameters such as node size in the above example, in this embodiment, each leaf node 102L may include two blocks—a large block 601 and a small block 602. To achieve this, pointers to the boundaries between the blocks can be stored in the node's header 603. For example, the pointers could be represented as offsets relative to the start of the large block 601. The large block 601 stores items in sorted order, and the small block 602 stores the most recently updated log. When the small block 602 becomes larger than a threshold (e.g., set to 512 bytes), it is merged with the large block 601. By dividing the node 102L into large and small blocks, read operations benefit from accessing most of the sorted data without the overhead of sorting the node on each update.

[0097] The advantage of a two-block layout is that it balances the workload between the writer and the reader. If all entries are simply added in chronological order, not by key, this will be very fast for the writer. However, the reader must sort the items on read to perform a single read or range scan (at least a range scan requires sorting – a single read can be performed using sorting in the same way as a lookup and a range scan, which is simpler). On the other hand, if the writer places all new entries in sorted order each time it performs a write, this will make the reader very fast, but will place a greater burden on the writer on write. The two-block approach provides a trade-off between these two approaches.

[0098] The entries in small block 602 can be newly inserted items or deletion markers. As a further optional optimization, each entry in small block 602 can further store a pointer (e.g., 2 bytes long) to an item in large block 601. For convenience, this can be referred to as a "return pointer" (although the term "return" does not necessarily imply restriction to any particular direction or similar direction in the physical address space). In an example implementation, for a newly inserted item 101, the pointer points to the first item in large block 601 with a key greater than the key of the new item added to small block 602. Or, for a deletion marker, the pointer points to a deleted item. The pointer can be represented as an offset within node 102. Reader 402 can use it to establish the order of items 101 in large and small blocks without comparing their keys, a useful optimization, especially if the reading is implemented in hardware.

[0099] As another optional optimization, each item in block 602 can include a field (e.g., 1 byte long) that stores its index in the sorted block when the item is inserted. That is, it is the order in which the items were inserted. This can be referred to as a "sequence hint". These indexes are "replayed" by reader 402 during the search as it scans the block to reconstruct the currently sorted order. This order is stored in a temporary "indirect" array in reader 402, such as in an FPGA register, for accessing the items in the sorted order. This makes sorting more efficient, especially if implemented in hardware, and does not introduce significant latency.

[0100] Figure 7 An example of the steps for inserting a new item 101 into block 602 is shown. Step a) shows node 102 before insertion. In step b), item 101 is copied into block 602. Reader 402 is not yet visible. In step c), the size of the data indicated in node header 603 is changed to include the new item.

[0101] As another optional optimization, to optimize the search for keys within a node, large block 601 can be divided into segments of approximately equal size. Keys at segment boundaries, along with pointers to those boundaries, are stored in shortcut block 604, which can be stored immediately following the node header 603. The search for keys begins by scanning the shortcut blocks and identifying segments that may contain keys. The search only checks those segments and small blocks, reducing the amount of data read from memory. This optimization can significantly improve performance, for example, if the performance bottleneck is the PCIe bandwidth between the reader (e.g., FPGA) 402 and system memory 403. For keys of 32 bytes and smaller, the search reads approximately 1.5KB of data from an 8KB node, resulting in a 5-fold performance improvement compared to a more straightforward implementation that scans the entire node. Shortcut keys can be selected during the merging of large and small blocks 601, 602. Node caching

[0102] FPGA 402 may have several gigabytes of DRAM 404 directly connected to it (e.g., 4GB). This memory may not be sufficient to store the entire B-tree 103 in all use cases, but it may be large enough for the first few levels, so in this embodiment it can be used as a cache to reduce latency and increase B-tree throughput. For example, cache 404 may be allocated at, for example, 8KB node granularity to reduce metadata volume, but data may be fetched in 256-byte blocks to reduce the amount of unwanted data read via PCIe 405. FPGA 402 may maintain a 32-bit occupancy bitmap for each cache node 102 to track blocks in cache 404. Page consistency between cache 404 and system memory 403 can be maintained by invalidating cache pages whenever the page mapping is updated. When the page mapping is updated, software on CPU 401 can send a page table update command via PCIe 405, and FPGA 402 can invalidate the page by clearing the occupancy bitmap. Page Table

[0103] Reader 402 (e.g., FPGA) can maintain a local copy of page table 501 stored in its local memory 404 (e.g., DRAM), which can be attached to FPGA device 402. In an embodiment, each entry in the table is 8 bytes in the FPGA memory but 16 bytes in the system DRAM. This is because, in an embodiment, the copy in the system DRAM stores two addresses (virtual and physical), while the copy in the FPGA DRAM memory stores one address (physical only).

[0104] The entries in page table 501 store the memory addresses mapped to the nodes 102 corresponding to their respective LIDs. Typically, these addresses can be physical or virtual addresses, depending on the implementation. In an embodiment, the writer employs virtual memory mapping, and the page table 501 on the writer side stores the virtual addresses mapped to the nodes with their respective IDs. In an embodiment where a copy of the page table is stored in FPGA 402, the entries in the FPGA page table copy can store the physical addresses mapped to the nodes with their respective LIDs. For example, with 8-byte entries and 4GB of DRAM, the system supports trees up to 4TB, which is large enough for the main memory trees on today's servers. Larger trees can be supported by enabling the reader 402 (e.g., the FPGA) to access page table 501 in system memory 403 or an NVMe device, or by increasing the B-tree node size.

[0105] Page table 501 is not absolutely necessary in all possible embodiments; a less preferred alternative is to directly use memory addresses as node pointers in internal node 102I. Synchronization (Multi-Version Concurrency Control, MVCC)

[0106] According to this disclosure, lookup and scan operations are wait-free—an update operation of writer 401 (e.g., executed on the CPU) does not prevent lookups and scans of reader 402 (e.g., executed on the FPGA) or cause them to abort and retry. In other words, read operations are wait-free, meaning they can be completed without waiting for other concurrent operations. For example, in an implementation where the writer is implemented in software and the reader in custom hardware, this means that lookups and scans on the combined software-hardware tree have the same low and predictable latency as a pure hardware implementation.

[0107] To achieve this, a form of multi-version concurrency control is used to ensure that read operations always have access to a consistent snapshot of data without waiting for ongoing update operations. Two shared version numbers are maintained: a global write version and a global read version. For example, each version number might be 64 bits long. For a 64-bit version number, this is large enough that it will never wrap around in practice. This might happen if the version bit width is smaller, depending on how often the system is used, but additional steps can be taken to handle wrapping.

[0108] Each write operation reads and increments the global write version, using the read value as its write version. Reads and increments are best performed atomically. Write operations version the item 101 they create using their write version. Write operations release updates in version order: each writer updates the global read version when it becomes the writer with the smallest write version. The writer also sends the write version to the reader 402 (e.g., the FPGA) when releasing an update. Each read operation observes the global read version at startup and uses it as its read version. Read operations (e.g., range scans) only read items whose version is less than or equal to the read version adopted by the operation, ignoring the rest. By using two global version numbers, read operations always access items that have been committed, without waiting for concurrent operations. This means reads are wait-free. At any point in time, items earlier than (version less than or equal to) the global read version are committed and accessible to new read operations. Nodes and items newer than the global read version are still in the commit process and are not accessed by the reader. Update operations can be synchronized using node write locks. These locks are ignored by read operations.

[0109] Releasing items after a sorted update adds cost to the update operation. A traditional alternative uses a single global version number, which avoids this cost. In the traditional approach, the global version is used to acquire the write and read versions of an operation. Update operations atomically read and increment the global version, using the read value as their write version. After acquiring the write version, the write operation assigns it to all items it is updating. Once the version is assigned and unlocked, the item is released (each newly inserted item initially has an undefined version, a special value, such as 0). Read operations read the global version at the start and use it as their read version. In this scheme, read operations are not guaranteed that items without assigned versions do not belong to their snapshot—a writer whose write version equals the read version may still be iterating over those items and assigning versions. Therefore, if a reader accesses an item without assigned versions, it must wait until a version is assigned. This is done by having the reader wait for a lock acquired by the writer. This means that reads are not wait-free. For example, if in Figure 4 The system implementation shown implies that operations running on the FPGA sometimes wait for operations running on the CPU, thus losing some of the benefits of building custom hardware. However, the currently disclosed scheme uses wait-free reads at the cost of some performance penalty for the writer. As an optional optimization, sorting overhead can be reduced through batch releases, for example, by using a release ring data structure that allows for efficient determination of the release order (see later references). Figure 13 (Discussion). The same data structure can also be used for batch processing of FPGA release timestamp updates.

[0110] Currently available version control schemes ensure linearity for writes and reads without requiring read wait locks. Linearity means that the effects of operations are performed as if each operation were executed in a single infinitesimal or atomic moment (even if each operation actually requires multiple cycles).

[0111] In this embodiment, the currently disclosed versioning scheme is used only for item 101 in small block 602. In this case, the versioning of items in small block 602 and large block 601 is different. All items 101 in the large block have the same version, which is stored in the header of node 102L. This can be referred to here as the node version. The node version is the write version of the update operation that creates the node buffer. Each node 102 (whether leaf or internal) can also store pointers to older versions of that tree node. A reader observing a node newer than its read version ignores that node and follows the old pointer chain. It follows the old pointer chain until it reaches a node with a version lower than or equal to its read version. The version of an item in a small block is stored along with the item. To reduce the size of items in a small block, the item version can be stored as a version increment relative to the node version (e.g., a 5-byte version increment instead of the entire 8-byte version). In this case, if an update results in an increment version wrapping, the small and large blocks of the node are merged. The reader ignores all items in the small block that are newer than their read version.

[0112] To ensure consistent snapshot observations across all operations, it's best to retain the deleted node 102 until no active operations are accessing the snapshot to which that node belongs. Merging or splitting nodes also involves deleting older nodes (see, for example, [link to previous section]). Figure 11 The combined leaf 102L, or Figure 12 (The internal nodes 102I and leaf nodes 102L in the tree are split, which will be discussed in more detail later). To garbage collect nodes that are no longer accessible, the system can use, for example, an epoch-based memory manager (similar to the RCU in the Linux kernel). In this case, each thread of writer 401 maintains an operation sequence number and exposes the current sequence number to the memory manager. Similarly, reader 402 (e.g., FPGA) assigns sequence numbers to each operation and exposes the sequence numbers of the latest and oldest active operations to the memory manager, for example, via PCIe. When a thread removes a node 102 from tree 103, it puts the node into the garbage collection mechanism list, along with the sequence numbers of all active threads and the sequence number of the latest active operation of reader 402 (e.g., FPGA). Threads periodically scan the garbage collection mechanism list and remove any nodes (e.g., hardware) that reader 402 cannot reach. If a thread's current sequence number is greater than the recorded sequence number, it can no longer reach that node. Similarly, if the oldest active operation of reader 402 is newer than the recorded FPGA sequence number, the reader cannot access that node. A node can be safely deleted when it is inaccessible to any writer or reader thread. Alternatively, node versions can be used to detect unreachable tree nodes.

[0113] Note: In the previous traditional approach, simply having the reader ignore locked items was not advisable. See the "Sidebars" section later for a discussion on this. Search

[0114] The lookup operation first reads the global read version and uses it as its read version. Starting from the root, it traverses each level of the internal tree node 102I to find a pointer to the next level node 102. Each lookup terminates at a leaf 102L. If the leaf contains the key, it returns the associated value. Otherwise, it returns a state indicating that the key is not stored in the tree.

[0115] In the embodiment where a separate node version is used for the first block 601, each node has a node version and a LID for the previous node version in the header. The node version acts as a form of timestamp. If the node version in the header is greater than the read version used by the lookup, the lookup reads the previous node version. It does the same check and follows the chain of previous nodes until it finds a node whose timestamp (i.e., node version number) is less than or equal to the read version of the lookup. See the previous discussion on following the chain of pointers to older nodes. All of this can happen even before the node content is viewed.

[0116] When accessing internal node 102I, the lookup initially retrieves the first portion of node 102 (e.g., the first 512 bytes), which contains the node header 603 and shortcut block 604. If a node version is used, and the node version is newer than the version being read, the lookup read address is stored in node 102 in a pointer to a previous version of that node (here referred to as the "old pointer"). The lookup follows the chain of old pointers until it reaches the node belonging to the snapshot it is reading. In an embodiment, the key of the node can be searched by examining all three blocks 601, 602, and 604 of the node. The lookup begins by searching shortcut block 604 to find the last key less than or equal to the target key. It follows the pointers of shortcut entries to find the segment of large block 601. The lookup retrieves the large segment from memory and searches it to find the last key less than or equal to the target key. Next (or in parallel), the lookup retrieves small block 602 from memory and searches it for the last key less than or equal to the target key. Entries in small block 602 with a version newer than the version read by the lookup are ignored. If back pointers are used, the lookup will follow the pointers stored in the larger item 101 found in the smaller and larger chunks, without comparing keys—if the back pointer of the smaller item points after the larger item, it follows the pointer in the smaller chunk; otherwise, it follows the pointer in the larger chunk. If the target key is less than the first key in the larger and smaller chunks, the lookup follows the leftmost pointer.

[0117] When the search reaches leaf 102I, it searches in a manner similar to internal nodes. The main difference is that it searches for exact matches in both small and large blocks, meaning that it does not require sorting of items within the large and small blocks.

[0118] Small-block search: The search for small blocks 602 can begin by sorting items 101 before examining them. By sorting, the search for small blocks can stop immediately upon encountering the first item with a key greater than the target key. In this embodiment, small-block sorting does not compare keys. Instead, it uses a sequence hint field stored in each small-block item to establish the sort. The sequence hint field stores the index of the item in the sorted small block as the item is inserted. The index is "replayed" while the small blocks are scanned to reconstruct the current sort order. The established order is stored in a temporary indirection array, such as in an FPGA register, without copying the items. The indirection array is used to access items in the sort order. Sorting does not introduce significant latency, especially in hardware, and can be performed in parallel while searching for shortcut block 604 and large block 601. In software, the sort order can be maintained in a small indirection array.

[0119] As mentioned earlier, in this embodiment, Xiaohei's version number can be stored as an increment relative to the version numbers of each node. However, this is an optional implementation detail.

[0120] Figure 8 This illustrates an example sequence for inserting node 102. The numbers shown in each item 101 represent its key. For clarity, elements in block 601 are omitted. The numbers above the items represent their order hints. In step a), block 602 in the node is initially empty. In step b), as an example, an item with key 90 is inserted and assigned order hint 0 because it is the first item in the block. Next, in step c), an item with key 60 is inserted. It is the smallest item in the block, so it is assigned order hint 0. The order hints in the existing items are not changed. The order hints are determined according to the sorting order of the blocks, which is established before inserting the new item. Next, in step d), an item with key 30 is inserted. It is again the smallest in the block, so its order hint is also 0. The last item in step e) has key 45, and its order hint is 1.

[0121] Figure 9 This illustrates the sorting of block 602. The diagram shows the state of the indirect array 901. The indirect array stores the offsets of items in the block, but in this diagram, each offset is represented by its key for illustrative purposes.

[0122] To sort block 602, items 101 are processed in the order they are stored, and their order hints are used to sort the items in the indirect array. Indirect array 901 stores the offsets of items within the block. When processing an item with order hint i, its offset within the block is incremented by its position i in the indirect array. All items at positions j ≥ i are shifted to the right. After the entire block has been processed, the indirect array contains the offsets of the items in the sorted order. Figure 9 Explanation Figure 8 The diagram shows the indirect array sorting of the small blocks. In one implementation, each step in the diagram is performed in a single loop on the FPGA, in parallel with the key comparisons. In step a), item 90 is inserted into the first slot. In step b), all items are shifted to the right. Item 60 is inserted into the first slot with order hint 0. In the next step d), all items are shifted to the right again to make room for item 30 with order hint 0. In the final step d), items 60 and 90 are shifted to the right, and item 45 with order hint 1 is inserted into the slot with index 1. The final state e) shows the state of the indirect array after steps a)-d).

[0123] Figure 10 An example implementation of an indirect array sorter is shown. It displays a 4-item wide array. The sorter includes 9-bit wide registers. The inputs to the registers are their outputs, the outputs of the registers to the left, or the input offset. The register input is selected by comparing the input index to the (constant) index of the register. If they are equal, the input offset is selected; if the input index is less than the register index, the output of the register to the left is selected; if the input index is greater, the register output is written to itself. This example implementation does not cover the case of deleting items in small blocks, but it can be adjusted to accommodate register inputs so that the register output is also brought to the right.

[0124] Of course, that's understandable. Figure 10 The layout shown is merely one possible hardware implementation. Once the disclosure of this document is given, those skilled in the art can use conventional design tools to design other implementations. Furthermore, a software implementation on the reader 401 is also possible.

[0125] The sorting implementation on the CPU 401 likely follows a similar approach. The indirect array is small and easily fits into the L1 cache, making operations fast. The main difference is that shifting items in the array to the right requires multiple cycles. Range scan

[0126] The range scan operation traverses tree 103 from the root, as described above, to find the start key of the target range. It then scans backward from the start key until it reaches the end key of the range or the end of the tree.

[0127] If using the node version, when reading a leaf, the scan will follow the old pointer as needed, just like a lookup.

[0128] In one embodiment, a range scan can use sibling pointers to move between leaves 102L. In such an embodiment, each leaf 102L has a pointer to its left and right siblings, allowing range scans to be performed in both directions. However, using sibling pointers is not mandatory. Without sibling pointers, the scan would return to the tree to find the next leaf. In most cases, it might only look at the first parent and get the LID of the next sibling. However, if it has reached the end of a parent, it would go to its parent to find its sibling. The scan might have to go all the way up to the root. In a further alternative embodiment, all leaves might have only one pointer to the right sibling, instead of the left. If only forward scans are needed, then only forward pointers are meaningful.

[0129] A range scan returns item 101 sorted by its key. To maintain the order of items, a scan of leaf 102L can process large and small blocks 601 and 602 together. In a cover scan, the scan first finds the item with the highest key, which is less than or equal to the start of the range in both the large and small blocks. A "standard" contain scan is similar, except it starts with the lowest key that is greater than or equal to the start of the range. In either case, the scan moves forward through the nodes in the large and small blocks, looking for the end of the interval and returning the item in the interval. The scan stops when an item with a key greater than the right boundary is found in both the large and small blocks. During a scan, if both the next small item and the next large item are within the interval, the search can use the return pointer in the small item to determine which to return next: if the small item points to the large item, the small item is returned and the search moves to the next small item; otherwise, the large item is returned and the search moves to the next large item. To handle shortcut key 604, the scan can follow the end of the current large segment. In this case, at the beginning of the next segment, the key is retrieved from the shortcut block and the value is retrieved from the large block. In the middle of the segment, the key and value are in the larger block. insert

[0130] The insertion operation begins with a tree traversal to find the node 120L where the new item 101 will be inserted. The traversal is similar to the traversal in the lookup process. The difference is that, in this embodiment, it reads all items regardless of their version to achieve the semantics of insertion and modification operations. Before updating a node, the writer 401 may lock it with a write lock to ensure that the node's state is the same as observed during the traversal (note that in this embodiment, the memory 403 may support different types of locks, write locks, and read locks; using a write lock does not necessarily mean using a read lock). For example, a write lock may be stored in a 32-bit lock word in the node header, where the lock word consists of a lock bit and a 31-bit node sequence number, incremented as the node is updated. The writer 401 can acquire a write lock by atomically setting the lock bit using atomic compare and swap instructions on the lock word, while keeping the sequence number the same as during the traversal. If the compare and swap succeed, it means the node has not changed since the traversal, so insertion can continue. If the compare and swap fail, the node has changed since the traversal, so the writer retryes the operation by performing the tree traversal again.

[0131] Fast path insertion: In a common scenario, insertion does not require splitting node 102L or merging large and small blocks 601 and 602. This common insertion can be performed in-place. Figure 7 The writer 401 atomically appends the new item by first copying it after block 602 ends, and then resizing the block to include the new item 101. To do this, the writer 401 may perform the following steps after acquiring a write lock: copying the item to the block after acquiring the lock; acquiring the write version by performing a get and add operation on the global write version (thus atomically updating the global write version as well); setting the version of the item in the block; updating the node size; incrementing the node's sequence number; unlocking the node; and releasing the operation. Optionally, the version number in the block is represented as an increment relative to the corresponding node version. In an embodiment, the lock word and the block size in the header may share the same 64-bit word, so the writer 401 may increment the node version, update the node size, and release the lock using a single instruction. Concurrent readers do not observe the new item being copied because it is stored outside the end of the block, as currently specified in header 603. They either observe nodes without the item or nodes where the item has been fully written. The reader ignores any items that have not yet been freed (i.e., before the global read version is greater than or equal to the item version), because their version is older than the item's version.

[0132] In this embodiment, the reader 402 (e.g., an FPGA) does not cache leaf node 102L. If it did, the writer 401 would have to issue a command via PCIe 405 to invalidate cache 404, which would introduce additional overhead for common-case insertions.

[0133] In one embodiment, the system also supports multiple write atomic operations, whereby two or more write operations (to two or more corresponding keys) are executed by the same single machine code instruction.

[0134] Large-Small Merging: If small block 602 becomes too large, writer 401 merges the large block and small blocks 601 and 602 (see...). Figure 11 It allocates a new memory buffer for node 102L and sorts all items 101 of the node into a large block in the new buffer. In this embodiment, new items are added to the merged large block (instead of a new small block). That is, the small block of the new node is empty after the merge. In principle, another approach could be taken (i.e., the new item is the first item added to the new small block), but storing the new item in the large block would immediately result in slightly fewer large-small merges over time.

[0135] In the embodiment using shortcut 604, writer 401 selects shortcuts when sorting items (it selects them at that point because large blocks are immutable). For each processed item, writer 401 decides whether to place it in a shortcut or a large block based on, for example, the number of bytes copied so far, the number of bytes remaining to be copied, and / or the average size of the keys and values. It ideally maximizes the number of shortcuts while maintaining a similar size for large block segments.

[0136] When all entries are copied to the new buffer, writer 401 sets the old pointer of the buffer to the address of the old buffer, and in this embodiment, also sets the version of the new buffer to the maximum value (e.g., a maximum 64-bit value). Writer 401 then atomically replaces the mapping of node LIDs in page table 501 with the address of the new buffer. To update the LID mapping, in this embodiment, writer 401 updates the LID entries in the page table CPU and FPGA copies. It acquires a software lock on the LID entry, issues a page table update command to the FPGA, and releases the software lock after the FPGA command completes. The parent node does not need to be changed because the node's LID remains unchanged. In this embodiment, the page table mapping is not versioned, meaning that concurrent read operations can read the new LID mapping. Those operations still use the old buffer by following the old pointer because their version is older than the node's version. The writer then acquires the write version and uses it to version the new buffer. It releases the update by signaling to the reader, for example, using a release ring described in more detail later. After release, the node becomes visible to new read operations. More specifically, the write operation increments the global write version and stores it in the header of the new node. A write becomes visible after the global read version is updated to a version greater than or equal to the node write version, and the write has therefore been released. In some cases, multiple versions can be released with a single update to the global read version, for example, by updating it from 5 to 10, instead of updating to 6 first, then 7, then 8, then 9, then 10.

[0137] Finally, the writer puts the old memory buffer into the garbage collection list, unlocks it, and sets a "deleted" flag at its head to ensure that concurrent writers do not update the old buffer. Ongoing operations can still read from the old buffer, but if an operation needs to update a deleted node, it will retry. During the retry, it will use the new read version and the new node mapping.

[0138] The garbage collection mechanism used can be any suitable scheme, such as standard explicit garbage collection mechanisms for concurrent systems, such as RCU, epoch-based memory managers, or dangerous pointers.

[0139] Figure 11 This shows an example of merging large and small blocks during an insertion operation. Inserting the new item into node 102L causes small block 602 to exceed its maximum size, so the small block will now be merged into large block 601 along with the new item. Step a) shows tree 103 before the insertion into the rightmost node 102L. Step b) shows the allocation and merging of the new buffer. The old pointer to the inserted node is shown (dashed line). In step c), the page table is updated to point to the new buffer, and the old buffer is put into garbage collection (GC) once any ongoing operations no longer require it.

[0140] One advantage of this approach is that complex updates, such as merges, can be performed without placing read locks on the old instance of the node at the original memory address (although some operations may still use write locks, which only prevent multiple writers but allow one writer and one or more readers to access the node). If reader 402 attempts to read node 102L while writer 401 is updating it in the new buffer but before the page table is updated, reader 402 simply reads the old instance from the existing memory address currently specified in page table 501. Once the write to the new buffer (i.e., the new memory location) is complete, writer 401 can then quickly update page table 501 to switch the corresponding page table entry for the node's LID to the new memory address. In this embodiment, this can be done in a single atomic operation. Any subsequent reads will read the new instance of the node from the new address based on the updated page table entry. A similar approach can be used for other complex updates, such as node splits or node merges.

[0141] When a new memory buffer is allocated for an LID, its sequence number can be safely set to 0 because it ensures the buffer is inaccessible to any operation before it is deleted and reused. For example, with some example sizes, if large and small blocks 601 and 602 are merged when small blocks are larger than 512 bytes and small block entries are at least 10 bytes (deleted entries are 10 bytes, inserted entries are at least 13 bytes), the maximum version of the buffer is 41, meaning the 31-bit sequence number will never wrap around. In fact, in one implementation, the minimum size is actually an average of 6 bytes, but even with 1-byte entries, the sequence number will not overflow. Node splitting

[0142] If there is not enough space in leaf 102L for insertion, writer 401 splits the leaf in two and inserts a pointer to the new item into the parent. Figure 12 If the parent node is full, writer 401 will also split the parent. Splits can propagate upwards to the root in this way. The updated but not split internal node 102I is called the split root. The writer replaces the entire subtree below the split root by updating the mapping of the split root in page table 501. Before updating any data, the writer can acquire write locks on all nodes 102 that the operation is updating (these are locks to prevent interference from other writers). If locking fails due to a node version mismatch, the writer releases all acquired write locks and restarts the operation. The writer also allocates all memory buffers needed to complete the operation. It allocates two new nodes (with a new LID and a new buffer) for each node it splits, and one memory buffer for the split root. If memory allocation fails due to a system memory overflow, the writer may give up and return an appropriate status code to the caller.

[0143] After acquiring any locks and allocating memory, the writer processes nodes from the leaves upwards, splitting them into newly allocated buffers. Each of the two nodes resulting from the split eventually receives approximately half the data from the original node. The writer merges large and small chunks during the split. When it reaches the root of the split, the writer copies it into a new memory buffer. It modifies the new buffer to include pointers to the new child nodes and keys at the boundaries between nodes. At this point, the new subtree is complete, but it is not yet visible to reader 402.

[0144] The write operation of writer 401 first versiones the new node as "unspecified". It then sets the old pointer, as previously described. Next, it updates the page table mapping to change the split root. Finally, it obtains the write version by updating the global write version and uses it to version all allocated nodes. The writer sets the old pointer of the split root to the address of the old buffer of the split root. This ensures that operations of the old version traverse the old subtree. It also sets the old pointers of the two new leaves to the addresses of the leaves being split. This ensures that scans using the sibling leaf pointers traverse the old leaves. To swap the two subtrees, the writer updates the mapping of the split root in the page table with the address of its new memory buffer. Swapping subtrees in this way ensures that the reader either observes the old subtree or the new subtree. The writer then releases the updates, for example using a release ring, and puts the memory buffers of all split nodes, their LIDs, and the old memory buffer of the split root into the garbage collection list. Finally, it unlocks all memory buffers and marks them as deleted.

[0145] The new root after the split will point to the old buffer as the old pointer. Additionally, the two new leaves will point to the leaves that split into the old pointers.

[0146] Figure 12 An example of splitting a node during insertion is shown. Dashed boxes represent newly allocated node buffers. Filled boxes represent new nodes with new LIDs and memory buffers. Step a) shows tree 103 before the split. The rightmost node is being split. The root of the split is the root of the tree. In step b), two new nodes are allocated for the split, and a new buffer is allocated for the root of the split. In step c), the mapping of the root of the split is updated, and the node is placed in the garbage collection (GC) list.

[0147] In this embodiment, the writer also updates the sibling leaf pointer used during the scan operation. It locks the sibling leaf pointer and updates it after the swap to the new subtree. Even though the sibling pointer and the split root may not be updated atomically, the reader observes a consistent state of the tree. All updates performed during the operation are invisible until the updates are released, meaning the reader either observes all updates performed during the split or none of them. This is a property of the MVCC scheme.

[0148] If the root of the tree cannot accommodate a new item, the writer splits it and creates a new root, increasing the tree's height. The writer performs the split as described above, except that it allocates a LID and memory buffer for the new root, not just a memory buffer for the split root. It initializes the new root with a leftmost pointer set to the left half of the old root and a single item pointing to the right half of the old root. It can then send commands to a reader (e.g., an FPGA) to update the root and tree height. In one example implementation, reader 402 records the root node's LID and the number of tree levels in its local register file, rather than in cache 404. Data copies in the cache, such as older root nodes, will eventually become invalid or may be evicted when their LIDs are reused on another physical node because the old data won't be accessed for a short time. Therefore, the cache only needs to focus on maintaining the consistency of the cached node data. delete

[0149] Deleting an item 101 is similar to inserting a new item. Writer 401 inserts the deleted entry into block 602. This entry points to the item it is deleting, so deleted items are ignored during lookup operations. During the merging of large and small blocks 601, 602, the space occupied by the deleted items is reclaimed. If node 102 becomes empty, it is deleted. Node deletion can be similar to node splitting from leaf 102L up the tree 103. In this embodiment, the same technique as described above is used to ensure its atomicity. Revise

[0150] Modification operations can be performed as a combination of insertion and deletion. The writer 401 appends the deletion entries of the old and new items to the block 602 and atomically releases them by updating the node's header 603. side branches

[0151] In previous traditional schemes, simply having the reader ignore and discard locked items was undesirable because it introduced problems with multiple update operations. For the sake of a running example, suppose an operation inserts 30 and 90 simultaneously (using the same write version). To support multiple update operations, the reader needs to wait to resolve whether an object belongs to its read snapshot (object_ver <= read_ver). If it doesn't wait, it might return a result containing only 30 or 90, which is non-linear. Furthermore, even without multi-key updates, not waiting can cause problems with the tree. In the currently disclosed tree implementation, leaves can be reached via two paths: from the root by following pointers in the internal nodes, and from other leaves by following sibling pointers. When a leaf L is split in two, this creates two new leaves L1 and L2 for the parent P, as well as a new buffer P1. The new leaves L1 and L2 are linked to the linked list formed by the leaves using sibling pointers. Once linked, L1 and L2 can be accessed from other leaves during a scan. L1 and L2 are also set to point to L as their old version to allow earlier read operations (and therefore with older read versions) to read the consistent state of the tree. The mapping from P to P1 is changed. After this, L1 and L2 can also be accessed from the root. P1 is set to point to P as its old version to allow older reads to read the consistent state. On the other hand, if a single version is used, as in the previous scheme, it is not possible to atomically make both paths of L1 and L2 visible without forcing the reader to wait. Problematic execution will occur when the writer splits as described above, but is delayed after releasing one of the paths. Note that in the previous scheme using a single global version number, the release must be done by setting the versions of L1, L2, and P1. The writer might simply insert a single item in the tree, such as 30. This will cause L to split. In a variant of the tree, the writer first sets the version of P1, and then sets the versions of L1 and L2. The reader searches for 30. It obtains a read version equal to the version written by the writer, starting from the root path, and finds 30. The reader then scans the range (0, 100). Assume keys 0-20 are stored in L1 and L2 before L0 in the leaf list. The reader will find L0 by following the root pointer. It will then follow a sibling pointer and reach L1. However, it will see that L1 has an undefined version. If it ignores it, it will follow the old pointer to L and continue scanning from there. Therefore, it will return a result without 30. This is not linearizable because the operation performed before it returned 30. A similar execution can be constructed for a variant where the writer frees the path in reverse order (we just have the reader scan first, then lookup). This problem can be solved by using only the root path. However, this makes the scan operation somewhat complex and results in a more expensive hardware implementation. The scan requires storing the entire path from the root to finding the next leaf.Implementing this in software would be more complex, which would result in slower software performance. Release ring

[0152] To summarize, the following steps are performed in each embodiment of the update operation. 1. Execution: Perform traversal, acquire locks, and update data. 2. Get Write Version: Executes the retrieval and addition of the global write version. 3. Complete execution: Set the version of the settings and nodes, and release the lock. 4. Release Update: Wait for your turn in the version sequence and update the globally read version. 5. Clean up: Add deleted nodes to the garbage collection list.

[0153] The following describes a preferred example of how to release an update, as the release step can incur significant overhead and adversely affect system scalability. A naive approach to releasing an update is to spin (i.e., wait) on the global read version until it is 1 less than the write version, then update the reader 402 (e.g., the FPGA) and the global read version. This naive approach carries some signaling overhead. For example, in some implementations, this can be very costly due to the cost of communicating with the FPGA via PCIe—for example, sending a version to the FPGA and receiving an ack response takes approximately 1 microsecond; this means the system update throughput of the naive approach is limited to 1 Mops / s.

[0154] To amortize the cost of communicating with reader 402 (e.g., an FPGA), a release ring data structure can be used to batch communication with the reader. The release ring is a circular buffer with the number of slots equal to a small power of 2 (e.g., 128). Write versions are used to map update operations to slots in the ring via modulo operations. This can be done based on the least significant part of the write version (to calculate the modulo of a number n to the power of 2, e.g., 128, you would take the bottom log_2(n) bits of that number—e.g., 7 bits in the case of 128). An operation announces its readiness to release an update by changing the state of its slot, then checks the state of the first preceding operation. If the preceding operation has not yet released, the operation leaves it to one of the preceding operations to release it, in a batch. If the preceding operation has already released, the operation becomes the releaser and scans the ring forward to collect batches of operations to release. Each batch requires one write to reader 402. Note that concurrent write operations may exist running in different threads. If they write different keys in the tree (or more precisely, different parts of the tree), they can be performed in parallel.

[0155] Figure 13 An example of a release ring with eight slots is shown, along with how it can be used for batch processing. For illustration, Figure 13 The release ring shown has 8 slots. Figure 13 a) The slots are shown in counter-clockwise order. Version 29's operations are mapped to slot 5, and version 30's are mapped to slot 6. Figure 13 b) shows an example of a batch. Operations 26, 27, and 28 are all in the release phase; they are released by 26 as a batch. Operations 29 and 30 have received their write versions but are not yet ready to be released.

[0156] As an example implementation, a slot in the release ring contains a single 64-bit word that stores the slot's 2-bit state and the top bit of the write version of the last operation that used that slot. The version bits stored in the slot and the slot index form the entire version (e.g., for a ring with 128 slots, we store the top 57 bits of the slot as the bottom 7 bits determined by the index). Each slot is filled to the full cache line size to avoid error sharing. Slots can be in one of two states: completed and in progress.

[0157] To release its update, you need to write version v. w The operation first ensures that the slot is not in use by the previous occupant, i.e., it has version v. w The `-ringsize` operation rotates on the globally read version until it is greater than or equal to `v`, ensuring slot availability. w -ringsize means that the previous occupant of the slot has been freed. The operation then announces that it is freeing the update by storing its version in the slot, setting the slot's state to complete, and using a single write instruction. Once announced, the operation can be freed by one of the preceding operations as part of its free batch. The operation spins on the global read version to determine when it is its turn to free the update and whether it has already been freed (or, in other implementations of the ring, other versions of the ring, which can be done by checking the version and state of the preceding slots). If the global read version becomes equal to v... w -1, then the operation becomes the releaser. If it becomes greater than or equal to v w If the operation has been released, then the operation has been released. The release loop supports early return operations when it can be determined that the operation is guaranteed to be released by the preceding operation. To check if it can safely return, the operation checks the state of the preceding slot before rotating on the global version. If the preceding operation has not yet started releasing the batch, this is because the preceding slot is in a completed state or its version is less than v. w In the case of -1, the operation can return without rotating on the global read version.

[0158] To release updates, the operation scans the loop forward to collect a batch of operations to be released. It changes the state of each slot in the batch (including its own slot) to in progress. If an operation's slot is in a completed state and its version is one greater than the previous operation in the batch, that operation is included in the batch. The scan stops when a slot with a version lower than the previous operation is encountered. All operations in the batch are released by updating the global read version by sending the write version of the last operation in the batch to reader 402 (e.g., FPGA). The in-progress state allows for early return—if an operation observes that the previous operation has not moved to the in-progress state, it is guaranteed that the releaser will check its slot and eventually release it.

[0159] Early return can improve concurrency in a system because it reduces waiting, but it can weaken consistency guarantees. For example, if user code inserts a key into tree 103 and the insertion returns early before the operation is released, subsequent lookups for the key may not find it in the tree, leading to unintuitive behavior. Therefore, embodiments may not use early return by default, leaving the decision of whether to use early return to the user. In this case, the user can specify a flag in the API call to request early return. It is also possible to allow the user to wait until a previously executed early-returned operation is released. This allows user code to overlap updated releases with code that has not yet accessed the tree, improving performance.

[0160] Alternative release ring schemes: Several alternative ring implementations are described below. The writer experimentally determined that they do not offer a performance advantage over the versions described above.

[0161] Several different release loop schemes have been considered, each with different trade-offs. The simplest scheme uses only one state: Completed. To release its update, it has a write version v. w The operation first ensures that the slot is not in use by the previous occupant, i.e., it has version v. w The `-ringsize` operation rotates on the global read version until it is greater than or equal to v. w -ringsize. It then sets its version in the slot and rotates until the globally read version is greater than or equal to v. w -1. If it becomes larger, it means the operation has been released as part of another operation in a batch, and therefore the release is complete. If not, the operation becomes the releaser. It scans the ring forward to collect a batch of operations being released. It compares the version stored in the access slot with the expected release version. The expected release version is set to v. wThe scan increments with each newly accessed slot. The scan stops when the first slot with an unexpected (old) version is encountered. All operations in the batch are released by updating the global read version by sending the write version of the last operation in the batch to the reader.

[0162] This simple scheme requires all operations to read the global read version twice: once to determine if the slot is available and once to determine when the release is complete. To reduce contention for the global read version, we introduce another state: released. The releaser moves all slots in the batch to the released state after updating the global read version. It updates slots backward—from the last operation in the batch to the first. To check if its slot is available, it has a write version v. w The operation ensures that the slot version is equal to v. w -ringsize and the slot is in a released state. It does not need to read the global read version. To release an update, the operation first sets the version of its slot to v. w The status is set to completed, and then the preceding slot is checked. It waits until the version of the previous slot is v. w -1 and the state is released, or until its version is greater than v. w It then checks the status of its slot. If its slot is still in version v w Once the operation is complete, it becomes the new releaser. Otherwise, the operation has been released, and it can be returned to the user.

[0163] This scheme supports batching, does not read the global read timestamp, but does not allow early return to non-releasers. To support early return, an in-progress state may still be used. This state allows an operation to distinguish whether its slot is being read by a releaser. If the slot is not being read, the operation can return immediately because it is guaranteed that one of the older operations will release it. If it is being read, it needs to wait to determine whether the releaser includes it in the batch. To release a batch of operations, the releaser scans forward the ring, including every operation in the current batch that is in a completed state with the expected version. When an operation is included in the batch, its slot is set to the in-progress state. After encountering the first operation that cannot be included in the batch, the releaser sends the highest version in the batch to the reader and updates the global read version. It then iterates through all operations in the batch from the last operation to the first operation and sets their states to released. An operation determines whether it should become a releaser as before: it checks the state of the preceding slot after setting the version and state of its own slot to completed. If the preceding slot has the expected version and is in a in-progress state, the operation waits until the state becomes released or the version changes. It then checks its own slot. If its slot has been freed, it returns; otherwise, it becomes a freer. If the preceding slot is in a freed state, or its version is higher than expected, it checks its own slot to decide whether it should become a freer in the same way. If the preceding slot has the expected version and is in a completed state, or its version is lower than expected, the operation can return immediately—ensuring that the freer has not yet read its slot and will read it in the future.

[0164] Note that the above techniques can be mixed and matched – for example, a release mechanism could be implemented that uses a globally read version number to determine if a slot is available, but only checks if it has been released by examining the state of its slot and the previous slot. An alternative to the above approach is to use a dedicated release thread that collects the operation batches in the ring and releases them. Multi-key update

[0165] The disclosed MVCC synchronization scheme can be extended to support multi-key updates or atomic operations that perform mixed read and write operations in a straightforward manner. These operations can be referred to as transactions in this document, and user code executes transactions in the following way. 1. Explicitly start a transaction by calling the Start Transaction call. This call creates a transaction descriptor that ignores all operations to be performed within the transaction. Assign a read version to the transaction by reading the global read version. 2. All lookup and scan operations use the read version of the transaction, rather than retrieving their own. They execute in the same way as independent operations, except that they need to handle post-write read access within the same transaction. The following describes how post-write reads and post-write writes are handled. 3. Insert, update, or delete operations use the read version of the transaction when traversing the tree. It performs all steps as a separate operation, including acquiring locks, until the write version is acquired. The key difference is that the size of a node is not directly changed. Instead, it is stored in a shadow field in the node header. This facilitates handling read-after-write and write-after-write operations. If an operation cannot acquire a lock because it is held by a different operation, the entire transaction will be aborted and retried. 4. When user code executes all operations within a transaction, it calls the transaction commit. The commit operation ensures that the entire transaction is consistent, and if so, it atomically releases all updates made by the transaction. a. The transaction first obtains a write version by atomically incrementing the global timestamp. This version is used to complete all write operations. b. Next, the transaction checks whether the sequence numbers of all accessed leaves remain unchanged and whether they are locked by different transactions. If this is not the case, the transaction aborts. If the required consistency model is snapshot isolation, the verification of the read set can be ignored. If the verification succeeds, the transaction commits. c. All update operations have been completed, similar to standalone operations. This includes assigning node and block versioning, setting node size, marking old nodes for deletion, and releasing locks. d. The transaction enters the release ring to atomically release all updates. e. Similar to standalone operations, perform cleanup. This mainly involves storing the deleted nodes in the GC list.

[0166] Transactions use the same mechanism as independent operations to atomically release all updates. No read operation will observe any changes made by the transaction until the node is fully released, even after the node is unlocked. Transactions also use write locks to synchronize with write operations and rely on early aborts to avoid inconsistencies and deadlocks.

[0167] Operations performed as part of a transaction may need to read or write data that was written by an earlier operation within the same transaction. This means the operation will access nodes that are locked by the same transaction and whose size has changed. It will also access small blocks of unassigned versions. To efficiently handle these read-after-read and write-after-read accesses, we extended the node lock to include a 16-bit thread identifier. Each thread can only execute a single transaction at a time, so the identifier can be used to determine if a node is locked by the current transaction. When a read encounters a node locked by its transaction, it will access that node without checking its version or following the old node pointer. The version of nodes created by a transaction is set to the maximum value, so operations not belonging to the same transaction will ignore it and follow the old pointer. The header is also extended to store the shadow node size, which is only used within the transaction. Write operations performed within a transaction update the shadow node size, and read operations read the shadow node size instead of the public node size. Using the shadow node size allows operations to efficiently read the transaction-local node size. Another option is to store it in a transaction-local table, which incurs higher overhead. Finally, items added to a small block will be assigned an unspecified version number (e.g., 0) during transaction execution, so read operations belonging to the same transaction will access items created in the same transaction.

[0168] When a transaction aborts, it rolls back all changes made so far. This includes rolling back changes to the page table, releasing all newly allocated nodes, and unlocking all locked nodes that have been written to. in conclusion

[0169] It is understood that the above embodiments are described by way of example only.

[0170] More generally, according to one aspect disclosed herein, a system is provided comprising: a memory storing a data structure comprising a plurality of items, each item comprising key-value pairs; a writer configured to perform a plurality of write operations, each write operation writing a corresponding item, or a new item being added to the data structure, or an existing item in the data structure being modified; and a reader configured to perform a group read operation to read any item having a key within a specified range from the data structure; wherein the writer is configured to maintain a global write version, and the reader is configured to maintain a global read version; wherein the writer is configured to perform each of the write operations as follows: i) read and subsequently increment the global write version, ii) write the corresponding item in the data structure, including versioning the corresponding item using a corresponding version number equal to the global write version read by the corresponding write operation prior to the increment, and subsequently iii) generating an index. The writer is configured to send a release indicator corresponding to at least some of the release indicators to the reader, each release indicator signaling the corresponding version number, but not to send a release indicator corresponding to any release indicator indicating a higher corresponding version number until the release indicator indicating all lower version numbers has been generated; the reader is configured to update the global read version to equal the corresponding version number signaled in the release indicator after receiving each release indicator sent by the writer; and the reader is configured to perform the group read operation as follows: read the global read version at the beginning of the group read operation, and then selectively read items from the specified range of keys in the data structure based on a version number equal to or less than the version number of the global read version read at the beginning of the group read operation, ignoring items with a larger version number in the specified range.

[0171] In an embodiment, the writer can be configured to perform at least two write operations within the same atomic operation.

[0172] In one embodiment, the system may include a bus connecting the writer and the reader, and the writer may be configured to send the release signal to the reader via the bus.

[0173] In one embodiment, the writer software may be implemented as stored in a computer-readable storage device of the system and configured to run on one or more processors of the system.

[0174] In this embodiment, the reader may be implemented in a PGA, FPGA, or dedicated hardware circuitry.

[0175] In an embodiment, group reading can be a range scan, which scans through a specified range in order of keys.

[0176] In an embodiment, the data structure includes a tree structure comprising multiple nodes, each node having a node ID, some nodes being leaf nodes, and other nodes being internal nodes. Each internal node is the parent node of a corresponding set of one or more child nodes in the tree structure, each child node being either a leaf node or another internal node. Each leaf node is a child node but not a parent node, and each of the leaf nodes includes a corresponding set of one or more items, and each of the internal nodes maps the node ID of each of its corresponding child nodes to a range of keys contained by the corresponding child node. The writer may be configured to, upon performing each write operation, identify which leaf node to write to as follows: following the key-to-node ID mapping through the tree structure. The reader may be configured to, upon performing a group read operation, determine which leaf node(s) to read from as follows: following the key-to-node ID mapping through the tree structure up to at least one of the leaf nodes.

[0177] In an embodiment, each leaf node further indicates the node ID of one or more sibling leaf nodes that contain a range of keys adjacent to the key of the corresponding leaf node. The reader is configured to determine, if the range scan is read from a range of items spanning more than one leaf node, a leaf node containing one of the keys in the scan range as follows: by following the key-to-node ID mapping through the tree structure, and then by using the ID of at least one of the sibling leaf nodes indicated in the leaf node to determine at least one other leaf node containing at least one other key in the scan range.

[0178] In an embodiment, each leaf node includes a corresponding first block and a corresponding second block, the first block including a plurality of items of the corresponding leaf node ordered in key order in the address space of the memory. A writer may be configured to, upon performing each write operation, write the corresponding new or modified item into the second block of the identified leaf node in the order of writing, rather than the order of key order. A reader may be configured to, upon performing the group read operation, search for items within the specified range at each determined leaf node based on: a) the order of items already ordered in the corresponding first block and b) the reader ordering the items in the corresponding second block by key relative to the items in the corresponding first block. The ignored items may be at least those in the corresponding small block whose version number is greater than the global read version read at the start of the group read operation.

[0179] In an embodiment, the writer is operable to perform a small-large block merge operation at at least some of the leaf nodes, wherein the small-large block merge operation reads and subsequently increments the global write version, merges the corresponding second block with the corresponding first block, and versiones the node with a node version equal to the global write version read by the small-large block merge operation before the increment. In each leaf node where the first and second blocks have been merged, the node also stores a pointer to a copy of a previous version of the node prior to the merge. The reader can be configured to, upon performing the group read operation, and subsequently in each determined leaf node, if any item within the specified range having a corresponding version number greater than the global read version read at the start of the group read operation is encountered in the corresponding first block, follow the pointer to the previous version of the node and attempt to read those items from the previous version.

[0180] In an embodiment, each second block may have a maximum size, and if writing to a leaf node would cause the corresponding second block of the identified node to exceed the maximum size, the writer may be configured to perform a small-to-large merge operation on the leaf node.

[0181] In an embodiment, the writer may be configured not to send a release signal corresponding to each of the release indicators to the reader, but to send at least some release signals corresponding to release indicators having discontinuous version numbers one after another, so as to batch some of the releases together.

[0182] In an embodiment, the writer may include multiple concurrent threads, and at least some write operations are performed by different threads. The system may include a circular buffer with N slots having indices i = 0…N-1. Each write operation may place its release indicator in slot i_r of the circular buffer with N slots, where i_r = v modulo N, and v is the corresponding version number. The writer may be configured to, when a write operation places its corresponding release indicator in the circular buffer, check whether the slot with the immediately preceding index contains a release indicator indicating the next lowest version number, and...

[0183] If not, the corresponding release indicator is left in the circular buffer to allow the corresponding item to be released by another write operation, but

[0184] If so, a batch of release indicators is determined, including the corresponding release indicator and any release indicators occupying consecutive slots immediately preceding the corresponding release indicator of the consecutive run with the indicated version number, and a single release signal is sent to the reader, the single release signal signaling the determined maximum version number in the run, causing the reader to update the global read version to this version number, thereby releasing a batch of items corresponding to the batch of release indicators.

[0185] According to other aspects disclosed herein, methods and / or computer programs, or entire systems, for performing operations of a writer and / or reader, may be provided, as in any of the embodiments disclosed herein.

[0186] Other variations or applications of the disclosed technology will become apparent to those skilled in the art given the disclosure herein, and the scope of this disclosure is not limited to the described embodiments but to the appended claims.

Claims

1. A system comprising: A memory that stores a data structure containing multiple items, each item consisting of key-value pairs; The writer is configured to perform multiple write operations, each write operation writing a corresponding item, or adding a new item to the data structure, or modifying an existing item in the data structure. as well as A reader is configured to perform a group read operation to read any item from the data structure that has a key within a specified range; The writer is configured to maintain a global write version, and the reader is configured to maintain a global read version; The writer is configured to perform each write operation as follows: i) read and subsequently increment the global write version, ii) write the corresponding item into the data structure, including versioning the corresponding item with a corresponding version number equal to the global write version read by the corresponding write operation before the increment, and iii) generate a release indicator indicating the corresponding version number. The writer is configured to send a release signal to the reader corresponding to at least some of the release indicators, each release signal signaling the corresponding version number, but not to send a release signal corresponding to any release indicator indicating a higher corresponding version number until the release indicator indicating all lower version numbers has been generated; The reader is configured to update the global read version to be equal to the corresponding version number signaled in the release signal after receiving each release signal sent by the writer; as well as The reader is configured to perform the group read operation as follows: at the beginning of the group read operation, read the global read version, and then selectively read items from the specified range of keys in the data structure based on a version number equal to or less than the global read version read at the beginning of the group read operation, ignoring items in the specified range with a larger version number.

2. The system of claim 1, wherein the writer is configured to perform at least two write operations within the same atomic operation.

3. The system of claim 1 or 2, wherein the system includes a bus connecting the writer and the reader, and the writer is configured to send the release signal to the reader via the bus.

4. The system of claim 1 or 2, wherein the writer is implemented by software stored in a computer-readable storage medium of the system and is disposed on one or more processors of the system.

5. The system according to claim 1 or 2, wherein the reader is implemented in a PGA, FPGA or dedicated hardware circuit.

6. The system of claim 1, wherein the group read is a range scan that scans the specified range in order of keys.

7. The system according to claim 6, wherein: The data structure includes a tree structure, which includes multiple nodes, each node having a node ID. Some nodes are leaf nodes, and other nodes are internal nodes. Each internal node is the parent node of a corresponding set of one or more child nodes in the tree structure. Each child node is either a leaf node or another internal node. Each leaf node is a child node but not a parent node. Each of the leaf nodes includes a corresponding set of one or more items. Each of the internal nodes maps the node ID of each of its corresponding child nodes to a range of keys contained in the corresponding child node. The writer is configured to, upon performing each write operation, identify which leaf node to write to as follows: by following the key-to-node ID mapping through the tree structure; and The reader is configured to determine, when performing the group read operation, which leaf nodes(s) to read from as follows: following the key-to-node ID mapping through the tree structure up to at least one of the leaf nodes.

8. The system according to claim 7, wherein: Each leaf node further indicates the node ID of one or more sibling leaf nodes that contain a range of keys adjacent to the key of the corresponding leaf node; and The reader is configured such that if the range scan reads from a range of items spanning more than one leaf node, then: - A leaf node containing one of the keys in the scanned range is determined as follows: by following the key-to-node ID mapping through the tree structure, and then... - By using the ID of at least one of the sibling leaf nodes indicated in one of the leaf nodes, at least one other leaf node containing at least one other key in the scanned range is determined.

9. The system according to claim 7 or 8, wherein: Each leaf node includes a corresponding first block and a corresponding second block, the first block including a plurality of the items of the corresponding leaf node ordered in key order in the address space of the memory; The writer is configured to, when performing each write operation, write the corresponding new or modified item into the second block of the identified leaf node in the order of writing rather than the order of key sorting; as well as The reader is configured to, when performing the group read operation, search for items within the specified range at each determined leaf node based on: a) the order of items already sorted in the corresponding first block and b) the reader sorting the items in the corresponding second block by key relative to the items in the corresponding first block; The ignored items are at least those items in the corresponding small block that have a version number greater than the version of the global read read at the beginning of the group read operation.

10. The system according to claim 9, wherein: The writer is operable to perform a small-large block merge operation at each of at least some of the leaf nodes, wherein the small-large block merge operation reads and then increments the global write version, merges the corresponding second block with the corresponding first block, and versiones the node with a node version equal to the global write version read by the small-large block merge operation before the increment. In each leaf node of the first and second blocks that have been merged, the node also stores a pointer to a copy of the previous version of the node prior to the merge; The reader is configured to, when performing the group read operation, then in each determined leaf node, if any item in the specified range with a corresponding version number greater than the global read version read at the start of the group read operation is encountered in the corresponding first block, follow the pointer to the previous version of the node and attempt to read those items from the previous version.

11. The system according to claim 10, wherein: Each second block has a maximum size; and The writer is configured to perform a small-to-large block merge operation on the leaf node if writing an item to the leaf node would cause the corresponding second block of the identified node to exceed the maximum size.

12. The system of claim 1 or 2, wherein the writer is configured not to send a release signal corresponding to each of the release indicators to the reader, but to send at least some release signals corresponding to release indicators having discontinuous version numbers one after another, so as to batch some of the releases together.

13. The system according to claim 1 or 2, wherein: The writer includes multiple concurrent threads, and at least some of the write operations are performed by different threads within the writer; The system includes a circular buffer comprising N slots with indices i=0…N-1; Each write operation places its release indicator in slot i_r of a circular buffer with N slots, where i_r = v modulo N, and v is the corresponding version number; The writer is configured to, when a write operation places a corresponding release indicator in the circular buffer, check whether the slot with the immediately preceding index contains the release indicator indicating the next lowest version number, and - If not, leave the corresponding release indicator in the circular buffer to allow the corresponding item to be released by another write operation, but - If so, determine a batch of release indicators, which includes the corresponding release indicator and any release indicators occupying consecutive slots immediately preceding the corresponding release indicator of the consecutive run with the indicated version number, and send a single release signal to the reader, the single release signal signaling the determined maximum version number in the run, causing the reader to update the global read version to this version number, thereby releasing the batch of items corresponding to the batch of release indicators.

14. A method comprising: The storage consists of a data structure containing multiple items, each item consisting of key-value pairs; The writer performs multiple write operations, each write operation writes a corresponding item, or a new item is added to the data structure, or an existing item in the data structure is modified. The reader performs a group read operation to read any item with a key within a specified range from the data structure; The writer maintains a global write version, and the reader maintains a global read version; Each of the write operations includes: i) reading and subsequently incrementing the global write version, ii) writing the corresponding item in the data structure, including versioning the corresponding item with a corresponding version number equal to the global write version read by the corresponding write operation before the increment, and iii) generating a release indicator indicating the corresponding version number; The method includes: the write sending a release signal to the reader corresponding to at least some of the release indicators, each release signal signaling the corresponding version number, but not sending a release signal corresponding to any release indicator indicating a higher corresponding version number until the release indicator indicating all lower version numbers has been generated; The reader, upon receiving each release signal from the writer, updates the global read version to be equal to the corresponding version number indicated by the signal in the release signal; and The group read operation includes: reading the global read version at the beginning of the group read operation, and then selectively reading items from the specified range of keys in the data structure based on a version number that is equal to or less than the version number of the global read version read at the beginning of the group read operation, ignoring items in the specified range that have a larger version number.

15. A computer storage medium storing computer-executable instructions, which, when executed by a device, cause the device to perform the method according to claim 14.

16. A computer program product tangibly stored in a computer storage medium and comprising computer-executable instructions that, when executed by a device, cause the device to perform the method according to claim 14.

Images