Tree-based data structure

Abstract

A writer writes items to leaf nodes of the tree, and a reader reads items from the leaf nodes. Each node has a respective first block and a second block, and the first block has a number of items for the respective leaf sorted in key order. When a writer writes a new item to a leaf, it writes the new item to the second block of the identified leaf node in the order in which it was written, rather than sorted in key order. When a reader reads one or more target items from a leaf, it searches the leaf for one or more target items based on a) the order of the items already sorted in the first block, and b) the reader sorts the items in the second block by keys associated with the items in the first block.

Description

Background Art

[0001] Background Tree structures such as B-trees can be used as index data structures for storing key-value pairs (each pair includes a value which is the content to be stored and a key which is used to index that value). Each leaf node of the tree contains a plurality of items which are key-value pairs. As one proceeds further up the tree from the leaf, each internal node of the tree contains an indication of the range of key values encompassed by each of its children (which can be leaves or other internal nodes). When a new item is added to the tree, the writer uses the tree structure to determine which leaf contains the range of keys into which the key of the new item falls. If this leaf is not full, the new item can simply be added to the existing leaf. However, if the leaf is full (leaves and internal nodes typically have a maximum size in bytes), the leaf is split. That is, a new leaf is created. This also means updating the parent node of these two leaves so as to correctly reference the key ranges of the old leaf and the new leaf. This update to the parent may cause the parent to be split if it exceeds its maximum size, and the reference to the grandparent to be updated, and so on. When an item is deleted, this may also involve merging leaves or internal nodes.

[0002] In certain applications, there are several operations that may be required of a writer or reader. A reader can read out individual items with a specific key or perform a range scan to read out from a range of keys. A writer can write new entries, modify existing entries, or delete entries.

[0003] Index data structures are used in a wide range of software systems, such as data stores and databases, file systems, and operating systems. Index data structures store key-value pairs and support various operations, including lookup, scan, insertion, and deletion of key-value pairs. A B-tree is one such index data structure. A B-tree is an organized index, meaning it also supports scan operations. A scan operation returns all key-value pairs stored in the tree that have keys within a specified range. [Overview of the project]

[0004] overview In tree-based data structures such as B-trees, it is desirable to balance the workload between writers and readers. For example, if new entries are simply added chronologically and not sorted by key, this is very fast for writers. However, readers must sort items at read time in order to perform individual reads or range scans (at least range scans require sorting—individual reads are simply easier to implement with sorting, both lookups and range scans). On the other hand, if the writer sorts all new entries in order each time it writes, this makes reading very fast for readers but places a greater burden on the writer at write time. It is desirable to find a compromise between these two approaches.

[0005] According to one embodiment disclosed herein, a system is provided having memory for storing a data structure, the data structure comprising a tree structure comprising a plurality of nodes, each having a node ID. Some nodes are leaf nodes, and others are internal nodes, each internal node being the parent of each set of one or more children in the tree structure. Each child is either a leaf node or another internal node, and each leaf node is a child but not a parent. Each leaf node comprises each set of one or more items, each having a key-value pair, and each internal node maps each node ID of each child to a range of keys encompassed by each child. The system further comprises a writer, implemented in software, hardware, or a combination thereof, configured to write items to the leaf nodes, and a reader, implemented in software, hardware, or a combination thereof, configured to read items from the leaf nodes. Each leaf node comprises its first block and its second block, the first block comprising a plurality of items of the leaf node sorted in key order in the address space of memory. The writer, when writing a new item to the tree structure, identifies which leaf node to write to by traversing the tree structure and the mapping of keys to node IDs, and is configured to write the new item to the second block of the identified leaf node in the order it was written, rather than sorted by key order. The reader, when reading one or more target items from the tree structure, determines which leaf node to read from by traversing the tree structure and the mapping of keys to node IDs, and is then configured to look up the determined leaf node for one or more target items based on a) the order of items already sorted in the first block and b) the reader sorting the items in the second block by the keys associated with the items in the first block.

[0006] In embodiments, the first block may have a larger maximum size than the second block, and thus the first block may be called the “large block” and the second block the “small block.” In embodiments, the writer may be implemented in software, while the reader may be implemented in custom hardware, such as a programmable gate array (PGA) or field-programmable gate array (FPGA). More generally, however, the writer may be implemented in hardware (e.g., a hardware accelerator), and / or the reader may be implemented in software.

[0007] Brief explanation of the drawing To aid in understanding the embodiments disclosed herein and to illustrate how such embodiments may be carried out, the following drawings are referenced, merely as examples. [Brief explanation of the drawing]

[0008] [Figure 1] A schematic diagram illustrates a range scan in the form of an inclusive scan (where the interval formed by the keys of the returned items is the largest subset that falls within the input interval) on an exemplary key interval [14,22]. [Figure 2] A schematic diagram illustrates a range scan in the form of a covering scan (where the interval formed by the keys of the returned items is the smallest superset containing the input interval) on an exemplary key interval [14,22]. [Figure 3] A schematic diagram illustrates a range scan in the form of a covering scan for an exemplary key interval (14,22). [Figure 4] This is a schematic block diagram of a system according to an embodiment disclosed herein. [Figure 5] A schematic diagram of a data structure with a tree structure is shown. [Figure 6] A schematic diagram of a node consisting of small and large blocks is shown. [Figure 7]A schematic diagram illustrates how to insert a new item into a small block. [Figure 8] A schematic diagram illustrates how to include order hints to sort small blocks. [Figure 9] A schematic diagram illustrates how to sort small blocks based on order hints. [Figure 10] This is a schematic circuit diagram of a 4-element small block sorter. [Figure 11] A schematic diagram illustrates how to merge large and small blocks during an insertion operation. [Figure 12] A schematic diagram illustrates how leaf nodes are divided during insertion. [Modes for carrying out the invention]

[0009] Detailed description of the embodiment This disclosure provides a node layout in which each node consists of a first and a second block. Preferably, the first block has a larger maximum size (in bits or bytes) than the second block, and thus the first and second blocks may be referred to as the large block and the small block, respectively. The following description will be based on the large block and the small block as examples, but wherever below they may be replaced in principle with the more general terms "first block" and "second block," respectively.

[0010] Large blocks are sorted, which enables efficient node searching. In this context, sorting means that items are sorted by key so that the order of items by key matches the order in which items appear in the memory address space (or virtual or physical address space, depending on the implementation). Small blocks, on the other hand, contain the most recent insertions and deletions to nodes and are chronological, which enables fast updates. Alternatives include keeping the entire node unsorted, which provides fast update performance but degrades search performance, or keeping the entire node sorted, which provides fast search performance but degrades update performance.

[0011] Optionally, shortcut keys may be used within large blocks. This allows node searches to examine only a small portion of the entire node set, resulting in improved performance.

[0012] As another, alternative, or additional optimization, in some embodiments, “back pointers” may be included in the small blocks. These allow for the establishment of order between items in the small and large blocks without comparing keys. This results in a more efficient implementation (e.g., in hardware) because it eliminates the need to compare the entire key.

[0013] As another optional optimization that can be used with or independently of shortcut keys and / or the back pointer, an embodiment may include “order hints” within the subblock. Order hints within the subblock enable highly efficient hardware for establishing a sorted order of items within the subblock without comparing keys.

[0014] Another optional optimization is to provide synchronization for complex updates. In this case, when nodes are split or merged, the writer creates a copy of the subtree (e.g., a thread-private copy) and then updates the tree by swapping it with a single pointer update of the page table.

[0015] In this embodiment, the writer may be implemented in software running on one or more processors, while the reader may be implemented in specialized hardware. The hardware may include a dedicated (i.e., fixed-function) circuit network such as an FPGA or PGA, or an ASIC (Application-Specific Integrated Circuit).

[0016] The special hardware implementing the index data structure is hardware having a dedicated pipeline for executing data structure operations. This is in contrast to a general-purpose CPU that implements an instruction set and executes a program that implements data structure operations using that instruction set. The special hardware has several advantages. It can have a higher throughput per watt, be more predictable and have less latency, and can be less expensive than the same functionality implemented as software on a CPU. The main drawbacks of using special hardware are that the design is more difficult, it takes longer to build than software, and in the case of dedicated hardware, it cannot be changed after construction and is thus typically only used for very widely deployed functionality. However, it should be noted that the scope of the present disclosure is not limited to implementing writes in software and reads in hardware. As another possibility, the writer may be implemented in some form of hardware such as an FPGA or ASIC, and / or the reader may be implemented in software. The embodiments disclosed herein provide an in-memory B-tree that benefits from both advantages. That is, all operations can be executed in software on a CPU, and the lookup and scan operations can be executed on the special hardware we designed. In many workloads, the lookup and scan are the main operations, meaning that most operations benefit from hardware offloading. The lookup and scan are simpler than update operations, meaning that the system as a whole can be built and deployed faster. In one exemplary implementation, the memory may be a DRAM-based host memory, but this is not limiting, and in other examples, the B-tree can be stored in a storage NVM device such as an SSD or a 3D-Xpoint drive.

[0017] Examples of the techniques described above will now be described in more detail with reference to the figures.

[0018] The following is described from the perspective of a B-tree, but more generally, the ideas disclosed herein, such as different types of read and write operations, the use of large and small blocks, shortcut keys, back pointers, order hints, and / or synchronization of complex updates (splitting and merging), can be applied in any tree structure for storing key-value pairs.

[0019] Tree structure Key-value stores such as B-trees store data as key-value pairs (also called items). Items may be of variable size, but depending on the implementation, the data structure may limit the total allowable size of items (e.g., up to 512 bytes), or in some implementations, they may be made of fixed size to simplify the implementation. Each key is associated with a single respective value, and multiple key-value pairs with the same key are not permitted.

[0020] Figures 1, 4, and 5 illustrate, as an example, elements of a tree structure 103, such as a B-tree. The tree 103 is a form of data structure implemented in memory, which can be used as a key-value store. The tree 103 comprises several nodes 102, some of which are leaf nodes 102L, and others are internal nodes 102I. Each node 102 has its own node ID, which may also be called a logical ID (LID) in this specification. Each leaf node 102L comprises its own set of one or more items 101, each item 101 comprising a key-value pair (for example, Figure 1 shows a node 102 undergoing a range scan, but this is not limiting and is merely an example of the type of read operation). The key-value pair acts as an index for the item, comprising a key mapped to a value which is the content of the item. The numbers shown inside the items 101 in the figures are illustrative keys (in reality, keys can reach much larger numbers, but this is for illustrative purposes only). Each internal node 102I specifies each set of one or more children of the internal node. When the tree is constructed, at least one or more internal nodes will each have two or more children (however, if the tree has only one leaf, for example when it has just been created, the root node will have only one child). Each internal node 102I also indicates which ranges of the key are encompassed by each of its children. The specific scheme shown in Figure 5, where the key is sandwiched between pointers, as described later, is just one possible example of how to specify this mapping.

[0021] Regardless of the means by which the mapping is implemented, the node ID of an internal node 102I thus defines the edges between parent and child nodes, thereby forming a tree structure. Each internal node 102I is the parent of at least one of its children. Once the tree is constructed, each of at least one or more internal nodes becomes the parent of multiple children (however, as mentioned above, if the tree has only one leaf, the root has a single child). A child of a parent can be either a leaf node 102L or another internal node 102I. A leaf node 102L is only a child and not a parent (i.e., a leaf is the bottom of the tree, or in other words, the end of a branch of the tree).

[0022] In this specification, a key being “contained” by a child means that the child is a leaf node 102 containing an item (key-value pair) having that key, or that the child is another internal node 102I that, through node ID-to-key mapping, ultimately connects to a leaf node 102I containing an item having that key, one or more levels down the tree hierarchy (or “generations”).

[0023] The root node is one of the internal nodes 102I at the top of the tree. When a writer or reader writes to or reads from an item with a specific key, it begins by querying the mapping of node IDs to the key range specified on the root node to find which child of the root contains the required key. If that child itself is another internal node 102I, the writer or reader queries the mapping specified on that node to find which child of the next generation contains the required key, and so on, until it finds a leaf node 102L that contains the item with the required key.

[0024] In this embodiment, key comparisons may follow the semantics of the C memcmp function, meaning that integer keys should be stored in big-endian format to maintain the semantics of integer comparisons. However, this is only one possible example.

[0025] In this embodiment, the B-tree may take the form of a B+ tree, where only the leaf nodes 102L contain key-value pairs, and only the internal nodes 102I contain node ID-to-key range mappings.

[0026] Read / write operation Tree structures such as B-trees can support read operations of individual lookups and / or range scans. They can also support write (update) operations of insert, modify, and / or delete. Exemplary semantics of such operations are described below.

[0027] Lookup: This takes a key as an argument and returns the value associated with that key if it exists in the tree. Otherwise, it returns the status "not-found".

[0028] Scan: This takes two keys as input arguments and returns all key-value pairs from a tree containing keys within the closed interval [low-key,high-key]. In embodiments, two types of scans are supported: comprehensive scans and covering scans. Figures 1, 2, and 3 illustrate the difference. (l,u) means that even if the lower bound l and upper bound u are in the tree, they are not included in the result. [l,u] means that if they are in the tree, they are included. Combinations such as [l,u) where l is included but u is not, and (l,u) where l is not included but u is, are also possible.

[0029] In a comprehensive scan (Figure 1), the interval formed by the keys of the returned items is the largest subset that falls within the input interval. A comprehensive scan can be used when providing a database query such as "Retrieve all items with keys between 36 and 80". The result will not include items with key 35, even if there are no items with key 36. In a covering scan (Figures 2 and 3), the interval formed by the keys of the returned items is the smallest superset that contains the input interval. In Figure 2, an item with key 22 is returned, but in Figure 3, it is not. A covering scan is useful when querying storage metadata such as "Retrieve disk blocks that store data from the beginning of the file in the offset range of 550 to 800". The result will include the largest key before 550 and the smallest key after 800 (i.e., keys on both sides of the specified range), meaning all disk blocks that store data within the requested range are returned.

[0030] Insert: This takes a new key-value pair to insert. If an item with the specified key does not exist in the tree, that key-value pair is inserted into the tree. Otherwise, this operation does not update the tree and returns the status "already-exists".

[0031] Correction: This takes an existing key-value pair to update. If an item with the specified key exists in the tree, the value is updated to the value of the input argument. Note that the value may be of a different size than before. Otherwise, this operation does not update the tree and returns the status "Not Found".

[0032] Delete: This takes the key of the item to remove. If the item exists in the tree, it is removed. Otherwise, this operation returns the status "Not Found".

[0033] System Architecture Figure 4 shows an exemplary system according to an embodiment. This system comprises a writer 401, a reader 402, system memory 403, and optionally, local memory 404 of the reader 402. The tree structure 103 is stored in the system memory 403. The writer 401 and the reader 402 are operably coupled to the system memory 403. For example, the reader 402 may be operably coupled to the writer 401 via a bus 405 such as a PCIe bus, while the writer 401 may be operably coupled to the system memory 403 via a separate local connection or dedicated connection 406 that does not require communication via the bus. The writer 401 may be configured to access the system memory 403 to write to the tree 103 via the local connection 406, but the reader 402 may need to access the memory 403 via the bus 405 to read from the tree. The reader 402 may be connected to its local memory 404 via its own local connection 407, thereby being configured to cache a portion of the tree 103 in the reader's local memory 404.

[0034] Memory 403 can take any suitable form and may comprise one or more memory media embodied in one or more memory units. For example, the memory media may comprise electronic silicon-based media such as SRAM (Static Random Access Memory) or DRAM (Dynamic RAM), EEPROM (Electrically Erasable and Programmable ROM), flash memory; or magnetic media such as magnetic disks or tapes; or more exotic forms such as rewritable optical media or synthetic biological storage; or any combination of any of these and / or other types. The one or more memory units in which memory 403 is embodied may comprise on-chip memory units on the same chip as writer 401, separate units on the same board as writer 401, or external units such as SSDs (Solid State Drives) or hard drives, or any combination thereof, which may be mounted in the same housing or rack as writer 401, or in a different location in a separate housing or rack. In a particular embodiment, as shown, memory 403 comprises DRAM which may be mounted on the same board or chip as writer 401. For example, in a particular case, the system uses a DRAM board that is plugged into the board where the CPU resides. However, this is not the only example.

[0035] The reader's local memory 404, when used, can also take any suitable form comprising any suitable medium in any one or more units. In a particular embodiment, as shown, the reader's local memory 404 comprises DRAM which may be integrated into the reader 402 or mounted on the same chip or board as the reader 402.

[0036] In some embodiments, the writer 401 is implemented in software that is stored in the system's non-temporary computer-readable storage and is configured to run on one or more processors of the system, e.g., a CPU, while the reader 402 is implemented in hardware that may take the form of an FPGA, PGA, or dedicated (fixed-function) hardware network, or a combination thereof. The following may be described in terms of such embodiments, but it should be recognized that none of the techniques described below exclude the possibility that the writer 401 may instead be implemented in hardware and / or the reader 402 may be implemented in software.

[0037] The storage used to store the software of writer 401 (and / or reader in alternative embodiments where the reader is implemented in software) may be any of the above, or alternatively, any suitable form such as read-only memory (ROM) such as electronic silicon-based ROM, or optical media such as CD-ROM. One or more processors may include, for example, a general-purpose CPU (central processing unit); or an application-specific processor or accelerator processor such as a GPU (graphics processing unit), DSP (digital signal processor), cryptographic processor, or AI accelerator processor; or any combination of these and / or other types.

[0038] In certain embodiments, as illustrated, the writer 401 is implemented in software running on a CPU, and the reader 402 is implemented on an FPGA. The following embodiments may be described in terms of such examples, but it should be noted that this is not limiting.

[0039] The B-tree stores data in system memory 403. In some embodiments, this allows for the storage of trees the same size as the system memory (typically several hundred GB), but at the cost of the reader 402 (e.g., FPGA) accessing system memory via PCIe 405. The reader (e.g. FPGA) 402 can maintain a cache of the upper layers of the tree in its local memory (e.g., DRAM) 404 to speed up execution. In one exemplary implementation, the reader 402 records the LID of the root node and the number of levels in the tree in its local register file.

[0040] B-trees may be designed to allow data to be stored in secondary storage as well, enabling the storage of even larger trees. The emergence of ultra-low latency NVMe (Non-Volatile Memory Express) devices makes this an attractive option even in low-latency scenarios.

[0041] In this embodiment, FPGA 402 performs only lookup and scan operations, while update operations are performed by CPU 401. Synchronization between the CPU and FPGA is preferably lightweight. Only when B-tree nodes are split or merged may it require communication via PCIe bus 405 (see the Data Structures section for further details). By supporting only read operations to the FPGA, read-intensive use cases can benefit from hardware offloading without having to implement all operations in hardware.

[0042] data structure In this embodiment, the B-tree 103 is a B+ tree, meaning that key-value pairs 101 are stored only in the leaf nodes 102L, and the internal nodes 102I store key indices and pointers to child nodes.

[0043] Figure 5 shows an example of a two-level tree, but the principle described here can be applied to any number of levels (i.e., any number of parent and child levels).

[0044] The data structure comprising the tree 103 further comprises a page table 501. The page table 501 maps node IDs (also called logical IDs, LIDs) to the actual memory addresses in which each node 102 is stored in system memory 403. In this way, the writer 401 or reader 402 can determine the location of node 102 in memory 403 based on the node ID (LID) by looking up the address based on the node ID in the page table 501.

[0045] As mentioned above, each internal node 102I has information that maps each of its child node IDs to an indication of the range of keys encompassed by each child. Figure 5 illustrates an example of how this can be implemented.

[0046] In this example, each internal node 102I has a set of node pointers 502 and key indices 503. Each node pointer 502 is a pointer to a child node specified by the node ID (logical ID, LID). Each key indice 502 in this example specifies one of the keys at the boundary of the range contained by the child. In each internal node 102I, a pointer 502 with key l on its left and key u on its right points to a subtree that stores keys in the interval [l, u). In addition, each internal node 102I stores a pointer 502 to a subtree with keys smaller than the smallest key within that internal node. This leftmost pointer is stored in the node header 603 (see, for example, Figure 6). In other words, each key indice is sandwiched between a pair of pointers, and the key indice specifies the boundary between the ranges contained by the children pointed to by the pointers on both sides of the key indice. To put it another way, an internal node stores leftmost child lids. These essentially store X keys and X l pointers. The leftmost point is the +1 pointer. In the specific example shown in Figure 5, new items with keys within the interval [22, 65) are inserted into the middle leaf, items with keys less than 22 are inserted into the leftmost leaf, and items with keys 65 or greater are inserted into the rightmost leaf.

[0047] It will be recognized that the scheme for mapping children to key ranges, shown in Figure 5, is just one example. Another example is, for instance, to first store all boundary keys in a node, and then all LIDs follow them.

[0048] In some embodiments, each leaf 102L may also store a pointer to the left leaf and a pointer to the right leaf in order to simplify the implementation of range scanning.

[0049] B-tree 103 grows from the bottom (like other B-trees). If a new item 101 cannot fit into the leaf 102 it is mapped to during insertion, the leaf is split in two. The items in the old leaf are split in half, and a pointer to the new leaf is inserted into the parent node. When the parent node becomes full, it is also split. Splitting can continue recursively down to the root of the tree. When the root is split, a new root is created, which increases the height of the tree. When a leaf becomes empty because all items have been removed from it, it is removed from the parent. Alternatively, another option is to merge the leaves when they fall below half their original size in order to maintain the spatial invariance of the B-tree.

[0050] Node layout In this embodiment, the B-tree nodes 102 may be 8KB in size to align them with the DRAM page size. They may be allocated to pinned memory that cannot be moved by the operating system, allowing the reader 402 (e.g., FPGA) to access the nodes 102 using physical memory addresses. However, this is not required.

[0051] Internal nodes 102I preferably do not directly store the addresses of child nodes. Instead, they store the logical identifier (LID) of each node. In one example, the size of the LID may be 6 bytes, which would allow the maximum size of the tree to be 2 61The number of bytes is limited. The mapping from a node's LID to its virtual and / or physical address is maintained in page table 501. Page table 501 can be stored both in system memory 403 and FPGA-attached memory 404. When a new node mapping is created or a node mapping is changed, the writer 401 (e.g., CPU) updates table 501 in system memory 403. If a cache is maintained on the reader side, the writer 401 also issues a command to the reader 402 (e.g., FPGA) (e.g., via PCIe) to update a copy of the table in the reader's attached memory 404. Alternatively, the reader may poll the tree for updates.

[0052] Addressing nodes using LID provides a level of indirection, enabling the storage of nodes on NVMe devices. It also helps synchronize access to the tree, as update operations can sometimes perform copy-on-write operations on nodes.

[0053] In this embodiment, the layout of the internal node 102I and the leaf node 102L are the same. See, for example, Figure 6. The node header 603 may be stored, for example, in the first 32 bytes of node 102. The header 603 may contain a field specifying the type of node 102 (internal or leaf), the number of bytes used in the node, a lock bit and node version number for synchronizing access to the node, and / or several other fields described later. In an example as shown in Figure 5, the header 603 of the internal node 102I also stores the LID of the leftmost child. The header of the leaf node 102L may store the LIDs of its left and right siblings. Since the keys and values ​​may be of variable size, they may be stored as blobs. Each blob may have, for example, a 2-byte header specifying its size. Since the LID's size is known, it may be stored without a header. The values ​​may be stored inline in the B-tree node 102L to improve sequential access performance. Values ​​larger than 512 bytes may be stored outside of the B-tree 103, and a pointer to the value is stored in leaf 102L.

[0054] Regardless of the selection of specific implementation parameters such as the node size exemplified above, according to the embodiments disclosed herein, each leaf node 102L comprises two blocks: a large block 601 and a small block 602. To implement this, a pointer to the boundary between the blocks may be stored in the node header 603. This pointer can be represented, for example, as an offset relative to the starting position of the large block 601. The large block 601 stores items in sorted order, and the small block 602 stores a log of recent updates. When the small block 602 exceeds a threshold (e.g., set to 512 bytes), it is merged into the large block 601. By dividing the node 102L into a large block and a small block, read operations benefit from accessing nearly sorted data without the overhead of sorting the node after each update. Entries in the small block 602 may be either newly inserted items or deletion markers. As an optional optimization, each entry in the small block 602 may further store a pointer (e.g., 2 bytes long) to an item in the large block 601. For convenience, this is sometimes called a "back pointer" (although the term "back" does not necessarily imply limiting it to a specific direction, such as in the physical address space). In one exemplary implementation, for a newly inserted item 101, the pointer points to the first item in large block 601 with a key greater than the key of the new item added to small block 602. Or, for a delete marker, the pointer points to the deleted item. The pointer can be represented as an offset within node 102. It can be used by reader 402 to establish the order between items 101 in large and small blocks without comparing their keys, which is a useful optimization, especially when reads are implemented in hardware.

[0055] As another optional optimization, each item within small block 602 may have a field (e.g., 1 byte long) that stores its index within the sorted small block at the time of insertion. This is the order of the item at insertion. This is sometimes called an "order hint." These indices are "recreated" by reader 402 as it scans the small blocks during retrieval to reconstruct the current sorted order. This order is stored in reader 402's temporary "indirect reference" array (e.g., FPGA registers), which is used to access items in the sorted order. This makes sorting more efficient and avoids introducing large latency, especially when implemented in hardware.

[0056] Figure 7 shows an example of the steps for inserting a new item 101 into small block 602. Step a) shows node 102 before insertion. In step b), item 101 is copied into small block 602. It is not yet visible to reader 402. In step c), the size of the data shown in node header 603 is changed to include the new item.

[0057] As another optional optimization, to optimize key lookup within a node, the large block 601 may be divided into segments of approximately equal size. Keys at segment boundaries are stored in a shortcut block 604, which may be stored immediately after the node header 603, along with a pointer to the boundary. Key lookup begins by scanning the shortcut block and identifying segments that may contain the key. The search examines only these segments and small blocks, which reduces the amount of data read from memory. For example, if the performance bottleneck is the PCIe bandwidth between the reader (e.g., FPGA) 402 and the system memory 403, this optimization significantly improves performance. For keys of 32 bytes or less, the search reads approximately 1.5KB of data from an 8KB node, thereby achieving a 5x performance improvement compared to a simpler implementation that scans the entire node. Shortcut keys may be selected during the merging of the large block 601 and small block 602.

[0058] Node cache FPGA 402 may have several gigabytes (e.g., 4GB) of DRAM 404 directly attached to it. While this memory may not be large enough to store the entire B-tree 103 in all use cases, it may be large enough for the first few levels, so in embodiments it can be used as a cache to reduce B-tree latency and improve B-tree throughput. For example, cache 404 may be allocated at, for example, 8KB node granularity to reduce the amount of metadata, but data may be fetched in 256-byte chunks to reduce the amount of unnecessary data read through PCIe 405. FPGA 402 may maintain a 32-bit occupancy bitmap for each cache node 102 to keep track of the chunks in cache 404. Page consistency between cache 404 and system memory 403 can be maintained by invalidating cached pages whenever the page mapping is updated. Software on CPU 401 can send a page table update command through PCIe 405 when the page mapping is updated, and FPGA 402 can invalidate the page by clearing the occupancy bitmap.

[0059] Page Table A reader 402 (e.g., an FPGA) can maintain a local copy of the page table 501 stored in its local memory 404 (e.g., DRAM), which may be attached to the FPGA device 402. In this embodiment, each entry in the table is 8 bytes in FPGA memory but 16 bytes in system DRAM. This is because, in this embodiment, the copy in system DRAM stores two addresses (virtual and physical), while the copy in FPGA DRAM memory stores one address (physical only).

[0060] The entries in page table 501 store the memory addresses of nodes 102 mapped to each LID. Generally, these may be physical or virtual addresses depending on the implementation. In embodiments, the writer employs virtual memory mapping, and the page table 501 on the writer side stores the virtual addresses of nodes mapped to each ID. In embodiments where a copy of the page table is held on FPGA 402, the entries in the FPGA page table copy may store the physical addresses of nodes mapped to each LID. For example, with 8-byte entries and 4GB of DRAM, the system can support trees up to 4TB, which is large enough for the main memory trees of today's servers. Larger trees can be supported by allowing the reader 402 (e.g., FPGA) to access the page table 501 of system memory 403 or an NVMe device, or by increasing the node size of the B-tree.

[0061] The page table 501 is not absolutely necessary in all possible embodiments. A less preferred alternative is to use the memory address directly as the node pointer for the internal node 102I.

[0062] Look Up The lookup starts from the root and traverses the internal tree nodes 102I at each level to find a pointer to the node 102 at the next level. Each lookup ends at a leaf 102L. If the leaf contains the key, it returns the associated value. Otherwise, it returns a status indicating that the key is not stored in the tree.

[0063] When accessing the internal node 102I, the lookup may first fetch the first part of node 102 (e.g., the first 512 bytes) containing the node header 603 and the shortcut block 604. The lookup begins by searching the shortcut block 604 to find the last key below the target key. It then follows the pointer of the shortcut item to find the segment in the large block 601. The lookup fetches the large segment from memory and searches it to find the last key below the target key. Next (or concurrently), the lookup fetches the small block 602 from memory and searches it for the last key below the target key. If a back pointer is used, the lookup follows the pointer stored in the larger of the items 101 found in the small and large blocks without comparing keys. If the back pointer of the small block item points immediately after the large block item, it follows the small block pointer; otherwise, it follows the large block pointer. If the target key is smaller than both the first key in the large block and the first key in the small block, the lookup follows the leftmost pointer.

[0064] When a lookup reaches leaf 102I, it searches for it in the same way as an internal node. The main difference is that it searches for an exact match in both small and large blocks, meaning there is no need to order the items in the large and small blocks.

[0065] Small Block Search: Searching small blocks 602 may begin with sorting them before examining item 101. By sorting, the small block search can be stopped as soon as it encounters the first item with a key greater than the target key. In embodiments, sorting small blocks does not involve key comparison. Instead, it establishes ordering using an order hint field stored in each small block item. The order hint field stores the index of the item in the sorted small block when the item is inserted. These indices are "recreated" as the small blocks are scanned to rebuild the current sorted order. The established order is stored in a temporary indirect reference array (e.g., FPGA registers) without copying the items. The indirect reference array is used to access items in the sorted order. Sorting does not introduce significant latency, especially in hardware, and it can be performed in parallel while searching shortcut blocks 604 and large blocks 601. In software, the sorted order may be held in a small indirect reference array.

[0066] Figure 8 illustrates an exemplary insertion sequence into node 102. The numbers shown for each item 101 represent its key. Elements within the large block 601 are omitted for clarity. The numbers above the items represent their order hints. In step a), the small block 602 within the node is initially empty. In step b), an item with key 90 is inserted as an example, and since it is the first item in the small block, it is given an order hint of 0. Next, in step c), an item with key 60 is inserted. Since it is the smallest item in the small block, it is given an order hint of 0. The order hints of existing items are not changed. The order hints are determined from the sort order of the small block established before inserting new items. Next, in step d), an item with key 30 is inserted. Again, since it is the smallest in the small block, its order hint is also 0. In step e), the key of the last item is 45, and its order hint is 1.

[0067] Figure 9 illustrates the sorting of small block 602. This figure shows the state of the indirect reference array 901. The indirect reference array stores the offsets of items within the small block, but in the figure, for illustrative purposes, each offset is represented by its key.

[0068] To sort the small block 602, items 101 are processed in the order in which they were stored, and their order hints are used to sort the items in the indirect reference array. The indirect reference array 901 stores the offsets of the items in the small block. When an item with order hint i is processed, its offset in the small block is inserted into position i in the indirect reference array. All items at position j≧i are moved to the right. When the entire small block has been processed, the indirect reference array contains the offsets of the items in the sorted order. Figure 9 illustrates the indirect reference array sort of the small block shown in Figure 8. In one implementation, each step in the figure is completed in a single cycle on the FPGA, in parallel with key comparison. In the first step a), item 90 is inserted into the first slot. In the next step b), all items are moved to the right, and item 60 with order hint 0 is inserted into the first slot. In the next step d), all items are moved to the right again to make space for item 30 with order hint 0. In the final step d), items 60 and 90 are moved to the right, and item 45, which has order hint 1, is inserted into the slot with index 1. The final state e) shows the state of the indirect reference array after steps a) to d).

[0069] Figure 10 shows an exemplary implementation of an indirect reference array sorter. It shows a 4-item wide array. The sorter has a 9-bit wide register. The input to the register is either its output, the output of the register to its left, or an input offset value. The register input is selected by comparing the input index with the (constant) index of the register. If they are equal, the input offset is selected; if the input index is smaller than the register's index, the output of the register to its left is selected; and if the input index is larger, the register's output is written to itself. This exemplary implementation does not cover the case of having deleted items within a small block, but this is to allow the input to the register to also take the output of the register to its right.

[0070] Of course, the layout shown in Figure 10 is only one possible hardware implementation, and those skilled in the art will recognize that other designs can be made using conventional design tools given the disclosures herein. Software implementations on the read-side 401 are also possible.

[0071] The sorting implementation on CPU401 may follow a similar method. The indirect reference array is small and fits comfortably in the L1 cache, resulting in fast operation. The main difference is that moving items to the right within the array takes multiple cycles.

[0072] Range scan The range scan operation traverses tree 103 from the root as described above to find the start key of the target range. It then scans forward from the start key until it reaches the end key of the range or the end of the tree.

[0073] In some embodiments, to navigate between leaves 102L, the range scan may use sibling pointers. In such embodiments, each leaf 102L holds pointers to its left and right siblings, thereby enabling range scans in both directions. However, the use of sibling pointers is not mandatory. If sibling pointers are not used, the scan instead traverses the tree upwards to find the next leaf. In most cases, it only needs to look at the first parent and take the LID of the next sibling. However, if it has reached the end of the parent tree, it will go up to its parent to find its siblings. The scan may need to go all the way up to the root.

[0074] A range scan returns items 101 sorted by key. To maintain the sorted state of the items, the scan of leaf 102L may process large block 601 and small block 602 together. In a covering scan, the scan first finds the item with the largest key less than or equal to the range's starting position in both the large and small blocks. A "standard" covering scan is similar, except it starts with the smallest key greater than or equal to the range's starting position. In either case, the scan then scans the nodes forward in both the large and small blocks, searching for the end of the interval and returning the items within the interval. If an item with a key greater than the right boundary is found in both the large and small blocks, the scan stops. During the scan, if both the next small item and the next large item are within the interval, the search can use the back pointer of the small item to decide which to return next. If the small item points to the large item, the small item is returned and the search moves to the next small item; otherwise, the large item is returned and the search moves to the next large item. To handle shortcut key 604, the scan may record the end of the current large segment. In this case, at the beginning of the next segment, the key is taken from the shortcut block and the value from the large block. In the middle of the segment, the key and value are located within the large block.

[0075] Insertion An insert operation begins with a tree scan to find node 120L to insert the new item 101. This scan is similar to the scan during a lookup. The difference is that, in embodiments, it reads all items regardless of version in order to implement the semantics of insert and modify operations. Before updating a node, the writer 401 can lock it with a write lock, thereby ensuring that the node's state is the one observed during the scan (note that in embodiments, memory 403 can support different types of locks, namely write locks and read locks, and the use of a write lock does not necessarily imply the use of a read lock). For example, a write lock can be stored in the node header as a 32-bit lock word, which consists of a lock bit and a 31-bit node sequence number, which is incremented when the node is updated. The writer 401 can obtain a write lock by atomically setting the lock bit using an atomic compare-and-swap instruction on the lock word, while keeping the sequence number the same as during the scan. If the compare and swap is successful, this means the node has not changed since the traverse, so the insertion can proceed. If the compare and swap fails, the node has changed since the traverse, so the writer retries the operation by traversing the tree again.

[0076] Fast Path Insertion: In a common case, insertion does not require splitting node 102L or merging large block 601 and small block 602. This common case insertion can be performed in place (Figure 7). Writer 401 atomically appends the new item by first copying small block 602 after its end, and then resizing the small block to include the new item 101. To do this, writer 401 may perform the following steps after obtaining a write lock: it copies the item to the small block, updates the node size, increments the node's sequence number, and unlocks the node. In embodiments, since the lock word in the header and the size of the small block may share the same 64-bit word, writer 401 can increment the node version, update the node size, and unlock in a single instruction. Since the new item is stored beyond the end of the small block currently specified in header 603, a parallel reader does not observe the new item while it is being copied. They observe the node either without items or with fully written items.

[0077] In this embodiment, the reader 402 (e.g., FPGA) does not cache the leaf node 102L. If it did, the writer 401 would have to disable the cache 404 by issuing a command via PCIe 405, which would introduce additional overhead for common-case insertions.

[0078] Large-small merge: When small block 602 becomes too large, writer 401 merges large block 601 and small block 602 (see Figure 11). It allocates a new memory buffer for node 102L and sorts all items 101 in the node into the large block in the new buffer. In this embodiment, new items are added to the merged large block (not to a new small block). That is, the small block of the new node becomes empty after the merge. In principle, this is also possible in other ways (i.e., new items are the first items added to the new small block), but immediately storing new items in the large block slightly reduces the number of large-small merges over time.

[0079] In embodiments using shortcut keys 604, the writer 401 selects shortcut keys while sorting the items (it selects them at that point because the large blocks are immutable). For each item processed, the writer 401 decides whether to put it in a shortcut or a large block, for example, based on the number of bytes copied so far, the number of bytes remaining to be copied, and / or the average size of the key and value. It preferably maximizes the number of shortcut keys while keeping the large block segments of similar size.

[0080] Once all items have been copied to the new buffer, the writer 401 atomically replaces the LID mappings of the nodes in page table 501 with the addresses of the new buffer. To update the LID mappings, in this embodiment, the writer 401 updates the LID entries in both the CPU and FPGA copies of the page table. It obtains a software lock on the LID entries, issues a page table update command to the FPGA, and releases the software lock after the FPGA command is complete. Since the node's LID remains the same, there is no need to change the parent node. Finally, the writer places the old memory buffer in the garbage collection list, unlocks it, and sets a "deleted" flag in the header to ensure that parallel writers do not update the old buffer. An ongoing operation can still read from the old buffer, but if the operation needs to update a deleted node, it retries. Upon retrying, it uses the new node mappings.

[0081] The garbage collection used may be any suitable scheme, such as a standard explicit garbage collection for concurrent systems, e.g., RCU, an epoch-based memory manager, or a hazard pointer. Figure 11 shows an example of merging a large and small block during an insertion operation. When a new item is inserted into node 102L, the small block 602 exceeds its maximum size, so here the small block is merged into the large block 601 along with the new item. Step a) shows tree 103 before insertion into the rightmost node 102L. Step b) shows the allocation of a new buffer and merging into it. The old pointer (dashed line) to the node is inserted. In step c), the page table is updated to point to the new buffer, and the old buffer is put into garbage collection (GC) as it is no longer needed by the ongoing operation.

[0082] The advantage of this approach is that complex updates, such as merging, can be performed without the need to place a read lock on the old instance of the node at its original memory address (although for some operations, write locks may still be used, allowing one writer and one or more readers to access the node, only preventing multiple writers). While writer 401 is updating node 102L with a new buffer, if reader 402 attempts to read it before the page table is updated, reader 402 simply reads the old instance from the existing memory address currently specified in page table 501. Once the write to the new buffer (i.e., the new memory location) is complete, writer 401 can then quickly update page table 501 to switch each page table entry for the node's LID to the new memory address. In embodiments, this can be done in a single atomic operation. All subsequent reads then read the new instance of the node from the new address based on the updated page table entry. A similar approach can be used for other complex updates, such as splitting or merging nodes.

[0083] When a new memory buffer is allocated for the LID, its sequence number can be safely set to 0, as it is ensured that the buffer is unreachable by any operation before it is deleted and reused. To give some size examples, when large block 601 and small block 602 are merged, and the small block is larger than 512 bytes and the size of the small block entries is at least 10 bytes (delete entries are 10 bytes, and insert entries are at least 13 bytes), the maximum version of the buffer is 41, meaning the 31-bit sequence number never wraps around. In fact, in some implementations, the minimum size is actually 6 bytes on average, but even a 1-byte entry will not cause the sequence number to overflow.

[0084] Node splitting If there is not enough space in leaf 102L for insertion, writer 401 splits the leaf into two and inserts a pointer to the new item into the parent (Figure 12). If the parent is full, writer 401 also splits the parent. Splits can thus propagate down to the root. The internal node 102I that is updated but not split is called the root of the split. Writer replaces the entire subtree below the root of the split by updating the mapping of the root of the split in page table 501. Before updating the data, writer can obtain write locks on all nodes 102 that will be updated by the operation (these are locks to prevent interference by other writers). If the locks fail due to a node version mismatch, writer releases all acquired write locks and resumes the operation. Writer also allocates all memory buffers necessary to complete the operation. It allocates two new nodes (with new LIDs and new buffers) for each node it splits, and a memory buffer for the root of the split. If the system fails to allocate memory due to insufficient memory, the writer abandons the process and returns an appropriate status code to the caller. After acquiring the lock and allocating memory, the writer processes the nodes from the leaf upwards and splits them into newly allocated buffers. Each of the two resulting nodes ends up with approximately half the data of the original node. The writer merges the large and small blocks during the split. When it reaches the root of the split, the writer copies it into a new memory buffer. It modifies the new buffer to include pointers to the new child nodes and keys at the boundaries between the nodes. To swap the two subtrees, the writer updates the page table mapping of the split root with the address of the new memory buffer. Swapping the subtrees in this way ensures that the reader observes either the old or the new subtree. The writer then puts the memory buffers of all the split nodes, their LIDs, and the old memory buffer of the split root into the garbage collection list.Finally, it unlocks all memory buffers and marks them as deleted.

[0085] Figure 12 shows an example of splitting a node during insertion. Dashed boxes represent newly allocated node buffers. Filled boxes represent new nodes with new LIDs and memory buffers. Step a) shows tree 103 before splitting. The rightmost node is to be split. The root of the split is the root of the tree. In step b), two new nodes are allocated for the split, and a new buffer is allocated for the root of the split. In step c), the mapping of the root of the split is updated, and the node is placed in the garbage collection (GC) list.

[0086] In the embodiment, the writer also updates the sibling leaf pointer used during the scan operation. It locks the sibling leaf and updates the pointer after it swaps a new subtree. The reader observes a consistent state of the tree, even though the sibling pointer and the root of the split may not be updated atomically.

[0087] If the root of the tree cannot accommodate a new item, the writer splits it to create a new root, thereby increasing the height of the tree. The writer performs the split as described above, except that it allocates a LID and memory buffer for the new root instead of just a memory buffer for the split root. It initializes the new root with its leftmost pointer set to the left half of the old root, and a single item pointing to the right half of the old root. It may then send a command to the reader (e.g., FPGA) to update the root and tree height. In one exemplary implementation, the reader 402 records the LID and tree level number of the root node in its local register file, rather than in the cache 404. Data copies in the cache, for example, older root nodes, are eventually invalidated when their LID is reused for another physical node, or the old data is retired in some cases because it will no longer be accessed for a while. Therefore, the cache only needs to focus on maintaining the consistency of the cached node data.

[0088] delete Deleting item 101 is similar to inserting a new item. Writer 401 inserts a deletion entry into small block 602. Since this entry points to the item being deleted, the deleted item is ignored during the lookup operation. During the merging of large block 601 and small block 602, the space occupied by the deleted item is reclaimed. When node 102 becomes empty, it is deleted. Node deletion, like node splitting, can proceed upwards from leaf 102L to tree 103. In embodiments, the same technique as described above is used to ensure its atomicity.

[0089] Correction Modification operations can be performed as a combination of insertion and deletion. Writer 401 atomically publishes them by appending the deletion entry for the old item and the new item to the small block 602 and updating the node header 603.

[0090] conclusion It will be recognized that the embodiments described above are merely illustrative. More generally, according to one aspect disclosed herein, a system is provided comprising memory for storing a data structure, the data structure comprising a tree structure having a plurality of nodes, each having a node ID, some of which are leaf nodes and others are internal nodes, each internal node being the parent of each set of one or more children in the tree structure, each child being either a leaf node or another internal node, each leaf node being a child but not a parent, each leaf node comprising each set of one or more items, each having a key-value pair, and each internal node mapping each node ID of each child to a range of keys encompassed by each child. The system further comprises a writer positioned to write items to a leaf node and a reader positioned to read items from a leaf node. Each leaf node comprises its first block and its second block, the first block comprising a plurality of items of the leaf node sorted in key order in the address space of the memory. The writer, when writing a new item to the tree structure, identifies which leaf node to write to by traversing the tree structure and the mapping of keys to node IDs, and is configured to write the new item to the second block of the identified leaf node in the order it was written, rather than sorted by key order. The reader, when reading one or more target items from the tree structure, determines which leaf node to read from by traversing the tree structure and the mapping of keys to node IDs, and is then configured to look up the determined leaf node for one or more target items based on a) the order of items already sorted in the first block and b) the reader sorting the items in the second block by the keys associated with the items in the first block.

[0091] In embodiments, the writer may be implemented in software stored in the system's computer-readable storage and configured to run on one or more processors of the system. In embodiments, the reader may be implemented in a PGA, FPGA, or dedicated hardware network. Alternatively, the writer may be implemented in hardware and / or the reader may be implemented in software, or either in a combination of software and hardware.

[0092] In this embodiment, the tree structure can take the form of a B-tree (for example, a B+ tree where items are not stored in intermediate nodes).

[0093] In an embodiment, in at least some of the leaf nodes and / or internal nodes, each first block may be divided into segments, and the node may further comprise shortcut blocks with a plurality of shortcuts, each shortcut comprising i) an indication of a key at the boundary of the corresponding segment of each first block, and ii) an offset relative to the corresponding segment in the leaf node. In such an embodiment, when performing the above search of the first block, the reader may be configured to use shortcuts to search only for segments that may contain one or more items based on the key.

[0094] In one embodiment, the writer may be configured to include a back pointer associated with each item in each second block when writing to at least one of the leaf nodes, with each back pointer having an offset to an item in the first block having the next largest or next smallest key compared to the associated item in the second block. In such an embodiment, the reader may be configured to use the back pointers to sort the items in the second block by the keys associated with each first block when performing the sorting of the second block.

[0095] In one embodiment, the writer may be configured to include, when writing to at least one of the leaf nodes, an order hint for each item in the second block indicating the order of the item's key relative to the current key of the second block at the time the item is written to the second block. In such an embodiment, the reader may be configured to use the order hint to sort the items in the second block by the key associated with each of the first blocks when performing the sorting of the second block.

[0096] For example, the reader may use order hints to sort the items in the array by processing the items in each second block in the order in which they were stored, and by using the offsets of the items in the second block. When each item with an order hint i is processed, its offset in the second block is inserted into position i in the array, and after the entire second block has been processed, all items at position j≧i are moved to the right so that the array contains the offsets of the items in the sorted order.

[0097] In some embodiments, the data structure may include a page table that maps node IDs to their respective memory addresses. In this case, the reader is configured to use the page table to determine the memory address of the determined node in memory when reading from the node with the determined node ID. In some such embodiments, the writer may be configured to write the updated version of the node to the new memory address when updating a node to write, split, or merge the node, and once written, update the respective memory address in the page table to point to the new memory address.

[0098] In the embodiment, each second block may have a maximum size. The writer may be configured to merge the second blocks into the respective first blocks if it writes a new item that would cause each second block of an identified node to exceed its maximum size.

[0099] In embodiments where the data structure includes a page table that maps node IDs to their respective memory addresses, the writer may be configured to write the updated version of the node, including the merge, to the new memory address when updating the node to merge the second and first blocks, and to update the respective memory addresses in the page table to point to the new memory address once the merge is complete.

[0100] In the embodiment, each leaf node may further indicate the node IDs of one or more sibling leaf nodes that encompass a range of keys adjacent to the key of the respective leaf node. The reader may also be configured to perform a range scan to read items from a range of keys. In such an embodiment, when the reader performs a range scan to read items from a range of items spanning two or more leaf nodes, it may be configured to determine one of the leaf nodes that encompass one of the keys in the scanned range by traversing the tree structure to the mapping of keys to node IDs, and then to determine at least one other leaf node that encompasses at least one other key in the scanned range by using at least one ID of the sibling leaf node indicated in the aforementioned one of the leaf nodes.

[0101] Another aspect disclosed herein provides a method comprising the operation of a memory, writer, and / or reader according to any embodiment disclosed herein. Another aspect provides a computer program embodied on one or more non-temporary computer-readable media, comprising code configured to perform writer writes and / or reader reads when executed on one or more processors. Other variations or applications of the disclosed techniques may be apparent to those skilled in the art if the disclosure herein is given. The scope of this disclosure is not limited by the embodiments described herein, but is limited only by the appended claims.

Claims

1. A memory for storing a data structure, wherein the data structure comprises a tree structure having a plurality of nodes, each having a node ID, some nodes being leaf nodes and others being internal nodes, each internal node being the parent of each set of one or more children in the tree structure, each child being either a leaf node or another internal node, each leaf node being a child but not a parent, each of the leaf nodes comprising each set of one or more items having key-value pairs, and each of the internal nodes mapping each of the node IDs of each of its children to a range of keys encompassed by each of its children, A writer positioned to write items to the aforementioned leaf node, A reader positioned to read items from the aforementioned leaf node, A system equipped with, Each of the leaf nodes comprises a first block and a second block, the first block comprising a plurality of the items of each leaf node sorted in key order in the address space of the memory, The writer is configured to identify which leaf node to write to by tracing the mapping of keys to node IDs through the tree structure when writing a new item to the tree structure, and to write the new item to the second block of the identified leaf node in the order in which it was written, rather than sorted by the order of the keys. The reader is configured to determine which leaf node to read from the tree structure when reading one or more target items, by traversing the tree structure and the mapping of keys to node IDs, and then to search for the determined leaf node for the one or more target items based on a) the order of the items already sorted in the first block and b) the reader sorting the items in the second block by the keys associated with the items in the first block. system.

2. The system according to claim 1, wherein the writer is implemented in software that is stored in the computer-readable storage of the system and is configured to run on one or more processors of the system.

3. The system according to claim 1 or 2, wherein the leader is implemented in a PGA, FPGA, or dedicated hardware circuit network.

4. The system according to claim 1 or 2, wherein the tree structure takes the form of a B-tree.

5. The system according to claim 4, wherein the B-tree is a B+ tree in which items are not stored in intermediate nodes.

6. In at least a portion of the leaf nodes and / or internal nodes, each of the first blocks is divided into segments, and the node further comprises a shortcut block containing a plurality of shortcuts, each shortcut comprising i) an indication of the key at the boundary of the corresponding segment of each of the first blocks, and ii) an offset relative to the corresponding segment within the leaf node. The system according to claim 1 or 2, wherein the reader is configured to use the shortcut to search only for segments that may contain the one or more items based on a key when performing the search of the first block.

7. The writer is configured to include, when writing to at least one of the leaf nodes, a back pointer associated with each item in each of the second blocks at the time of writing, wherein each back pointer has an offset to an item in the first block having the next largest key or the next smallest key compared to the associated item in the second block. The system according to claim 1 or 2, wherein the reader is configured to use the back pointer to sort the items of the second block by a key associated with each of the first blocks when sorting the second block.

8. The writer is configured to include, when writing to at least one of the leaf nodes, an order hint for each of the items in the second block, indicating the order of the item's key relative to the current key of the second block at the time the item is written to the second block. The system according to claim 1 or 2, wherein the leader is configured to use the ordering hints to sort the items of the second block by keys associated with each of the first blocks when sorting the second block.

9. The aforementioned leader, - Processing the items in each of the second blocks in the order in which they were stored, - Using the order hint to sort the items in the array, wherein the array stores the offset of the items in the second block, The system is configured to use the aforementioned ordering hints, The system according to claim 8, wherein as each item having an order hint i is processed, its offset in the second block is inserted into its position i in the array, and after the entire second block has been processed, all of the items at position j ≥ i are moved to the right so that the array contains the offset of the items in the sorted order.

10. The data structure comprises a page table that maps the node ID to its respective memory address, The reader is configured to use the page table to determine the memory address of the determined node in memory when reading from the node with the determined node ID. The system according to claim 1 or 2, wherein when the writer updates the node in order to write, split, or merge the node, it writes the updated version of the node to a new memory address, and once written, updates each of the memory addresses in the page table to point to the new memory address.

11. Each second block has the maximum size, The system according to claim 1 or 2, wherein the writer is configured to merge the second blocks into the first blocks when the writer writes a new item that causes each of the second blocks of the identified node to exceed the maximum size.

12. The data structure comprises a page table that maps the node ID to its respective memory address, The reader is configured to use the page table to determine the memory address of the determined node in memory when reading from the node with the determined node ID. The system according to claim 11, wherein when the writer updates the node to merge the respective second and first blocks, it writes the updated version of the node, including the merge, to a new memory address, and once the merge is complete, it updates the respective memory addresses in the page table to point to the new memory address.

13. Each leaf node further indicates the node ID of one or more sibling leaf nodes that encompass the range of keys adjacent to the key of each leaf node, The reader is configured to perform a range scan to read items from a range of keys, and when performing a range scan to read items from a range spanning two or more leaf nodes, - Determining one of the leaf nodes that contains one of the keys in the scanned range by traversing the mapping of keys to node IDs through the tree structure, - Determining at least one other leaf node that includes at least one other key in the scanned range by using the ID of at least one of the sibling leaf nodes indicated in one of the leaf nodes, The system according to claim 1 or 2, configured to perform the following:

14. A computer, Storing a data structure in memory, wherein the data structure comprises a tree structure having a plurality of nodes, each having a node ID, some nodes being leaf nodes and others being internal nodes, each internal node being the parent of each set of one or more children in the tree structure, each child being either a leaf node or another internal node, each leaf node being a child but not a parent, each of the leaf nodes comprising each set of one or more items having key-value pairs, and each of the internal nodes mapping each of the node IDs of each of its children to a range of keys encompassed by each of its children. Writing an item to the aforementioned leaf node, Reading items from the aforementioned leaf node, A method comprising, Each of the leaf nodes comprises a first block and a second block, the first block comprising a plurality of the items of each leaf node sorted in key order in the address space of the memory, The writing process includes, when writing a new item to the tree structure, identifying which leaf node to write to by traversing the mapping of keys to node IDs through the tree structure, and writing the new item to the second block of the identified leaf node in the order it was written, rather than sorting it by key order. A method comprising: determining which leaf node to read from when reading one or more target items from the tree structure by traversing the mapping of keys to node IDs through the tree structure; and then searching for the determined leaf node for the one or more target items based on a) the order of the items already sorted in the first block and b) the reader sorting the items in the second block by the keys associated with the items in the first block.

15. A computer program embodied on one or more non-temporary computer-readable media, comprising code configured to perform the write and / or read operations described in claim 14 when executed on one or more processors.

Images