Method, computer readable medium and computer system for implementing a trie data structure with a sub-trie tree data structure

Abstract

This application discloses techniques related to tree data structures capable of storing information indicative of database key codes. A computer system can operate a database. The computer system can store a multi-level tree data structure capable of being used to perform key code lookups for the database. In various cases, the multi-level tree data structure can be stored in system memory as a plurality of sub-tree data structures, each sub-tree data structure comprising a set of linked nodes. A given one of the plurality of sub-tree data structures can be stored in system memory as a respective contiguous block of information. The computer system can access the respective contiguous block of a first particular sub-tree data structure encompassing a particular range of levels in the multi-level tree data structure. The access can be performed without accessing one or more other sub-tree data structures encompassing one or more levels within the particular range of levels.

Description

Technical Field

[0001] The publicly available text generally relates to database systems, and more specifically, to tree-like data structures capable of storing information indicating database keys. Background Technology

[0002] Modern database systems typically implement management systems that enable users to store and efficiently access and manipulate sets of information in an organized manner. In some cases, these management systems maintain a Log Merge Tree (LSM) with multiple levels, where each level stores information from database records as key-value pairs. An LSM typically comprises two high-level components: a memory buffer and persistent storage. In operation, the database system first writes database records to the memory buffer and then flushes them to persistent storage. As part of flushing the database records, the database system writes the records to a new file stored on persistent storage. Attached Figure Description

[0003] Figure 1 This is a block diagram illustrating exemplary elements of a system capable of using a collection of trie data structures according to some implementation schemes.

[0004] Figure 2 This is a block diagram illustrating exemplary elements of a trie data structure having a set of subtrie data structures according to some embodiments.

[0005] Figure 3 This is a block diagram illustrating exemplary elements of pointer information for nodes in a trie data structure according to some implementations.

[0006] Figure 4 This is a block diagram illustrating exemplary elements of a trie created and updated by a database application according to some implementation schemes.

[0007] Figures 5 to 6 This is a flowchart illustrating an exemplary method related to the use of a trie data structure according to some implementation schemes.

[0008] Figure 7 This is a block diagram illustrating an exemplary computer system according to some implementation schemes.

[0009] The published text includes references to “one implementation” or “implementation.” The appearance of the phrase “in one implementation” or “in an implementation” does not necessarily refer to the same implementation. Specific features, structures, or characteristics may be combined in any suitable manner consistent with the published text.

[0010] In public texts, different entities (which may be referred to differently as “units,” “circuits,” other components, etc.) may be described or referred to as being “configured” to perform one or more tasks or operations. The expression—[entity] configured to [perform one or more tasks]—is used here to refer to a structure (i.e., a physical thing, such as an electronic circuit). More specifically, this expression is used to indicate that the structure is arranged to perform one or more tasks during operation. Even if the structure is not currently being operated, it can be said that the structure is “configured” to perform certain tasks. “A network interface configured to communicate over a network” is intended to cover, for example, an integrated circuit having circuitry that performs this function during operation, even if the integrated circuit in question is not currently in use (e.g., a power supply is not connected to the integrated circuit). Therefore, an entity described or stated as being “configured” to perform a task refers to a physical thing, such as a device, circuit, memory storing program instructions executable to perform that task, etc. Therefore, the expression “configured to” is not used here to refer to software entities such as application programming interfaces (APIs).

[0011] The term “configured to” does not mean “configurable to”. For example, an unprogrammed FPGA is not considered “configured to” perform a specific function, although it can be “configurable to” perform that function and can be “configured to” perform that function after programming.

[0012] As used herein, the terms “first,” “second,” etc., used as labels for the nouns preceding them do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) unless otherwise specified. For example, in a processor with eight processing cores, the terms “first” and “second” processing cores can be used to refer to any two of the eight processing cores. In other words, for example, the first and second processing cores are not limited to processing cores 0 and 1.

[0013] As used herein, the term "based on" is used to describe one or more factors that influence the determination. This term does not exclude other factors that may influence the determination. That is, the determination may be based solely on the specified factor or based on the specified factor along with other unspecified factors. Consider the phrase "A is determined based on B." This phrase specifies that B is a factor used to determine A or influence the determination of A. This phrase does not exclude that the determination of A may also be based on other factors, such as C. This phrase is also intended to cover implementations that determine A based solely on B. As used herein, the phrase "based on" is therefore synonymous with the phrase "at least partially based on."

[0014] As used herein, a “module” refers to software and / or hardware operable to perform a specified set of operations. A module can refer to a set of software instructions executable by a computer system to perform the set of operations. A module can also refer to hardware configured to perform the set of operations. Hardware modules can constitute general-purpose hardware and non-transitory computer-readable media storing program instructions, or special-purpose hardware such as a custom ASIC. Therefore, a module described as “executable” to perform operations is a software module, while a module described as “configured” to perform operations is a hardware module. A module described as “operable” to perform operations refers to both software and hardware modules. Detailed Implementation

[0015] During operations of a database built around an LSM tree, the database system typically flushes database records from a memory buffer to files stored on persistent storage. These files usually contain multiple database records corresponding to various database key values. In operation, the database system processes transaction requests, which may involve accessing database records from files in persistent storage for database keys specified in those requests. In some cases, as part of processing transaction requests, the database system performs a single database record lookup, retrieving files from persistent storage and examining them for database records corresponding to the specific database keys identified in the transaction request. However, in many cases, the files retrieved by the database system do not contain database records corresponding to a specific database key. Therefore, the database system wastes time and resources retrieving those files from persistent storage.

[0016] In some cases, probabilistic data structures such as Bloom filters can be used to reduce the number of files fetched from persistent storage because they often provide indications of whether a corresponding database record is likely to exist in a file or definitely not in that file, based on the database key. Therefore, the database system can fetch only those files where the corresponding database record is likely to exist, skipping those where the corresponding database record is not present. However, in other cases, as part of processing a transaction request, the database system performs a key range lookup, where the database system searches for database records whose database keys fall within a specified key range. A specific type of probabilistic data structure called a "tribe" (or alternatively, a "tribe data structure") can be used in key range lookups to reduce the number of files fetched from persistent storage. Tribes can also be used to determine whether a particular database node includes database records whose database keys fall within a specified key range in its memory buffer. As used herein, the term "tribe" is used according to its established meaning and refers to a tree-like data structure whose branches consist of linked nodes corresponding to character values. Thus, a branch of a trie can represent a database key, where each node of the branch corresponds to a character of the database key.

[0017] While trie offers advantages over other probabilistic data structures (such as Bloom filters) in key-range lookups, current trie implementations have drawbacks. One drawback involves the way the trie is stored in system memory and then fetched from system memory into the system cache. Currently, these implementations result in a relatively large number of memory fetches when traversing branches of the trie. In particular, current implementations store the trie in system memory such that all nodes at a given level are in memory before all nodes at the next level in the trie. Therefore, as part of branch traversal, traversing down from one level to the next may involve moving a reasonable amount of memory within system memory. Imagine an instance where a computer system traverses from level 27 to level 28 of a trie. Since level 27 may contain a large number of nodes and a limited cache size, when the computer system issues a memory request to fetch data from system memory for level 27 into the cache, the data returned from system memory may not include data for the nodes at level 28. That is, a sequence of data blocks retrieved from system memory and stored in the system cache might only include data representing a portion of the nodes at level 27, excluding data representing the nodes at level 28. Therefore, when traversing down from level 27 to level 28 of the trie, the computer system will have to issue another memory request to retrieve data for level 28. This method of storing the trie in system memory results in the system having to issue a memory request for each node in the branch being extended downwards. Therefore, extending the branch (which is typically multiple levels of a data structure) involves a large number of memory requests, which, combined, can potentially take excessive time to process.

[0018] The publicly available text describes techniques for implementing trie data structures that overcome some or all of the drawbacks of other trie methods. In the various embodiments described below, a computer system maintains a trie data structure comprising sets of sub-tries, each sub-trie data structure having a set of linked nodes. In various cases, a first sub-trie may be linked to a second sub-trie data structure that is part of a specific branch of the trie data structure. In various embodiments, the first sub-trie data structure is stored in system memory as a contiguous block of information, including node information detailing the nodes of the first sub-trie data structure and pointer information identifying the location in system memory where information about the second sub-trie data structure is stored. The computer system can use the pointer information included in the first sub-trie data structure to load the second sub-trie data structure into the computer system's system cache. In various cases, the second sub-trie data structure and other sub-trie data structures may cover nodes within the same specific level range of the trie data structure. In various implementations, the computer system loads a second sub-tribe data structure without accessing one or more other sub-tribe data structures that cover nodes within one or more levels of the same specific hierarchical range as the second sub-tribe data structure.

[0019] Consider the preceding example, where the computer system traverses down to levels 27 and 28 of a specific branch. For instance, a trie data structure implemented using the techniques discussed in the publicly available text could include a first sub-trie data structure and a second sub-trie data structure. The first sub-trie data structure includes nodes from level 26 (one of which may correspond to a specific branch), and the second sub-trie data structure includes nodes from level 27 to level 34 of the specific branch. The second sub-trie data structure can be entirely stored in the system cache, including nodes from level 27 to level 34, since it is not necessary to include all nodes at these levels. As a result, when the computer system retrieves the second sub-trie data structure into the system cache using pointer information included in the first sub-trie data structure, the computer system can traverse from level 27 to level 34 of the specific branch without issuing another memory request. Therefore, these techniques can be superior to existing methods because they allow the computer system to issue fewer memory requests when traversing branches of the trie data structure. That is, when many nodes at a specific level are not related to the traversal of a specific branch of the trie, existing methods waste cache space to retrieve most or all of these nodes. Now, refer to... Figure 1 Let's begin by discussing exemplary applications of these technologies.

[0020] Now go to Figure 1 A block diagram of system 100 is shown. System 100 includes a collection of components that can be implemented by hardware or a combination of hardware and software routines. In the illustrated embodiment, system 100 includes a database 110 and database nodes 130, the database nodes including a computing integration device 140 and system memory 150. As further shown, database 110 includes files 115 and a trie 120, while computing integration device 140 executes a set of program instructions to implement a database application 146 and includes a system cache 144. In some embodiments, system 100 is implemented in a different manner than that shown. For example, system 100 may include multiple database nodes 130 that interact with each other and with database 110.

[0021] In various implementations, system 100 implements a platform service (e.g., a customer relationship management (CRM) platform service) that allows users of the service to develop, run, and manage applications. System 100 may be a multi-tenant system providing various functionalities to multiple users / tenants hosted by a multi-tenant system. Therefore, system 100 can execute software routines from various different users (e.g., providers and tenants of system 100), and provide code, web pages, and other data to users, databases, and other entities associated with system 100. For example, as shown, system 100 includes a database node 130 that can store and access data from file 115 of database 110 on behalf of users associated with system 100.

[0022] In various implementations, database 110 is a collection of information organized in a manner that allows access, storage, and manipulation of information. Therefore, database 110 may include supporting software that allows database nodes 130 to manipulate (e.g., access, store, etc.) the information stored in database 110. In some implementations, database 110 is implemented as a single storage device or multiple storage devices connected together on a network (e.g., a storage attached network (SAN)) and configured to redundantly store information to prevent data loss. Since storage devices can persistently store data, database 110 can be used as persistent storage. In various implementations, database 110 is shared among multiple database nodes 130, such that database records written to file 115 by one database node 130 are accessible by other database nodes 130.

[0023] In various implementations, database 110 implements a portion of a Log Structure Merge Tree (LSM tree), which has multiple levels of files 115 including database records. The LSM tree may contain two high-level sections: a memory section and a disk section. One or more “top” levels of the LSM tree may contain database records written to the memory buffer of database node 130. The remaining levels of the LSM tree may contain files 115, which include database records written to database 110. In various cases, system memory 150 and system cache 144 may facilitate the memory section of the LSM tree, while database 110 facilitates the disk section.

[0024] In various implementations, file 115 is a collection of database records. A database record can be a key-value pair containing data and a corresponding database key that can be used to look up the database record. For example, a database record can correspond to a data row in a database table, where the database record specifies the value of one or more attributes associated with the database table. In various implementations, file 115 is associated with one or more ranges of database key codes defined by the key codes of the database records included in file 115. Imagine that file 115 stores instances of three database records associated with the key codes “AA”, “AB”, and “AC”, respectively. Those three key codes span the database key code ranges AA to AC, so file 115 can be associated with the database key code ranges AA to AC. As discussed herein, database node 130 can use trie 120 to determine whether file 115 has a database record associated with a database key code that falls within a specified range of database key codes.

[0025] In various implementations, the trie 120 is a trie data structure containing a set of subtrie data structures, each subtrie data structure including a set of link nodes. Figure 2 An example of a trie 120 is shown. Trie 120 can be a probabilistic data structure capable of providing indications of a range of database keys associated with one or more files 115. As used herein, the term "probabilistic data structure" refers to a data structure that maintains information indicating that a particular item does not exist or may exist at a specific location within the system. As an example, a probabilistic data structure may store information indicating that a database record does not exist or may exist within a file. In various embodiments, trie 120 has a much smaller memory footprint than the corresponding file 115. Therefore, database node 130 can access trie 120 faster and more efficiently than accessing the corresponding file 115. Therefore, using trie 120 to check whether certain database records might be included in a given file 115, rather than directly accessing the given file 115 to check database records, can provide a significant performance improvement for system 100.

[0026] In various implementations, when a database record is written to file 115, database node 130 generates a trie 120 for file 115. Trie 120 can form a tree data structure, with branches corresponding to the database keys of the database records written to the respective file 115. Thus, branches can contain nodes representing the character values ​​of the corresponding database keys. In various implementations, when database node 130 wants to determine whether file 115 might contain a certain database record, database node 130 can perform a single key lookup, which involves searching the corresponding trie 120 for branches whose nodes match the character values ​​of the database key of the database record. Database node 130 can also perform a key range lookup, which involves searching the corresponding trie 120 for any branch whose aggregate node falls within a specified key range. If such a branch can be found in trie 120, database node 130 can determine that at least one database record falling within the specified key range may exist in the corresponding file 115.

[0027] In various implementations, system 100 is implemented using multiple database nodes 130, which first write database records to their own memory buffers before flushing them to database 110. During operation, database node 130 may wish to access the latest version of a data row as part of processing a database transaction. However, this latest version may be stored in the memory buffer of another database node 130. Therefore, a preceding database node 130 may issue a database record request to a subsequent database node 130 to determine whether the latest version is stored at the subsequent database node 130. In many cases, it is not stored at the subsequent database node 130, resulting in wasted resources. Therefore, in various implementations, the trie 120 can also be used to determine whether a database record might exist in the memory buffer of another database node 130, and thus whether a database record request should be issued.

[0028] In various implementations, database node 130 is hardware, software, or a combination thereof capable of providing database services such as data storage, data retrieval, and / or data manipulation. Such database services can be provided to other components within system 100 and / or components outside system 100. As an example, database node 130 may receive database transaction requests from an application server (not shown) that requests to write data to or read data from database 110. The database transaction request may specify an SQL select command to select one or more rows from one or more database tables. The content of the rows can be qualified in the database records, so database node 130 can locate and return one or more database records corresponding to the selected one or more table rows. In some cases, the database transaction request may instruct database node 130 to write one or more database records against an LSM tree. In various implementations, database node 130 first writes the database records to system cache 144 and system memory 150 before flushing them to database 110.

[0029] In various embodiments, computing integration device 140 is an aggregate of processors, processor cores, and / or caches 144. As an example, computing integration device 140 may include four processor cores having their own internal system caches 144. The processor cores of computing integration device 140 may be configured to execute instructions of a specific instruction set architecture (ISA), which may include operating system instructions and user application instructions (e.g., instructions executable to implement database application 146). These instructions may be stored in a non-transitory computer-readable medium, such as system memory 150 linked to computing integration device 140. System memory 150 may be volatile memory (e.g., dynamic random access memory) or non-volatile memory (e.g., NAND flash memory). Computing integration device 140 may retrieve a set of program instructions from system memory 150 and execute the program instructions to perform the functions described herein, such as database application 146.

[0030] In various implementations, database application 146 is an executable set of program instructions for managing database 110, including managing the LSM tree built around database 110. Therefore, database application 146 can process database transactions to read records from and write records to the LSM tree. Database application 146 can maintain metadata describing the structural layout of the LSM tree, including where file 115 is stored in database 110 and which records can be included in those files 115. In various implementations, the metadata includes a trie 120 corresponding to file 115 stored at database 110. As discussed, database application 146 can use the metadata to perform faster and more efficient key range lookups as part of processing database transactions.

[0031] To facilitate database transaction processing, in various implementations, database node 130 uses a trie 120. For example, as discussed, when searching for database records falling within the range of database keys, database node 130 can use trie 120 to identify files 115 and / or database nodes 130 that may contain such database records. Therefore, database node 130 may first retrieve one or more tries 120 from database 110 and then store them in system memory 150. In some implementations, when database node 130 wishes to examine a specific file 115 and / or other database node 130, database node 130 retrieves the corresponding trie 120 at this time and stores it in system memory 150. Once the trie 120 is stored in system memory 150, database node 130 can begin traversing the trie 120.

[0032] When traversing the trie 120, the database node 130 can issue a memory fetch 148 to the system memory 150 to load a sub-trie 125 of the trie 120 into the system cache 144. The sub-trie includes the root node of the entire trie 120. As described above, the trie 120 contains a collection of sub-tries 125. In various embodiments, a sub-trie 125 is a tree-like data structure having one or more branches consisting of linked nodes corresponding to character values. For example, a branch of a sub-trie 125 may correspond to a portion or all of a particular database key—the remainder of which may correspond to branches of other sub-tries 125 linked to a preceding sub-trie 125. When stored in the system cache 144 and the system memory 150, the sub-trie 125 can be stored in the memory address space as a contiguous block of information / data (bytes). However, in some cases, the trie 120 may not be stored as a contiguous block of information; that is, the individual sub-tries 125 may be discontinuous in memory. However, in some cases, the trie 120 can be stored in memory as a contiguous block of information. In various implementations, the storage size of the sub-trie 125 does not exceed the storage space of the system cache 144. Therefore, the database node 130 can load the entire sub-trie 125 into the system cache 144. However, the trie 120 can be larger than the storage space of the system cache 144, and therefore cannot be loaded entirely into the system cache 144 by the database node 130. (See reference...) Figure 2 An instance of subtribette 125 will be discussed.

[0033] After sub-tribe 125 has been loaded into system cache 144, database node 130 can then begin traversing branches of sub-tribe 125 to determine whether these branches correspond to database keys falling within a specific range of database keys. In various cases, while traversing a given branch of the first sub-tribe 125, database node 130 can reach a node in that branch that links to another node included in the second sub-tribe 125. Database node 130 can use pointer information included in the first sub-tribe 125 to load the second sub-tribe 125 into system cache 144; the pointer information identifies the storage location of the second sub-tribe 125 in system memory 150. Database node 130 can then continue traversing the branch via the second sub-tribe 125 stored in system cache 144. In this way, database node 130 can traverse branches that extend across multiple sub-tribes 125. In various implementations, when database node 130 accesses a sub-tribe 125 spanning a level range, database node 130 accesses a subset of all nodes within those levels, with the remaining nodes encompassed in other sub-tribes 125 spanning that level range. By not having to access all nodes within a level as part of traversing a particular branch, database node 130 can use the storage space that would otherwise be occupied by nodes unrelated to a particular branch to access more nodes in lower levels of the trie 120 that are relevant to the traversal of that particular branch.

[0034] Now go to Figure 2 A block diagram illustrating exemplary elements of a trie 120 and its layout within system memory 150 is shown. In the illustrated embodiment, trie 120 includes sub-tries 125A-D, each sub-trie having a set of nodes 210. As shown, sub-trie 125A includes nodes 210A, 210B, 210E, 210F, and 210K; sub-trie 125B includes nodes 210C and 210G; sub-trie 125C includes nodes 210D, 210H, 210I, 210J, and 210L; and sub-trie 125D includes node 210M. As further shown, the information representing sub-tries 125A-125D is stored contiguously in system memory 150. In some embodiments, trie 120 and / or its layout within system memory 150 may be implemented differently than those shown. As an example, trie 120 may be stored non-contiguously in system memory 150.

[0035] As described above, in various embodiments, the trie 120 is used to store database keys that have been written to database records in the file 115 corresponding to the trie 120. As shown, the trie 120 of the illustrated embodiment stores at least the keys “AM”, “ART”, “OX”, “TA”, “TIP”, and “TOP”. Each database key can be stored in the trie 120 as a set of link nodes 210 forming branches of the trie 120. For example, the key “TIP” is stored as nodes 210D, 210I, and 210L of the branch extension levels 215A-D of the trie 120. (See also: Regarding...) Figure 3 In more detail, in various implementations, node 210 is a data structure that stores information related to the character value it represents and how node 210 is linked to other nodes 210.

[0036] When performing a key range lookup (or a single key lookup), database node 130 can traverse levels 215 of the trie 120 by issuing a memory fetch 148 to load the sub-trie 125 from system memory 150 into system cache 144. Imagine that database node 130 wants to determine whether a file 115 corresponding to trie 120 might contain an instance of a database record falling within the key range "TCP" to "TLC". In various embodiments, database node 130 first issues a memory fetch 148 to system memory 150 to load the sub-trie 125, which includes node 210 representing the root of trie 120. In the illustrated embodiment, node 210A stores the root character value, and then the sub-trie 125A is fetched into system cache 144. As shown, five nodes 210 are fetched—this is likely because system cache 144 may have a capacity to store only the information of five nodes 210 at a time.

[0037] Once sub-tribe 125A is loaded into system cache 144, database node 130 can determine that node 210A is linked to three nodes: nodes 210B, 210C, and 210D. Database node 130 can further determine that nodes 210B and 210C represent character values ​​outside the key code range "TCP" to "TLC", while the character value "T" of node 210D falls within this range. Thus, database node 130 can traverse the branch that includes node 210D. Since sub-tribe 125A does not include node 210D, database node 130 can issue another memory fetch 148 to load sub-tribe 125C into system cache. In various embodiments, because node 210A is linked to node 210, which is stored as part of other sub-tribe 125, node 210A includes pointers 220A and 220B indicating the storage location of other sub-tribe 125 in system memory 150. Therefore, using pointer 220A, database node 130 can load sub-tribe 125C into system cache 144 without loading other sub-tribes 125.

[0038] Once the sub-tribe 125C is loaded into the system cache 144, the database node 130 can traverse the branches and determine the linked nodes 210D, 210I, and 210L to form keys falling within the key range "TCP" to "TLC". The corresponding file 115 can then store the database records falling within this key range. In various cases, the database node 130 can perform key lookups down through multiple levels 215, but it does not access all nodes 210 for those down-levels 215. In the previous example, the database node 130 did not fetch information into the system cache 144 for nodes 210C, 210G, and 215M. By not accessing all nodes 210 at the down-levels 215 in the key lookup, the database node 130 can reduce the number of memory fetches 148 issued, thereby saving time and resources.

[0039] Now go to Figure 3 A block diagram illustrating exemplary elements of nodes 210 included in sub-tribes 125 within trie 120 is shown. In the illustrated embodiment, sub-tribe 125A includes node 210A, and sub-tribe 125B includes node 210B. As shown, node 210A includes node information 310 and pointer information 320 indicating the location of node 210B. As further shown, pointer information 320 includes a pointer 220 and a storage size indicator 325. In some embodiments, node 210 is implemented in a manner different from that shown. As an example, pointer information 320 may not include the storage size indicator 325.

[0040] In various embodiments, node information 310 includes information indicating the character value represented by the corresponding node 210. Node information 310 may further indicate nodes 210 connected to the node 210 of node information 310. For example, node information 310 of node 210A may indicate that node 210A is linked to node 210C, and that node 210C is sequentially following node 210A. As another example, node information 310 of node 210A may indicate that node 210A extends downwards from another specific node 210. In some embodiments, node information 310 indicates at which level of the trie 120 the node 210 of node information 310 exists. For example, node information 310 may indicate that node 210A is at the second level downwards from the top of the trie 120.

[0041] In various embodiments, pointer information 320 includes information that can be used to access one or more nodes 210 linked to the node 210 of pointer information 320. In various cases, pointer information 320 may include pointers (e.g., storage offsets) to nodes 210 existing in the same sub-tribe 125 as the node 210 of pointer information 320. For example, nodes 210A and 210B (not shown) may be directly linked and included in a particular sub-tribe 125. Pointer information 320 for node 210A may indicate the location in a data structure of node 210B from which it can be accessed within the particular sub-tribe 125. In various cases, pointer information may include pointers (e.g., storage offsets) to nodes 210 existing in a different sub-tribe 125 than the node 210 of pointer information 320. As shown in the figure, the pointer information 320 of node 210A includes a pointer 220 that identifies the location of node 210C (e.g., a storage address in system memory 150) — that is, pointer 220 can identify the location in system memory 150 that includes information / data blocks describing the node 210 it points to (e.g., bytes defining node information 310 and pointer information 320).

[0042] Therefore, when traversing a branch from node 210A to node 210C, database node 130 can use pointer information 320 of node 210A to load sub-tribe 125B (which includes node 210C) from system memory 150 into system cache 144. In various embodiments, pointer information 320 includes a storage size indicator 325, which indicates the size (e.g., 64 KB) of the sub-tribe 125 including the pointed-to node 210. Therefore, when attempting to introduce sub-tribe 125B into system cache 144, database node 130 can determine how much data to request from system memory 150 to load sub-tribe 125B based on the size of sub-tribe 125B indicated by the storage size indicator 325 of pointer information 320 of node 210A. As an example, storage size indicator 325 may specify the number of cache lines consumed when sub-tribe 125 is stored in system cache 144. Therefore, database node 130 can retrieve the data valued at the cached row number from system memory 150. By using storage size indicator 325, database node 130 can avoid retrieving more data from system memory 150 than is required to load sub-tribe 125.

[0043] Now go to Figure 4 The diagram illustrates a block diagram of a database application 146 that writes files 115 and a trie 120 to a database 110. As further shown, the database application 146 receives a set of transaction requests 410 with keys 415. Transaction requests 410 may specify a set of database records (and their corresponding keys 415) to be written to the database 110 as files 115. Therefore, the database application 146 can process transaction requests 410 to write one or more files 115 to the database 110. As part of processing transaction requests 410, the database application 146 may generate a corresponding trie 120 for one or more files 115, the trie 120 including the keys 415 of the database records included in those one or more files 115. In other cases, as part of processing transaction requests 410, the database application 146 may generate the set of tries 120 based on the keys of the database records stored in a memory buffer of the database node 130 (which may be implemented using system cache 144 and system memory 150).

[0044] When generating the trie 120, the database application 146 can insert key 415 into the trie 120 as a branch of the trie tree structure. In various cases, key 415 can share a common prefix and therefore share common parts of the branches. For example, as Figure 2As shown, the keys “AM” and “ART” share a common node 210 (i.e., node 210B) in their respective branches. As part of generating the trie 120, in various embodiments, the database application 146 determines how to partition the nodes 210 of the trie 120 into sub-tries 125. The database application 146 may use various criteria to determine how to partition the nodes 210. One such criterion may include ensuring a ratio between the number of nodes 210 in the sub-trie 125 and the number of levels 215 in the sub-trie 125. For example, the database application 146 may ensure that for every four nodes, the sub-trie 125 extends down at least one additional level. Thus, if the sub-trie 125 comprises nine nodes 210 in this instance, then the sub-trie 125 comprises three levels 215. Another criterion may include ensuring a ratio between the number of nodes 210 in the trie 120 and the number of sub-tries 125. For example, if there are fifty nodes 210 in the trie 120, the database application 146 can partition these nodes 210 into five sub-tries 125, such that each sub-trie 125 has ten nodes 210. Another criterion may include ensuring that the average number of memory fetches 148 required to access the entire given branch of the trie 120 is minimized. In various implementations, after determining the partitioning of nodes 210, the database application 146 constructs each group of nodes 210 into corresponding contiguous blocks of data.

[0045] In some cases, database application 146 may update trie 120. For example, if a trie 120 is being used to determine whether a record for a specified key 415 is stored at another database node 130, and the nature of the record stored at the other database node 130 changes over time, trie 120 may be updated to reflect these changes. That is, if another database node 130 updates its memory buffer to include new database records, trie 120 used by database application 146 may be updated to incorporate the key 415 of that database record. In various cases, when updating trie 120, database application 146 adds one or more nodes 210 to trie 120. When adding one or more nodes 210, database application 146 may modify a particular subtrie 125 to include the added one or more nodes 210. This may allow a particular subtrie 125 to span one or more additional levels 215. In some cases, database application 146 may create new subtries 125 to include one or more added nodes 210. In other cases, database application 146 may divide a particular sub-tribe 125 into multiple sub-tribes 125, one of which contains one or more additional nodes 210.

[0046] Now go to Figure 5 A flowchart of method 500 is shown. Method 500 is an embodiment of a method executed by a computer system (e.g., database node 130) to facilitate the processing of a transaction request (e.g., transaction request 410). Method 500 can be performed by executing one or more program instructions stored on a non-transient computer-readable medium. In some embodiments, method 500 may include more or fewer steps than shown; for example, method 500 may include the step of a computer system receiving a transaction request.

[0047] Method 500 begins at step 510, whereby the computer system operates a database (such as database 110). In various embodiments, operating the database includes the computer system receiving a transaction request to execute a transaction that includes writing to the database a file (e.g., file 115) containing a set of records associated with a set of keys (e.g., a set of keys 415). The computer system can execute the transaction, which may include generating a multi-level tree data structure (e.g., a trie 120) such that nodes included in the multi-level tree data structure (e.g., node 210) correspond to characters in the set of keys. The multi-level tree data structure may be associated with a file.

[0048] In step 520, the computer system stores a multi-level tree data structure that can be used to perform key lookups against the database. In various cases, the multi-level tree data structure can be stored in system memory (e.g., system memory 150) as multiple subtree data structures (e.g., sub-tribe 125), each subtree data structure including a set of linked nodes. A given subtree data structure can be stored in system memory as a corresponding contiguous block of information. The storage size of the corresponding contiguous block of a given subtree data structure can not exceed the storage size of a system cache (e.g., system cache 144) associated with system memory. The multi-level tree data structure can be stored in system memory as a contiguous block of information.

[0049] In step 530, the computer system accesses a corresponding contiguous block of a first specific subtree data structure (e.g., sub-tribe 125A) that covers a specific level range (e.g., levels 215A-D) within the multi-level tree data structure. In various cases, the access can be performed without accessing one or more other subtree data structures (e.g., sub-tribes 125B and 125C) that cover one or more levels within the specific level range. The computer system may update the multi-level tree data structure to include one or more nodes. As part of updating the structure, the computer system may modify a specific one of the multiple subtree data structures such that the specific level range covered by the specific subtree data structure changes (e.g., the subtree data structure is expanded to include nodes from newly added different levels). In some cases, the update may include splitting one of the multiple subtree data structures into two or more subtree data structures.

[0050] In some cases, the set of linked nodes of a first specific subtree data structure may correspond to the characters of the key code inserted into the multi-level tree data structure. The specific key code may correspond to at least one node in the first specific subtree data structure and at least one node in the second specific subtree data structure. The corresponding contiguous block of the first specific subtree data structure may include pointer information (e.g., pointer information 320) that identifies the location of the corresponding contiguous block of the second specific subtree data structure stored in system memory. Thus, the computer system can use the pointer information included in the corresponding contiguous block of the first specific subtree data structure to fetch the corresponding contiguous block of the second specific subtree data structure from system memory into the system cache. The pointer information may identify the storage size of the corresponding contiguous block of the second specific subtree data structure. The storage size may indicate the number of cache lines consumed when the corresponding contiguous block of the second specific subtree data structure is stored in the system cache linked to system memory.

[0051] Now go to Figure 6 A flowchart of method 600 is shown. Method 600 is an embodiment of a method executed by a computer system (e.g., database node 130) to facilitate the processing of transaction requests (e.g., transaction request 410). Method 600 can be performed by executing one or more program instructions stored on a non-transitory computer-readable medium. In some embodiments, method 600 may include more or fewer steps than shown; for example, method 600 may include the step of a computer system receiving a transaction request.

[0052] Method 600 begins at step 610, whereby the computer system stores a tree data structure (e.g., a dictionary 120) comprising multiple subtree data structures (e.g., a sub-dictionary 125) in system memory (e.g., system memory 150). A first subtree data structure (e.g., a sub-dictionary 125A) connected to a second subtree data structure (e.g., a sub-dictionary 125C) that is part of a specific branch of the tree data structure may include pointer information (e.g., pointer information 320) identifying the location of the second subtree data structure stored in system memory. The pointer information may also identify the storage size of the second subtree data structure (e.g., storage size indication 325). In various embodiments, the storage size of the tree data structure may not allow the entire tree data structure to be loaded into the system cache, while the storage size of a given one of the multiple subtree data structures may allow the entire given subtree data structure to be loaded into the system cache. In some cases, the tree data structure may not be stored in system memory as a contiguous block of information.

[0053] In step 620, the computer system uses the tree data structure to perform a key range lookup that includes traversing specific branches. This execution may include loading a second subtree data structure into a system cache (e.g., system cache 144) using pointer information included in the first subtree data structure. In some embodiments, loading the second subtree data structure into the system cache is based on storage size (e.g., the computer system may issue a memory request only for data from system memory containing the second subtree data structure). The pointer information may further identify the location in system memory where a third subtree data structure (e.g., sub-tribe 125B) is stored. The first subtree data structure may be connected to a third subtree data structure that is part of another specific branch of the tree data structure. In some embodiments, the computer system updates the tree data structure to include a set of nodes. The update may result in the tree data structure including one or more additional subtree data structures.

[0054] Exemplary computer system

[0055] Now go to Figure 7A block diagram depicts an exemplary computer system 700 that can implement system 100, database 110, and / or database node 130. Computer system 700 includes a processor subsystem 780 connected to system memory 720 and I / O interface 740 via interconnect 760 (e.g., system bus). I / O interface 740 is connected to one or more I / O devices 750. Computer system 700 can be any of various types of devices, including but not limited to: server systems, personal computer systems, desktop computers, laptop or notebook computers, mainframe computers, tablet computers, handheld computers, workstations, network computers, and consumer devices such as mobile phones, music players, or personal digital assistants (PDAs). Although for convenience... Figure 7 A single computer system 700 is shown, but system 700 can also be implemented as two or more computer systems operating together.

[0056] Processor subsystem 780 may include one or more processors or processing units. In various embodiments of computer system 700, multiple instances of processor subsystem 780 may be coupled to interconnect 760. In various embodiments, processor subsystem 780 (or each processor unit within 780) may include cache or other forms of onboard memory.

[0057] System memory 720 can be used to store program instructions that can be executed by processor subsystem 780 to enable system 700 to perform the various operations described herein. System memory 720 can be implemented using different physical memory media, such as hard disk storage, floppy disk storage, removable disk storage, flash memory, random access memory (RAM-SRAM, EDO RAM, SDRAM, DDR SDRAM, RAMBUS RAM, etc.), read-only memory (PROM, EEPROM, etc.), etc. The memory in computer system 700 is not limited to main memory such as memory 720. Instead, computer system 700 may also include other forms of memory, such as buffer memory in processor subsystem 780 and auxiliary memory on I / O devices 750 (e.g., hard disk drives, storage arrays, etc.). In some embodiments, these other forms of memory may also store program instructions that can be executed by processor subsystem 780. In some embodiments, program instructions that implement database application 146 at execution time may be included / stored within system memory 720.

[0058] According to various implementations, the EO interface 740 can be any of a variety of types of interfaces configured to connect to and communicate with other devices. In one implementation, the I / O interface 740 is a bridge chip (e.g., a southbridge) from the front end to one or more back-end buses. The EO interface 740 can be connected to one or more EO devices 750 via one or more corresponding buses or other interfaces. Examples of I / O devices 750 include storage devices (hard disk drives, optical disk drives, removable flash drives, storage arrays, SANs, or their associated controllers), network interface devices (e.g., to a local area network or wide area network), or other devices (e.g., graphics, user interface devices, etc.). In one implementation, the computer system 700 is connected to a network (e.g., configured to communicate via WiFi, Bluetooth, Ethernet, etc.) via the network interface device 750.

[0059] Implementations of the subject matter of this application include, but are not limited to, the following examples 1 to 20.

[0060] 1. A method comprising:

[0061] The database is operated by a computer system;

[0062] A multi-level tree data structure stored by a computer system for performing key lookups against a database, wherein the multi-level tree data structure is stored in system memory as multiple sub-tree data structures, each sub-tree data structure containing a set of linked nodes, wherein a given one of the multiple sub-tree data structures is stored in system memory as a corresponding contiguous block of information; and

[0063] Access by a computer system to a corresponding contiguous block of a first specific subtree data structure covering a specific level range in a multi-level tree data structure, wherein the access is performed without access to one or more other subtree data structures covering one or more levels within the specific level range.

[0064] 2. The method according to Example 1, wherein the set of linked nodes of the first specific subtree data structure corresponds to the characters of the key code inserted into the multi-level tree data structure, and wherein the specific key code corresponds to at least one node in the first specific subtree data structure and at least one node in the second specific subtree data structure.

[0065] 3. The method according to Example 2, wherein the corresponding contiguous block of the first specific subtree data structure includes pointer information that identifies the location of the corresponding contiguous block of the second specific subtree data structure stored in the system memory.

[0066] 4. The method according to Example 3, wherein the access further includes:

[0067] The corresponding contiguous block of the second specific subtree data structure is retrieved from system memory and cached using pointer information included in the corresponding contiguous block of the first specific subtree data structure.

[0068] 5. The method according to Example 3, wherein the pointer information identifies the storage size of the corresponding contiguous block of the second specific subtree data structure.

[0069] 6. The method according to Example 5, wherein the storage size indicates the number of cache lines consumed when the corresponding contiguous block of the second particular subtree data structure is stored in a system cache associated with the system memory.

[0070] 7. The method according to Example 1, wherein the storage size of the corresponding contiguous block of a given subtree data structure does not exceed the storage size of the system cache associated with the system memory.

[0071] 8. The method according to Example 1 further includes:

[0072] A computer system updates a multi-level tree data structure to include one or more nodes, wherein the update includes modifying a specific one of a plurality of subtree data structures such that the specific subtree data structure covers a specific level range.

[0073] 9. The method according to Example 1, wherein operating the database includes:

[0074] Receive a request to execute a transaction, which includes writing a file to the database, the file comprising a set of records associated with a set of keys; and

[0075] Executing transactions includes generating a multi-level tree data structure such that the nodes in the multi-level tree data structure correspond to characters in a set of keys, wherein the multi-level tree data structure is associated with a file.

[0076] 10. The method according to Example 1, wherein the multi-level tree data structure is stored in the system memory as contiguous blocks of information.

[0077] 11. A non-transitory computer-readable medium having stored thereon program instructions that cause a computer system to perform operations, the operations including:

[0078] The storage provides a multi-level tree data structure capable of performing key lookups against a database. This multi-level tree data structure is stored in system memory as multiple sub-tree data structures, each containing a set of linked nodes. A given sub-tree data structure is stored in system memory as a corresponding contiguous block of information.

[0079] Access the corresponding contiguous block of the first specific subtree data structure that covers a specific level range in a multi-level tree data structure, wherein the access is performed without accessing one or more other subtree data structures that cover one or more levels within the specific level range.

[0080] 12. The medium according to Example 11, wherein a corresponding contiguous block of a first specific subtree data structure is accessed using pointer information included in a corresponding contiguous block of a second specific subtree data structure.

[0081] 13. The medium according to Example 11, wherein the operation further comprises:

[0082] Update the multi-level tree data structure to include a set of nodes, wherein the update includes expanding the set of linked nodes of one of the multiple subtree data structures to include one or more of the node sets.

[0083] 14. The medium according to Example 11, wherein the operation further comprises:

[0084] Update a multi-level tree data structure to include a set of nodes, wherein the update includes splitting one of the multiple subtree data structures into two or more subtree data structures.

[0085] 15. A method comprising:

[0086] A tree data structure containing multiple subtree data structures is stored in system memory by a computer system. The first subtree data structure, connected to a second subtree data structure that is part of a specific branch of the tree data structure, includes pointer information identifying the location of the second subtree data structure stored in system memory; and

[0087] The computer system performs a key range lookup using a tree data structure, which includes traversing specific branches, wherein the execution includes loading a second subtree data structure into the system cache using pointer information included in the first subtree data structure.

[0088] 16. The method according to Example 15, wherein the pointer information further identifies the location of the third subtree data structure stored in the system memory, and wherein the first subtree data structure is connected to the third subtree data structure as part of another specific branch of the tree data structure.

[0089] 17. The method according to Example 15, wherein pointer information identifies the storage size of the second subtree data structure, and wherein loading the second subtree data structure into the system cache is based on the storage size.

[0090] 18. The method according to Example 15, wherein the storage size of the tree data structure does not allow the entire tree data structure to be loaded into the system cache, and wherein the storage size of a given one of the plurality of subtree data structures allows the entire given subtree data structure to be loaded into the system cache.

[0091] 19. The method according to Example 15 further comprises:

[0092] Update the tree data structure to include a set of nodes, where the update causes the tree data structure to include one or more additional subtree data structures.

[0093] 20. The method according to Example 15, wherein the tree data structure is not stored in the system memory as a contiguous block of information.

[0094] Although specific embodiments have been described above, even where only a single embodiment is described with respect to a particular feature, these embodiments are not intended to limit the scope of the disclosed text. Unless otherwise stated, the examples of features provided in the disclosed text are illustrative rather than restrictive. The above description is intended to cover substitutions, modifications, and equivalents that will be apparent to those skilled in the art who benefit from the disclosed text.

[0095] The scope of the disclosed text includes any feature or combination of features disclosed herein (explicitly or implicitly), or any generalization thereof, whether or not it solves any or all of the problems raised herein. Therefore, during the examination of this application (or an application claiming priority thereto), new claims may be made for any such combination of features. In particular, with reference to the appended claims, features of dependent claims may be combined with features of independent claims, and features in the individual independent claims may be combined in any suitable manner, not just in the specific combinations listed in the appended claims.

Claims

1. A method comprising: A collection of records stored in a file by a computer system; The computer system stores a multi-level tree data structure capable of determining whether a record containing a database key is not stored in the file. This multi-level tree data structure is stored in the computer system's system memory as multiple sub-tree data structures. The storage size of the multi-level tree data structure does not allow it to be loaded entirely into the computer system's system cache. The storage size of a given sub-tree data structure allows it to be loaded entirely into the system cache. Furthermore, the given sub-tree data structure contains a set of linked nodes corresponding to the set of characters in the database key inserted into the multi-level tree data structure. As part of a key lookup for a specific database key, the computer system retrieves a first subtree data structure covering a specific level range in the multi-level tree data structure from the system memory into the system cache, wherein the retrieval is performed without accessing one or more other subtree data structures covering one or more levels within the specific level range, and wherein the first subtree data structure includes pointer information specifying the storage size of a second subtree data structure, the pointer information enabling the computer system to retrieve the second subtree data structure as a whole; and The computer system uses the pointer information to extract the second subtree data structure into the system cache.

2. The method according to claim 1, wherein the first character of the specific database key corresponds to a node in the first subtree data structure, and the second character of the specific database key corresponds to a node in the second subtree data structure.

3. The method according to claim 2, wherein the pointer information identifies the storage location of consecutive blocks of the second subtree data structure in the system memory.

4. The method of claim 1, wherein the storage size indicates the number of cache lines consumed when the second subtree data structure is stored in the system cache.

5. The method according to any one of claims 1 to 4, further comprising: The computer system updates the multi-level tree data structure to include one or more nodes, wherein the update includes modifying a specific one of the plurality of subtree data structures such that the specific subtree data structure covers a specific level range.

6. The method of claim 1, wherein operating the database comprises: Receive a request to execute a transaction, the transaction including writing the file to a database, the file including the set of records; and Executing the transaction includes generating the multi-level tree data structure, wherein the multi-level tree data structure is associated with the file.

7. The method according to claim 1, wherein the multi-level tree data structure is stored in the system memory as a continuous block of information.

8. A computer-readable medium having program instructions stored thereon, the program instructions being capable of causing a computer system to perform operations including: Store a collection of records in a file; The storage includes a multi-level tree data structure that can be used to determine whether a record for a database key is not stored in the file, wherein the multi-level tree data structure is stored in the system memory of the computer system as multiple sub-tree data structures, wherein the storage size of the multi-level tree data structure does not allow the multi-level tree data structure to be loaded entirely into the system cache of the computer system, and wherein the storage size of a given sub-tree data structure allows the given sub-tree data structure to be loaded entirely into the system cache, and wherein the given sub-tree data structure contains a set of linked nodes corresponding to the set of characters for the database key inserted into the multi-level tree data structure; As part of a key lookup for a specific database key, a first subtree data structure covering a specific level range in the multi-level tree data structure is retrieved from the system memory into the system cache, wherein the retrieval is performed without accessing one or more other subtree data structures covering one or more levels within the specific level range, and wherein the first subtree data structure includes pointer information specifying the storage size of a second subtree data structure, the pointer information enabling the computer system to retrieve the second subtree data structure as a whole; and The pointer information is used to extract the second subtree data structure into the system cache.

9. The medium according to claim 8, wherein the pointer information identifies the storage location of consecutive blocks of the second subtree data structure in the system memory.

10. The medium according to claim 8 or 9, wherein the operation further comprises: The multi-level tree data structure is updated to include a set of nodes, wherein the update includes expanding the set of linked nodes of one of the plurality of subtree data structures to include one or more of the node sets.

11. The medium according to claim 8 or 9, wherein the operation further comprises: The multi-level tree data structure is updated to include a set of nodes, wherein the update includes splitting one of the plurality of subtree data structures into two or more subtree data structures.

12. A computer system, comprising: At least one processor; and A memory having stored program instructions that can be executed by the at least one processor to perform the method of any one of claims 1 to 7.

Images