Tri-data structure having a sub-tri-data structure

Abstract

Techniques are disclosed for a tree data structure capable of storing information indicative of a database key. A computer system is capable of operating the database. The computer system can store a multi-level tree data structure usable to perform key lookups on the database. In various cases, the multi-level tree data structure can be stored in system memory as multiple subtree data structures, each including a set of linked nodes. A given one of the multiple subtree data structures can be stored in system memory as a respective contiguous block of information. The computer system can access the respective contiguous blocks for a first specific subtree data structure that encompasses a particular range of levels in the multi-level tree data structure. This access can be performed without accessing one or more other subtree data structures that encompass one or more levels within the particular range of levels.

Description

Technical Field

[0001] The present disclosure generally relates to database systems, and more specifically to a tree data structure capable of storing information indicating database keys.

Background Art

[0002] Modern database systems routinely implement a management system that can store a collection of information in an organized manner that can be efficiently accessed and manipulated by a user. In some cases, these management systems maintain a Log-Structured Merge Tree (LSM tree) having multiple levels that each store information as key-value pairs in database records. The LSM tree typically includes two high-level components, namely an in-memory buffer and persistent storage. In operation, the database system first writes database records to the in-memory buffer and later flushes the database records to persistent storage. As part of flushing the database records, the database system writes the database records to a new file stored in persistent storage.

Brief Description of the Drawings

[0003] [Figure 1] FIG. is a block diagram showing exemplary elements of a system that can use a set of trie data structures according to some embodiments. [Figure 2] FIG. is a block diagram showing elements of an example of a trie data structure having a set of sub-trie data structures according to some embodiments. [Figure 3] FIG. is a block diagram showing exemplary elements of pointer information of a node of a trie data structure according to some embodiments. [Figure 4] FIG. is a block diagram showing exemplary elements for creating and updating a trie by a database application according to some embodiments. [Figure 5]This flowchart illustrates exemplary methods related to the use of a tridata structure in several embodiments. [Figure 6] This flowchart illustrates exemplary methods related to the use of a tridata structure in several embodiments. [Figure 7] This is a block diagram illustrating an exemplary computer system in several embodiments. [Modes for carrying out the invention]

[0004] This disclosure includes references to “one embodiment” or “one embodiment.” Occurrences of the phrase “in one embodiment” or “in one embodiment” do not necessarily refer to the same embodiment. Certain features, structures, or characteristics may be combined in any suitable manner that is not inconsistent with this disclosure.

[0005] In this disclosure, various entities (which may be referred to in various ways as “units,” “circuits,” other components, etc.) may be described or claimed as “configured” to perform one or more tasks or operations. This formulation—[entity] configured to [perform one or more tasks]—is used herein to refer to a structure (i.e., something physical, such as an electronic circuit). More specifically, this formulation is used to indicate that the structure is arranged to perform one or more tasks during an operation. A structure can be said to be “configured” to perform some task even if the structure is not currently being operated. “Network interface configured to communicate over a network” is intended to cover, for example, an integrated circuit having a network that performs this function during an operation even if the integrated circuit is not currently being used (e.g., no power supply is connected to it). Thus, an entity described or referred to as “configured” to perform some task refers to something physical, such as a device, a circuit, or memory that stores executable program instructions to perform a task. This phrase is not used herein to refer to anything intangible. Therefore, the structure “configured to be” is not used in this specification to refer to software entities such as application programming interfaces (APIs).

[0006] The term "configured to" does not mean "configurable to." For example, an unprogrammed FPGA is not considered "configured to" perform a particular function, but it may be "configurable to" perform that function, and after programming, it may be "configured to" perform that function.

[0007] As used herein, the terms “first,” “second,” etc., are used as labels for preceding nouns and, unless otherwise specified, do not imply any type of ordering (e.g., spatial, temporal, logical, etc.). For example, in a processor having eight processing cores, the terms “first” and “second” processing cores can be used to refer to any two of the eight processing cores. In other words, the first and second processing cores are not limited to, for example, processing cores 0 and 1.

[0008] As used herein, the term “based on” is used to describe one or more factors that influence a decision. This term does not preclude further factors from influencing the decision. That is, the decision may be based solely on the specified factors, or on the specified factors and other unspecified factors. Consider the phrase “determine A based on B.” This phrase specifies that B is a factor used to determine A, or that influences the determination of A. This phrase does not preclude the determination of A from further on any other factor, such as C. This phrase is also intended to cover embodiments in which A is determined solely on B. Thus, as used herein, the phrase “based on” is synonymous with the phrase “at least in part on.”

[0009] As used herein, “module” refers to software and / or hardware capable of performing a specified set of operations. A module may also refer to a set of software instructions that a computer system can execute to perform a set of operations. A module may also refer to hardware configured to perform a set of operations. Hardware modules can consist of general-purpose hardware and non-temporary computer-readable media for storing program instructions, or specialized hardware such as customized ASICs. Thus, a module described as “executable” to perform operations refers to a software module, while a module described as “configured” to perform operations refers to a hardware module. A module described as “operable” to perform operations refers to both software and hardware modules.

[0010] During the operation of a database built around an LSM tree, the database system typically flushes database records from in-memory buffers to files stored in persistent storage. These files often contain multiple database records corresponding to various different database key values. In operation, the database system processes transaction requests, which may involve accessing database records from files in persistent storage for the database keys specified in those requests. As part of processing transaction requests, in some cases, the database system performs a single-database-record lookup, retrieving files from persistent storage and checking them for database records corresponding to specific database keys identified in the transaction request. However, in many cases, the files retrieved by the database system do not contain the database records corresponding to these specific database keys. As a result, the database system wastes time and resources fetching these files from persistent storage.

[0011] In some cases, probabilistic data structures such as Bloom filters can be used to reduce the number of files fetched from persistent storage because they often provide an indicator of whether a database record corresponding to a database key is likely to exist in that file, or is definitely not. Therefore, the database system can fetch only files where a corresponding database record might exist, and skip files where it does not. However, in some cases, as part of processing a transaction request, the database system performs a key range lookup, in which the database system searches for database records where a database key falls within a specified key range. To reduce the number of files fetched from persistent storage, certain types of probabilistic data structures called "tries" (or alternatively, "tri data structures") can be used in key range lookups. Furthermore, tries can be used to determine whether a particular database node contains database records in its in-memory buffer where a database key falls within a specified key range. Where used herein, the term "trie" is used in accordance with its established meaning and in the linked nodes corresponding to character values. branch This refers to the tree-like data structure in which the trine is created. branch That branch Each node can represent a database key, with each character corresponding to that key.

[0012] Tries offer advantages over other probabilistic data structures such as Bloom filters for key range lookups, but current implementations of tries have drawbacks. One drawback relates to how tries are stored in system memory and then fetched from system memory to the system cache. These implementations currently use tries branchTraversing a tri results in a relatively large number of memory fetches. In particular, the current implementation stores the tri in system memory such that all nodes at a given level precede all nodes at the next level in the tri in memory. As a result, branch Traversing from one level downwards to the next as part of a traversal may involve moving a reasonable memory distance within system memory. Consider an example where a computer system traverses from level 27 to level 28 of a particular try. Due to the possibility that level 27 may contain a large number of nodes and the limited cache size, when the computer system issues a memory request to fetch data from level 27 into the cache, the data returned from system memory is quite unlikely to contain data from the nodes at level 28. That is, the chunk of sequential data fetched from system memory to store in the system cache may contain data representing only some of the nodes at level 27, and not data representing the nodes at level 28. Consequently, when traversing downwards from level 27 to level 28 of a try, the computer system needs to issue another memory request to fetch data at level 28. The above method of storing a try in system memory is necessary because the system is descending. branch This results in the need to issue one memory request for each node. Consequently, branch Descending (which is often multiple levels of data structures) involves numerous memory requests, and these requests together can take an enormous amount of time to process.

[0013] This disclosure describes a method for realizing a tridata structure that overcomes some or all of the shortcomings of other tri approaches. In the various embodiments described below, a computer system maintains a tridata structure which includes a set of subtridata structures, each having a set of linked nodes. In various cases, the first subtridata structure is a specific tridata structure. branch It may be connected as part of a second subtri-data structure. In various embodiments, the first subtri-data structure is stored in system memory as a contiguous block of information including node information detailing the nodes of the first subtri-data structure and pointer information identifying the location in system memory where the information of the second subtri-data structure is stored. The computer system can use the pointer information contained in the first subtri-data structure to load the second subtri-data structure into the system cache of the computer system. In various cases, the second subtri-data structure and other subtri-data structures may encompass nodes within the same particular range of levels in the tri-data structure. In various embodiments, the computer system loads the second subtri-data structure without accessing one or more other subtri-data structures that encompass nodes within one or more levels of the same particular range of levels as the second subtri-data structure.

[0014] A computer system branch Consider the previous example of traversing downwards through the 27th and 28th levels. The tridata structure realized by the method discussed in this disclosure is, for example, a node at the 26th level (one of which is a specific branch A first subtri data structure including (which may correspond to) and specific levels from the 27th to the 34th level branchIt may include a second subtri data structure containing nodes from level 27 to level 34, while still being able to be fully stored in the system cache because the second subtri data structure does not need to contain all nodes at those levels. As a result, when the computer system uses the pointer information contained in the first subtri data structure to fetch the second subtri data structure into the system cache, the computer system can retrieve specific data without having to issue another memory request. branch It is possible to traverse from the 27th level to the 34th level. Therefore, these methods are for the tridata structure branch This may be advantageous over previous approaches because it allows the computer system to issue fewer memory requests when traversing a certain level. In other words, the previous approach had many nodes at a certain level in the trial. branch When not related to traversal, cache space is wasted fetching most or all of those nodes. Next, we will discuss an exemplary application of these techniques, beginning with reference to Figure 1.

[0015] Next, referring to Figure 1, a block diagram of system 100 is shown. System 100 includes a set of components that can be implemented by hardware or a combination of hardware and software routines. In an exemplary embodiment, system 100 includes a database 110 and a database node 130 which includes a compute complex 140 and system memory 150. Further shown, the database 110 includes files 115 and tri 120, the compute complex 140 executes a set of program instructions to implement a database application 146 and includes a system cache 144. In some embodiments, system 100 may be implemented differently from that shown. For example, system 100 may include a plurality of database nodes 130 that interact with each other and with the database 110.

[0016] System 100, in various embodiments, implements a platform service (e.g., a customer relationship management (CRM) platform service) that enables users of its service to develop, run, and manage applications. System 100 may also be a multitenant system that provides various functions to multiple users / tenants hosted by a multitenant system. Thus, System 100 may not only provide code, web pages, and other data to users, databases, and other entities associated with System 100, but may also execute software routines from various different users (e.g., providers and tenants of System 100). As shown in the figure, for example, System 100 includes a database node 130 that stores a file 115 of database 110 for users associated with System 100 and can access data from the file 115.

[0017] The database 110 is a collection of information organized in a manner that enables access, storage, and manipulation of that information in various embodiments. Therefore, the database 110 may include support software that enables database nodes 130 to perform operations (e.g., access, storage, etc.) on the information stored in the database 110. In some embodiments, the database 110 is implemented by one or more storage devices connected together on a network (e.g., a storage attached network, SAN) and configured to store information redundantly to prevent data loss. The storage devices may permanently store data, and therefore the database 110 may function as persistent storage. In various embodiments, the database 110 is shared among multiple database nodes 130 such that database records written to file 115 by one database node 130 are accessible by other database nodes 130.

[0018] In various embodiments, database 110 implements a portion of a log-structured merge-tree (LSM tree) having multiple levels of files 115 containing database records. The LSM tree can include two high-level portions, namely an in-memory portion and an on-disk portion. One or more “top” levels of the LSM tree can include database records that are written to an in-memory buffer of database node 130. The remaining levels of the LSM tree can include files 115 containing database records that are written to database 110. In various cases, system memory 150 and system cache 144 can facilitate the in-memory portion of the LSM tree, while database 110 facilitates the on-disk portion of the LSM tree.

[0019] File 115 is, in various embodiments, a set of database records. A database record may be a key-value pair containing data and a corresponding database key that can be used to look up that database record. For example, a database record may correspond to a data row in a database table, where the database record specifies the value of one or more attributes associated with the database table. In various embodiments, File 115 is associated with one or more database key ranges defined by the keys of the database records contained in File 115. Consider an example where File 115 stores three database records associated with the keys "AA", "AB", and "AC", respectively. These three keys extend to the database key range AA→AC, and therefore File 115 can be associated with the database key range AA→AC. As discussed herein, Tri 120 can be used by the database node 130 to determine whether File 115 has database records associated with database keys that fall within a specified database key range.

[0020] In various embodiments, the trie 120 is a trie data structure that includes a set of subtrie data structures each including a set of linked nodes. FIG. 2 shows an example of the trie 120. The trie 120 can be a probabilistic data structure that provides an index of a database key range associated with one or more files 115. As used herein, the term "probabilistic data structure" refers to a data structure that maintains information indicating whether a particular item either does not exist or may exist at a particular location within the system. As an example, a probabilistic data structure can store information indicating whether a database record does not exist or may exist within a file. In various embodiments, the trie 120 has a memory footprint that is reasonably smaller than the corresponding file 115. As a result, the database node 130 may be able to access the trie 120 more quickly and efficiently than accessing the corresponding file 115. As a result, instead of directly accessing a given file 115 to check for a database record, using the trie 120 to check whether a particular database record may be included in a given file 115 can provide a substantial performance increase to the system 100.

[0021] When a database record is written to the file 115, in various embodiments, the database node 130 generates a trie 120 for that file 115. The trie 120 can form a tree-like data structure, and its branch corresponds to the database key of the database record being written to the corresponding file 115. Thus, branch can include nodes representing the character values of the corresponding database key. When the database node 130 wants to determine whether the file 115 may contain a particular database record, in various embodiments, the database node 130 checks for nodes that match the character value of the database key of that database record branchA single-key lookup can be performed, including searching for the corresponding try 120. Database node 130 will then determine if the aggregate nodes fall within the specified key range. branch Further key range lookups may be performed, including searching for the corresponding try 120. branch If it can be found in try 120, the database node 130 can determine that at least one database record may exist in the corresponding file 115 that falls within the specified key range.

[0022] In various embodiments, system 100 is implemented using multiple database nodes 130 that first write database records to their own in-memory buffers before flushing the database records to database 110. During operation, database nodes 130 may want to access the latest version of a data row as part of processing a database transaction. However, that latest version may be stored in the in-memory buffer of another database node 130. Consequently, the former database node 130 may issue a database record request to the latter database node 130 to determine whether the latest version is stored in the latter database node 130. Often, it is not stored in the latter database node 130, and resources are wasted. Therefore, in various embodiments, a further try 120 can be used to determine whether a database record might exist in the in-memory buffer of another database node 130 and therefore whether a database record request should be issued.

[0023] The database node 130 is, in various embodiments, hardware, software, or a combination thereof that can provide database services such as data storage, data retrieval, and / or data manipulation. Such database services may be provided to other components within the system 100 and / or external components of the system 100. As an example, the database node 130 may receive a database transaction request from an application server (not shown) requesting that data be written to or read from the database 110. The database transaction request may specify an SQL SELECT command to select one or more rows from one or more database tables. The contents of the rows may be defined in a database record, and so the database node 130 can locate and return the location of one or more database records corresponding to the selected one or more table rows. In some cases, the database transaction request may instruct the database node 130 to write one or more database records to the LSM tree. In various embodiments, the database node 130 first writes those database records to the system cache 144 and system memory 150 before flushing them to the database 110.

[0024] The compute complex 140 is, in various embodiments, a collection of processors, processor cores, and / or caches 144. For example, the compute complex 140 may include four processor cores, each having its own internal system cache 144. The processor cores of the compute complex 140 can be configured to execute instructions of a particular instruction set architecture (ISA). This may include operating system instructions and user application instructions (e.g., instructions executable to implement a database application 146). These instructions can be stored in a non-temporary computer-readable medium, such as system memory 150 coupled to the compute complex 140. The system memory 150 may be volatile memory (e.g., dynamic random access memory) or non-volatile memory (e.g., NAND flash memory). The compute complex 140 can retrieve a set of program instructions from the system memory 150 and execute the program instructions to perform functions described herein, such as a database application 146.

[0025] Database application 146 is a set of program instructions executable to manage database 110, which in various embodiments includes managing an LSM tree built around database 110. Thus, database application 146 can process database transactions, read records from the LSM tree, and write records to the LSM tree. Database application 146 can maintain metadata that describes the structural layout of the LSM tree, including where files 115 are stored in database 110 and what records may be contained in those files 115. In various embodiments, the metadata includes tries 120 corresponding to files 115 stored in database 110. Database application 146 can use the metadata to perform faster and more efficient key range lookups as part of processing database transactions, as discussed.

[0026] To facilitate the processing of database transactions, in various embodiments, the database node 130 uses tries 120. For example, as discussed, when the database node 130 searches for a database record that falls within a database key range, it may use tries 120 to identify files 115 and / or database nodes 130 that may contain such database records. Thus, the database node 130 can first retrieve one or more tries 120 from database 110 and then store them in system memory 150. In some embodiments, when the database node 130 wants to check a particular file 115 and / or other database nodes 130, the database node 130 retrieves the corresponding try 120 at that time and stores it in system memory 150. Once the tries 120 are stored in system memory 150, the database node 130 can begin traversing those tries 120.

[0027] When traversing Tri 120, the database node 130 issues a memory fetch 148 to system memory 150 to load the sub-tries 125 of Tri 120, including the root node of the entire Tri 120, into the system cache 144. As previously mentioned, Tri 120 includes a set of sub-tries 125. In various embodiments, a sub-trie 125 is made up of one or more linked nodes corresponding to character values. branch It is a tree-like data structure having a subtry 125 branch This may correspond to part or all of a specific database key, and the remainder of that database key may correspond to other subtry 125s linked to the former subtry 125. branchThis may correspond to the following. When stored in the system cache 144 and system memory 150, a subtry 125 can be stored as a single contiguous block of information / data (bytes) in the memory address space. However, in some cases, a try 120 may not be stored as a single contiguous block of information, i.e., individual subtry 125s may not be adjacent to each other in memory. However, in some cases, a try 120 may be stored as a single contiguous block of information in memory. In various embodiments, the memory size of a subtry 125 does not exceed the memory space of the system cache 144. Therefore, the database node 130 can load the entire subtry 125 into the system cache 144. However, a try 120 may be larger than the memory space of the system cache 144, and therefore cannot be fully loaded into the system cache 144 by the database node 130. An example of a subtry 125 is discussed with respect to Figure 2.

[0028] After subtry 125 is loaded into system cache 144, database node 130 then... branch We began traversing those branch It is possible to determine whether it corresponds to a database key that falls within a specific database key range. Given the first subtry 125 branch When traversing, in various cases, database node 130 is linked to another node included in the second subtry 125. branch It may reach a node within the system. Database node 130 can load the second subtry 125 into the system cache 144 using pointer information contained in the first subtry 125 that identifies the memory location in system memory 150 where the second subtry 125 is stored. Then, database node 130 can access the second subtry 125 stored in the system cache 144. branchIt is possible to continue traversing. In this way, the database node 130 may extend across multiple subtry 125. branch It is possible to traverse. When a database node 130 accesses a subtry 125 that spans a range of levels, in various embodiments, the database node 130 accesses a subset of all nodes within those levels, while the other nodes are included in other subtry 125 that span that range of levels. branch As part of traversing, database node 130 does not need to access all nodes within a certain level, which means that it does not need to access all nodes within a certain level. branch Using the memory space that would be occupied by nodes unrelated to that particular branch This allows access to further nodes at lower levels of Tri-120 related to the traversal.

[0029] Next, referring to Figure 2, a block diagram of exemplary elements of trial 120 and its layout in system memory 150 are shown. In the exemplary embodiment, trial 120 includes sub-trials 125A to D, each of which has a set of nodes 210. As shown, sub-trial 125A includes nodes 210A, 210B, 210E, 210F, and 210K; sub-trial 125B includes nodes 210C and 210G; sub-trial 125C includes nodes 210D, 210H, 210I, 210J, and 210L; and sub-trial 125D includes node 210M. As further shown, information representing sub-trials 125A to 125D is stored sequentially in system memory 150. In some embodiments, the layout of trial 120 and / or its layout in system memory 150 may be implemented differently from that shown. For example, Tri 120 may be stored discontinuously in system memory 150.

[0030] As described above, in various embodiments, try 120 is used to store the database keys of the database records written to the file 115 corresponding to try 120. As shown in the figure, try 120 in the exemplary embodiment stores at least the keys "AM", "ART", "OX", "TA", "TIP", and "TOP". Each database key is in try 120 branch The set of linked nodes 210 that form the structure can be stored in Tri 120. For example, the key "TIP" descends levels 215A to D of Tri 120. branch These are stored as nodes 210D, 210I, and 210L, which form a node. As will be discussed in more detail with respect to Figure 3, node 210 is a data structure that stores information about the character value it represents and how node 210 is linked to other nodes 210 in various embodiments.

[0031] When performing a key range lookup (or single key lookup), the database node 130 can traverse level 215 of try 120 by issuing a memory fetch 148 to load subtry 125 from system memory 150 into the system cache 144. Consider an example where the database node 130 wants to determine whether file 115, which corresponds to try 120, may contain database records that fall within the key range "TCP" → "TLC". In various embodiments, the database node 130 first issues a memory fetch 148 to system memory 150 to load subtry 125, which includes node 210 representing the root of try 120. In the illustrative embodiment, node 210A stores the root character value, and therefore subtry 125A is fetched into the system cache 144. As shown in the figure, five nodes 210 are fetched, which may be because the system cache 144 may have the capacity to store information for only five nodes 210 at a time.

[0032] When subtry 125A is loaded into the system cache 144, database node 130 can determine that node 210A is linked to three nodes, namely nodes 210B, 210C, and 210D. Database node 130 can further determine that nodes 210B and 210C represent character values ​​outside the key range "TCP" → "TLC", while the character value "T" of node 210D falls within that range. Therefore, database node 130 includes node 210D. branch It is possible to traverse. Since subtry 125A does not include node 210D, database node 130 can issue another memory fetch 148 to load subtry 125C into the system cache. Since node 210A is linked to node 210 which is stored as part of the other subtry 125, in various embodiments node 210A includes pointers 220A and 220B that point to memory locations in system memory 150 where the other subtry 125 is stored. Thus, using pointer 220A, database node 130 can load subtry 125C into the system cache 144 without having to load the other subtry 125.

[0033] When subtry 125C is loaded into system cache 144, database node 130, branchBy traversing, it can be determined that linked nodes 210D, 210I, and 210L form a key that falls within the key range "TCP" → "TLC", and therefore, the corresponding file 115 may store database records that fall within that key range. In various cases, database node 130 can perform a key lookup that descends through multiple levels 215, but not all nodes 210 are accessed for those descended levels 215. In the previous example, database node 130 did not fetch information for nodes 210C, 210G, and 215M into the system cache 144. By not accessing all nodes 210 for the descended levels 215 in the key lookup, database node 130 can reduce the number of memory fetches 148 issued, saving time and resources.

[0034] Next, referring to Figure 3, a block diagram of exemplary elements of node 210 contained in sub-tri 125 within tri 120 is shown. In the exemplary embodiment, sub-tri 125A contains node 210A, and sub-tri 125B contains node 210B. As shown, node 210A includes node information 310 and pointer information 320 indicating the location of node 210B. Further shown, the pointer information 320 includes pointer 220 and memory size index 325. In some embodiments, node 210 is implemented differently from that shown. For example, the pointer information 320 may not include the memory size index 325.

[0035] Node information 310 includes information indicating the character value represented by the corresponding node 210 in various embodiments. Node information 310 may further indicate the nodes 210 connected to the node 210 of node information 310. For example, node information 310 for node 210A may indicate that node 210A is linked to node 210C and that node 210C follows node 210A in order. As another example, node information 310 for node 210A may indicate that node 210A descended from another specific node 210. In some embodiments, node information 310 indicates at what level of the trie 120 the node 210 of node information 310 resides. For example, node information 310 may indicate that node 210A is at the second level, descending from the top of the trie 120.

[0036] The pointer information 320 includes information that can be used in various embodiments to access one or more nodes 210 linked to node 210 of the pointer information 320. In various cases, the pointer information 320 may include a pointer (e.g., a memory offset) to a node 210 that resides in the same subtri 125 as node 210 of the pointer information 320. For example, nodes 210A and 210B (not shown) may be directly linked and reside in a particular subtri 125. The pointer information 320 for node 210A may indicate the location in the data structure of a particular subtri 125 from which node 210B can be accessed. In various cases, the pointer information may include a pointer (e.g., a memory offset) to a node 210 that resides in a different subtri 125 from node 210 of the pointer information 320. As shown in the figure, the pointer information 320 of node 210A includes a pointer 220 that identifies the location of node 210C (for example, a memory address in system memory 150), that is, the pointer 220 can identify a location in system memory 150 that contains a block of information / data (for example, bytes defining node information 310 and pointer information 320) that describes the pointed-to node 210.

[0037] Therefore, from node 210A to node 210C branch When traversing, the database node 130 can use the pointer information 320 of node 210A to load subtry 125B (including node 210C) from system memory 150 into the system cache 144. In various embodiments, the pointer information 320 includes a memory size index 325 indicating the size (e.g., 64KB) of subtry 125, which includes the pointed node 210. Thus, when attempting to route subtry 125B into the system cache 144, the database node 130 can determine how much data to request from system memory 150 to load subtry 125B based on the size of subtry 125B indicated by the memory size index 325 of the pointer information 320 of node 210A. As an example, the memory size index 325 may specify the number of cache lines that will be consumed when subtry 125 is stored in the system cache 144. Thus, the database node 130 can fetch that number of cache lines corresponding to the data from system memory 150. By using the memory size index 325, the database node 130 does not need to fetch more data from system memory 150 than is required to load the subtry 125.

[0038] Next, referring to Figure 4, a block diagram of a database application 146 that writes file 115 and try 120 to database 110. Further shown, database application 146 receives a set of transaction requests 410 having keys 415. A transaction request 410 can specify a set of database records (along with their corresponding keys 415) to be written to database 110 as file 115. Thus, database application 146 can process transaction requests 410 for writing one or more files 115 to database 110. As part of processing transaction requests 410, database application 146 can generate corresponding tries 120 for one or more files 115, containing the keys 415 of the database records contained in those one or more files 115. In some further cases, as part of processing transaction requests 410, database application 146 may generate a set of tries 120 based on the keys of database records stored in an in-memory buffer of database node 130 (which may be implemented using system cache 144 and system memory 150).

[0039] When generating the try 120, the database application 146 generates the tree structure of the try. branch Key 415 can be inserted into Tri 120 as follows. In various cases, key 415 shares a common prefix, and as a result, branch They can share common parts. For example, as shown in Figure 2, the keys "AM" and "ART" can share their respective branchIn this configuration, they share a common node 210 (i.e., node 210B). As part of generating trial 120, in various embodiments, the database application 146 determines the division of node 210 in that trial 120 into subtrials 125. The database application 146 can use various criteria to determine how to divide node 210. One such criterion may include ensuring proximity to the ratio between the number of nodes 210 in subtrial 125 and the number of levels 215 in subtrial 125. For example, the database application 146 may be confident that for every four nodes, subtrial 125 will descend to at least one further level. Thus, if subtrial 125 contains nine nodes 210 in this example, subtrial 125 will contain three levels 215. Another criterion may include ensuring proximity to the ratio between the number of nodes 210 in trial 120 and the number of levels 215 in subtrial 125. For example, if there are 50 nodes 210 in Trial 120, the database application 146 can divide these nodes 210 into 5 sub-trials 125, with 10 nodes 210 in each sub-trial 125. Another criterion is given Trial 120 branch This may include ensuring that the average number of memory fetches 148 required to access the entire system is minimized. After determining the partitioning of the nodes 210, in various embodiments, the database application 146 structures each group of nodes 210 as a respective consecutive block of data.

[0040] In some cases, the database application 146 can update the try 120. For example, if a particular try 120 is used to determine whether a record for a given key 415 is stored in another database node 130, and the nature of which records are stored in this other database node 130 changes over time, the try 120 may be updated to reflect the change. That is, if the other database node 130 updates its in-memory buffer to include a new database record, the try 120 used by the database application 146 can be updated to incorporate the key 415 of that database record. In various cases when updating the try 120, the database application 146 adds one or more nodes 210 to the try 120. When adding one or more nodes 210, the database application 146 can modify a particular subtry 125 to include the added one or more nodes 210. This allows the particular subtry 125 to extend to one or more further levels 215. In some cases, the database application 146 may create a new subtry 125 that includes one or more additional nodes 210. In even more cases, the database application 146 may split a particular subtry 125 into multiple subtrys 125, one of which includes one or more additional nodes 210.

[0041] Next, referring to Figure 5, a flowchart of Method 500 is shown. Method 500 is one embodiment of a method performed by a computer system (e.g., a database node 130) to facilitate the processing of a transaction request (e.g., transaction request 410). Method 500 may be performed by executing one or more program instructions stored on a non-temporary computer-readable medium. In some embodiments, Method 500 may include more or fewer steps than illustrated, for example, Method 500 may include the step of the computer system receiving a transaction request.

[0042] Method 500 begins in step 510 with the computer system operating on a database (e.g., database 110). In various embodiments, operating on the database involves the computer system receiving a transaction request to perform a transaction that includes writing a file (e.g., file 115) to the database containing a set of records associated with a set of keys (e.g., a set of keys 415). The computer system may perform this transaction, which may include generating a multilevel tree data structure (e.g., try 120) such that nodes (e.g., node 210) contained within the multilevel tree data structure correspond to characters in the set of keys. The multilevel tree data structure may be associated with a file.

[0043] In step 520, the computer system stores a multilevel tree data structure that can be used to perform key lookups against a database. In various cases, the multilevel tree data structure can be stored in system memory (e.g., system memory 150) as multiple subtree data structures (e.g., subtree 125), each containing a set of linked nodes. One of the multiple subtree data structures can be stored in system memory as a contiguous block of information. The memory size of each contiguous block for a given subtree data structure may not exceed the memory size of a system cache (e.g., system cache 144) coupled to system memory. The multilevel tree data structure can be stored in system memory as a contiguous block of information.

[0044] In step 530, the computer system accesses each contiguous block for a first specific subtree data structure (e.g., subtri 125A) that encompasses a particular range of levels in the multilevel tree data structure (e.g., levels 215A-D). In various cases, this access can be performed without accessing one or more other subtree data structures (e.g., subtri 125B and 125C) that encompass one or more levels within the aforementioned specific range of levels. The computer system can update the multilevel tree data structure to include one or more nodes. As part of updating the structure, the computer system can modify one particular subtree data structure so that the specific range of levels encompassed by this particular subtree data structure is changed (e.g., the subtree data structure is extended to include newly added nodes at different levels). In some cases, the update may include splitting one of the subtree data structures into two or more subtree data structures.

[0045] In some cases, a set of linked nodes in a first particular subtree data structure may correspond to characters of keys inserted into a multilevel tree data structure. A particular key may correspond to at least one node in the first particular subtree data structure and at least one node in the second particular subtree data structure. Each contiguous block for the first particular subtree data structure may contain pointer information (e.g., pointer information 320) that identifies the location in system memory where each contiguous block for the second particular subtree data structure is stored. Thus, the computer system can use the pointer information contained in each contiguous block for the first particular subtree data structure to fetch each contiguous block for the second particular subtree data structure from system memory to the system cache. The pointer information may identify the memory size of each contiguous block for the second particular subtree data structure. The memory size may indicate the number of cache lines consumed when each contiguous block for the second particular subtree data structure is stored in the system cache coupled to system memory.

[0046] Next, referring to Figure 6, a flowchart of Method 600 is shown. Method 600 is one embodiment of a method performed by a computer system (e.g., a database node 130) to facilitate the processing of a transaction request (e.g., transaction request 410). Method 600 may be performed by executing one or more program instructions stored on a non-temporary computer-readable medium. In some embodiments, Method 600 may include more or fewer steps than illustrated, for example, Method 600 may include the step of the computer system receiving a transaction request.

[0047] Method 600 begins in step 610 by having the computer system store a tree data structure (e.g., tri 120) containing multiple subtree data structures (e.g., subtri 125) in system memory (e.g., system memory 150). branch A first subtree data structure (e.g., subtri 125A) connected as part of a second subtree data structure (e.g., subtri 125C) may include pointer information (e.g., pointer information 320) that identifies the location in system memory where the second subtree data structure is stored. The pointer information may identify the memory size of the second subtree data structure (e.g., memory size index 325). In various embodiments, the memory size of a tree data structure may not allow the tree data structure to be fully loaded into the system cache, while the memory size of a given one of several subtree data structures may allow this given subtree data structure to be fully loaded into the system cache. In some cases, the tree data structure may not be stored in system memory as a single contiguous block of information.

[0048] In step 620, the computer system uses a tree data structure to determine branch Perform a key range lookup, which includes traversing the range. This execution may include loading a second subtree data structure into a system cache (e.g., system cache 144) using pointer information contained in the first subtree data structure. In some embodiments, loading the second subtree data structure into the system cache is memory size based (e.g., the computer system can only issue memory requests for data from system memory containing the second subtree data structure). The pointer information may further identify the location in system memory where a third subtree data structure (e.g., subtree 125B) is stored. The first subtree data structure is another specific tree data structure. branchAs part of this, it may be connected to a third subtree data structure. In some embodiments, the computer system updates the tree data structure to include a set of nodes. This update may result in the tree data structure containing one or more further subtree data structures.

[0049] Exemplary computer system Next, referring to Figure 7, a block diagram of an exemplary computer system 700 that can implement system 100, database 110, and / or database node 130 is shown. Computer system 700 includes a processor subsystem 780 which is coupled to system memory 720 and an I / O interface 740 via an interconnect 760 (e.g., a system bus). The I / O interface 740 is coupled to one or more I / O devices 750. Computer system 700 may be any of various types of devices, including, but not limited to, server systems, personal computer systems, desktop computers, laptop or notebook computers, mainframe computer systems, tablet computers, handheld computers, workstations, network computers, mobile phones, music players, or consumer devices such as personal data assistants (PDAs). For simplicity, a single computer system 700 is shown in Figure 7, but system 700 may be implemented as two or more computer systems working together.

[0050] The processor subsystem 780 may include one or more processors or processing units. In various embodiments of the computer system 700, multiple instances of the processor subsystem 780 may be coupled to the interconnect 760. In various embodiments, the processor subsystem 780 (or each processor unit within 780) may include a cache or other form of onboard memory.

[0051] System memory 720 is available for storing program instructions that can be executed by the processor subsystem 780 in order to cause the system 700 to perform the various operations described herein. System memory 720 may be implemented using different physical memory media, such as hard disk storage, floppy disk storage, removable disk storage, flash memory, random access memory (RAM, i.e., SRAM, EDO RAM, SDRAM, DDR SDRAM, RAMBUS RAM, etc.), and read-only memory (PROM, EEPROM, etc.). The memory of the computer system 700 is not limited to primary storage devices such as memory 720. Rather, the computer system 700 may further include other forms of storage devices, such as cache memory in the processor subsystem 780 and secondary storage devices on I / O devices 750 (e.g., hard drives, storage arrays, etc.). In some embodiments, these other forms of storage devices may also store program instructions that can be executed by the processor subsystem 780. In some embodiments, program instructions that, when executed, realize the database application 146 may be contained in / stored in system memory 720.

[0052] The I / O interface 740 may be any of various types of interfaces configured to connect to and communicate with other devices, according to various embodiments. In one embodiment, the I / O interface 740 is a bridge chip (e.g., a southbridge) from the front side to one or more back side buses. The I / O interface 740 may be connected to one or more I / O devices 750 via one or more corresponding buses or other interfaces. Examples of I / O devices 750 include storage devices (hard drives, optical drives, removable flash drives, storage arrays, SANs, or their associated controllers), network interface devices (e.g., to a local or wide area network), or other devices (e.g., graphics, user interface devices, etc.). In one embodiment, the computer system 700 is connected to a network via a network interface device 750 (e.g., configured to communicate via WiFi, Bluetooth®, Ethernet, etc.).

[0053] The realization of the subject matter of this application includes, but is not limited to, the following examples 1 to 20. 1. Steps to operate the database using a computer system, The computer system stores a multilevel tree data structure that can be used to perform key lookups against the database, wherein the multilevel tree data structure is stored in system memory as a plurality of subtree data structures, each containing a set of linked nodes, and a given one of the plurality of subtree data structures is stored in system memory as a respective continuous block of information. The steps include: accessing each of the contiguous blocks for a first specific subtree data structure that encompasses a specific range of levels in the multilevel tree data structure, wherein the access is performed without accessing one or more other subtree data structures that encompass one or more levels within the specific range of levels; A method that includes this. 2. The method according to Example 1, wherein the set of linked nodes in the first specific subtree data structure corresponds to the characters of keys inserted into the multilevel tree data structure, and a particular key corresponds to at least one node in the first specific subtree data structure and at least one node in the second specific subtree data structure. 3. The method according to Example 2, wherein each of the contiguous blocks for the first particular subtree data structure includes pointer information that identifies the location in the system memory where each of the contiguous blocks for the second particular subtree data structure is stored. 4. The aforementioned access is, The method according to Example 3, further comprising fetching the respective contiguous blocks for the second particular subtree data structure from the system memory to the system cache using the pointer information contained in the respective contiguous blocks for the first particular subtree data structure. 5. The method according to Example 3, wherein the pointer information identifies the memory size of each contiguous block for the second specific subtree data structure. 6. The method according to Example 5, wherein the memory size indicates the number of cache lines consumed when each contiguous block for the second particular subtree data structure is stored in a system cache coupled to the system memory. 7. The method according to Example 1, wherein the memory size of each contiguous block for the given subtree data structure does not exceed the memory size of the system cache coupled to the system memory. 8. A step of updating the multilevel tree data structure by the computer system to include one or more nodes, wherein the update includes modifying one particular subtree data structure such that the level of a particular range encompassed by the particular subtree data structure is changed. The method described in Example 1, further including the method described in Example 1. 9. Operating the aforementioned database is Receiving a request to execute a transaction that includes writing a file containing a set of records associated with a set of keys to the database, The transaction is executed, and includes generating the multilevel tree data structure such that the nodes included in the multilevel tree data structure correspond to the characters of the set of keys, and the multilevel tree data structure is associated with the file, The method described in Example 1, including the method described in Example 1. 10. The method according to Example 1, wherein the multilevel tree data structure is stored in the system memory as a contiguous block of information. 11. A computer-readable medium storing program instructions that can be used to execute an operation on a computer system, wherein the operation is: The system stores a multilevel tree data structure that can be used to perform key lookups against a database, wherein the multilevel tree data structure is stored in system memory as a plurality of subtree data structures, each containing a set of linked nodes, and a given one of the plurality of subtree data structures is stored in system memory as a contiguous block of information. The access is to each of the contiguous blocks for a first specific subtree data structure that encompasses a specific range of levels in the multilevel tree data structure, and such access is performed without accessing one or more other subtree data structures that encompass one or more levels within the specific range of levels. A medium that includes 12. The medium according to Example 11, wherein each of the contiguous blocks of the first specific subtree data structure is accessed using pointer information contained in each of the contiguous blocks of the second specific subtree data structure. 13. The above operation is, The medium according to Example 11, further comprising updating the multilevel tree data structure to include a set of nodes, wherein the update includes extending the set of linked nodes in one of the plurality of subtree data structures to include one or more of the set of nodes. 14. The above operation is, The medium according to Example 11, further comprising updating the multilevel tree data structure to include a set of nodes, wherein the update includes splitting one of the plurality of subtree data structures into two or more subtree data structures. 15. A computer system stores a tree data structure in system memory which includes a plurality of subtree data structures. branch The first subtree data structure, which is connected to a second subtree data structure as part of the first subtree data structure, includes pointer information that identifies the location in the system memory where the second subtree data structure is stored, The computer system uses the tree data structure to determine the specific branch The step of performing a key range lookup, which includes traversing a range, and the step of performing such lookup includes loading the second subtree data structure into the system cache using the pointer information contained in the first subtree data structure. A method that includes this. 16. The pointer information further identifies the location in the system memory where the third subtree data structure is stored, and the first subtree data structure is another specific of the tree data structure. branchThe method according to Example 15, which is connected as part of the third subtree data structure. 17. The method according to Example 15, wherein the pointer information identifies the memory size of the second subtree data structure, and loading the second subtree data structure into the system cache is based on the memory size. 18. The method according to Example 15, wherein the memory size of the tree data structure does not allow the tree data structure to be fully loaded into the system cache, and the memory size of a given one of the plurality of subtree data structures allows the given subtree data structure to be fully loaded into the system cache. 19. A step of updating the tree data structure to include a set of nodes, wherein the update results in the tree data structure including one or more further subtree data structures. The method described in Example 15, which further includes the method described in Example 15. 20. The method according to Example 15, wherein the tree data structure is not stored in the system memory as a single contiguous block of information.

[0054] While specific embodiments have been described above, these embodiments are not intended to limit the scope of the disclosure, even if only a single embodiment is described with respect to a particular feature. Examples of features provided in this disclosure are intended to be illustrative, not restrictive, unless otherwise specified. The above description is intended to cover alternatives, modifications, and equivalents that would be obvious to those skilled in the art who are interested in this disclosure.

[0055] The scope of this disclosure includes any feature or combination of features disclosed herein (either expressly or implicitly), or any generalization thereof, whether or not it mitigates any or all of the issues addressed herein. Therefore, new claims may be made for any such combination of features during the proceedings of this application (or any application claiming priority thereto). In particular, with reference to the appended claims, features from dependent claims may be combined with features from independent claims, and features from each independent claim may be combined in any suitable manner, not only in the specific combinations enumerated in the appended claims.

Claims

1. Computer systems store sets of records as files, The computer system stores a multilevel tree data structure that can be used to determine whether a record associated with a database key may be stored in the file, wherein the multilevel tree data structure is a tree-like data structure in which branches are made up of linked nodes, each of the linked nodes being linked to at least one other node, the multilevel tree data structure is stored in the system memory of the computer system as a plurality of subtree data structures, each containing a set of the linked nodes, the memory size of the multilevel tree data structure is not sufficient to load the entire multilevel tree data structure into the system cache of the computer system, the memory size of each of the plurality of subtree data structures is sufficient to load the entire each subtree data structure into the system cache, and the linked nodes of each subtree data structure correspond to the character values ​​of the database keys inserted into the multilevel tree data structure. As part of a key lookup for a database key, the computer system fetches a first subtree data structure from system memory into the system cache that covers a specific range of levels including multiple levels in the multilevel tree data structure, the fetch is performed without accessing one or more other subtree data structures that cover one or more levels in the specific range, the first subtree data structure includes pointer information that allows the computer system to fetch the entire second subtree data structure, the pointer information indicates the memory size of the second subtree data structure, the pointer information further indicates the location in system memory where a contiguous block of the second subtree data structure is stored, the contiguous block includes information on all the linked nodes of the second subtree data structure stored at a contiguous address in system memory, The computer system fetches the second subtree data structure from the system memory to the system cache using the pointer information. A method that includes this.

2. The method according to claim 1, wherein the database key includes a character corresponding to at least one node in the first subtree data structure and a character corresponding to at least one node in the second subtree data structure.

3. The method according to claim 1, wherein the memory size of the second subtree data structure indicates the number of cache lines consumed when the second subtree data structure is stored in the system cache.

4. The computer system updates the multilevel tree data structure to include one or more nodes, and the update includes modifying one particular subtree data structure such that the level of a particular range covered by the particular subtree data structure is changed. The method according to any one of claims 1 to 3, further comprising:

5. Manipulating a database is Receiving a request to perform a transaction that includes writing the aforementioned set of records, which is the file, to the database, The transaction is executed, which includes generating the multilevel tree data structure, and the multilevel tree data structure is associated with the file. The method according to any one of claims 1 to 4, including

6. The method according to any one of claims 1 to 5, wherein the multilevel tree data structure is stored in the system memory as a contiguous block of information.

7. A computer-readable medium storing program instructions that can be used to execute an operation on a computer system, wherein the operation is: Storing a set of records as a file, The present invention provides a multilevel tree data structure that can be used to determine whether a record associated with a database key may be stored in the file, wherein the multilevel tree data structure is a tree-like data structure with branches made up of linked nodes, each of the linked nodes being linked to at least one other node, the multilevel tree data structure is stored in the system memory of the computer system as a plurality of subtree data structures, each containing a set of the linked nodes, the memory size of the multilevel tree data structure is insufficient to load the entire multilevel tree data structure into the system cache of the computer system, the memory size of each of the plurality of subtree data structures is sufficient to load the entire each subtree data structure into the system cache, and the linked nodes of each subtree data structure correspond to the character values ​​of the database keys inserted into the multilevel tree data structure. As part of a key lookup for a database key, the system memory fetches a first subtree data structure from the system memory into the system cache that covers a specific range of levels, including multiple levels in the multilevel tree data structure, the fetch is performed without accessing one or more other subtree data structures that cover one or more levels in the specific range, the first subtree data structure includes pointer information that allows the computer system to fetch the entire second subtree data structure, the pointer information indicates the memory size of the second subtree data structure, the pointer information further indicates the location in the system memory where a contiguous block of the second subtree data structure is stored, the contiguous block includes information on all the linked nodes of the second subtree data structure stored at a contiguous address in the system memory, The computer system fetches the second subtree data structure from the system memory to the system cache using the pointer information. A medium that includes

8. The aforementioned operation is, The medium according to claim 7, further comprising updating the multilevel tree data structure to include a set of nodes, wherein the update includes splitting one of the plurality of subtree data structures into two or more subtree data structures.

9. At least one processor, A memory storing program instructions that can be executed by at least one processor to perform the method according to any one of claims 1 to 6, A computer system including a computer system.

Images