High-speed skip list scanning and insertion

Abstract

Techniques related to efficiently managing a skip list data structure are disclosed. In various embodiments, a computing system stores a skip list including a plurality of key / value records including one or more pointers to other ones of the plurality of key / value records. The computing system scans the skip list for a location associated with a particular key. The scanning includes identifying a particular portion of the skip list using a prefix of the particular key, where the particular portion includes key / value records having keys with the same prefix. The scanning further includes initiating a scan of a location within the identified portion. In some embodiments, the computing system inserts a key / value record into the skip list at the location associated with the particular key in response to the scanning identifying the location.

Description

Technical Field

[0001] The present disclosure generally relates to data storage, and more specifically to operations on skip list data structures.

Background Art

[0002] In the field of computer science, various complex data structures have been developed to facilitate the storage of information. These data structures are often created using multiple pointers to link a group of records together. When designing complex structures, developers often consider trade-offs related to the complexity of information insertion and retrieval and the overall size of the data structure. Skip lists are an example of a more complex data structure that is common because they can maintain large datasets while providing a maximum insertion complexity of O(log n) and a maximum search complexity of O(log n). In this type of data structure, records may be sorted based on key order and associated using a linked hierarchy of data record sequences, where each successive sequence skips fewer elements than the previous sequence. This linked hierarchy is implemented using a changing height of pointer towers, so within a given tower, pointers can be placed based on the number of records skipped. This ability to skip records when traversing a skip list can make it possible to find a given record more quickly than sequentially scanning the records.

Brief Description of the Drawings

[0003] [Figure 1] A block diagram showing one embodiment of a database system that processes concurrent database transactions using a skip list within a buffer data structure.

[0004] [Figure 2] A block diagram showing one embodiment of a record chain within a buffer data structure.

[0005] [Figure 3] This is a block diagram showing one embodiment of a skip list in a buffer data structure.

[0006] [Figure 4] This is a block diagram illustrating one embodiment of a slow skip list scan.

[0007] [Figure 5] This is a flowchart illustrating one embodiment of a high-speed skip list scan.

[0008] [Figure 6A] This block diagram shows an embodiment of key space partitioning that facilitates fast skip list scanning using key prefixes. [Figure 6B] This block diagram shows an embodiment of key space partitioning that facilitates fast skip list scanning using key prefixes.

[0009] [Figure 7A] This block diagram shows an embodiment of anchor record identification during a high-speed skip list scan. [Figure 7B] This block diagram shows an embodiment of anchor record identification during a high-speed skip list scan.

[0010] [Figure 8] This is a block diagram showing one embodiment of a climbing mountain during a high-speed skip list scan.

[0011] [Figure 9A] This block diagram shows one embodiment of backward walking through a skip list to insert a record. [Figure 9B] This block diagram shows one embodiment of backward walking through a skip list to insert a record.

[0012] [Figure 10A] This is a flowchart illustrating an embodiment of a method related to scanning a skip list. [Figure 10B] This is a flowchart illustrating an embodiment of a method related to scanning a skip list. [Figure 10C] This is a flowchart illustrating an embodiment of a method related to scanning a skip list.

[0013] [Figure 11] A block diagram showing one embodiment of an exemplary multi-tenant system.

[0014] [Figure 12] This is a block diagram showing an exemplary computer system for implementing the various systems described herein. [Modes for carrying out the invention]

[0015] In some cases, skip lists can be used to maintain large amounts of frequently manipulated information. For example, as will be described in more detail below, a database system can use a buffer data structure to store data for active database transactions until the database transactions are committed and their data can be flushed to the database system's persistent storage. The buffer data structure may include a skip list data structure that allows for efficient storage and lookup of transaction records in key order. Since this database system may handle a large number of transactions in parallel, efficient scanning of the skip list can be important for database performance.

[0016] The present disclosure describes embodiments for scanning records in a skip list and / or inserting new records into a skip list using a more efficient scan algorithm. As will be described later in various embodiments, a skip list can be partitioned into sections based on a prefix of keys such that a given section contains keys having the same prefix. When a scan is performed for a particular key, first the prefix of that key is used to determine the relevant section for that prefix. Thereafter, the scan is started within that section and can be performed on only a subset of the skip list. This is in contrast to other approaches where the scan starts at an initial node (i.e., a sentinel node) and is performed across the entire skip list. Since a given section is much smaller than the entire skip list, the memory accesses performed are far fewer. This can result in significant performance improvements, particularly in the exemplary database system described below, and in one embodiment, approximately 200,000,000 key-value records can be maintained in the skip list at a given time.

[0017] The present disclosure begins with a discussion of a database system related to FIGS. 1 and 2 that can maintain a skip list and use the fast scan algorithm described above. An exemplary skip list is discussed in relation to FIG. 3. An inefficient algorithm for scanning a skip list is discussed in relation to FIG. 4. The fast scan algorithm is discussed in relation to FIGS. 5 - 9B. Finally, methods and an exemplary computing system are discussed in relation to FIGS. 10A - 12.

[0018] Moving on to FIG. 1, a block diagram of the database system 10 is shown. In the illustrated embodiment, the database system 10 includes a transaction manager 104, a buffer data structure 106, and a database 108. As shown, the buffer data structure 106 includes a plurality of record chains 110, a hash table 120, an active transaction list 130, and a skip list 140. The record chain 110 includes key / value records 112. The hash table 120 includes a hash function 122 and an array of hash buckets 124 each including a latch 126. (As used herein, the terms “latch”, “lock”, and “semaphore” are generally used to refer to data structures that control access to resources shared among multiple potential consumers.) In the illustrated embodiment, the manager 104 also includes a scan engine 150. In some embodiments, the database system 10 may be implemented differently than that shown. For example, in some embodiments, the buffer data structure 106 may include more (or fewer) structures. Some of the structures 110, 120, 130, and 140 are shown separately for illustration purposes, but in some embodiments, the structures 110, 120, 130, and / or 140 may be intertwined with each other. For example, the skip list 140 may be implemented by adding pointers inside the key / value records 112 within the record chain 110.

[0019] In one embodiment, the transaction manager 104 includes executable program instructions for processing an incoming database transaction 102. Generally, a transaction 102 may be issued to read or write data to the database 108 and may be received from any of various sources, such as one or more client devices, application servers, or software running on the database system 10. As will be described in more detail below, this processing may involve the manager 104 initially storing the key / value pair record 112 of transaction 102 in a buffer data structure 106 until the record 112 can be flushed to persistent storage for record 112. Thus, various functionalities related to the buffer data structure 106, as described below, such as adding the key / value record 112 to the record chain 110, facilitating the acquisition of a hash bucket latch 126 for transaction 102, and modifications to the active transaction list 130 and the skip list 140, may be implemented by the transaction manager 104.

[0020] In one embodiment, the buffer data structure 106 is a data structure that buffers key / value pairs for active transactions until the transaction is committed. As described below, the buffer data structure 106 is structured in a way that allows for rapid insertion of key / value pairs, which may be executed concurrently in some cases, enabling efficient handling of a large number of transactions. Furthermore, the buffer data structure 106 may reside in local memory, enabling faster reads and writes than persistent storage in database 108 where data resides for longer periods. In various embodiments, the buffer data structure 106 allows simultaneous modifications for different transactions 102, but provides a concurrency control mechanism for the data within the buffer data structure 106 via a hash bucket latch 126. In some embodiments, committed transaction data is asynchronously flushed from the buffer data structure 106 to persistent storage in database 108. That is, instead of performing a flush at commit for the data of each transaction 102, the flush is performed periodically for multiple committed transactions 102. For example, in one embodiment, the transaction manager 104 initiates a flush to the database 108 in response to the buffer data structure 106 meeting a certain size threshold.

[0021] Database 108 can accommodate a suitable form of database implementation. In some embodiments, database 108 is a non-relational database implemented using a log-structured merge (LSM) tree for persistent storage. In some embodiments, layers of the LSM tree may be distributed across multiple physical computer systems providing persistent storage. In some embodiments, these computer systems are cluster nodes of a computer cluster providing a cloud-based system accessible to multiple clients. In some embodiments, database 108 may be part of a software-as-a-service (SaaS) model, and in other embodiments, database 108 may be directly operated by the user.

[0022] As described above, when the transaction manager 104 stores the key / value pairs of the active transaction 102 in the buffer data structure 106, a corresponding key / value record 112 containing the value and key may be created. If multiple transactions 102 attempt to write values ​​associated with the same key, a key / value record 112 may be generated for each value and linked together to form a record chain 110 corresponding to the key. For example, if a user withdraws a first amount from a bank account, resulting in a first database transaction 102, and then withdraws a second amount, resulting in a second database transaction 102, the record chain 110 corresponding to the account balance key may have two key / value records 112 reflecting these withdrawals. In various embodiments, each record 112 includes a transaction identifier (e.g., a transaction sequence number) that specifies the transaction 102 to which it is associated, and the records 112 may also be organized into a record chain 110 based on the order in which the transactions 102 are received. For example, with respect to Figure 2, as described below, the record chain 110 may be implemented using a linked list such that a new record 112 is inserted into the head of the linked list, and when a newer record 112 is created and the old record is flushed to the database 108, it is moved to the end. To facilitate quick access to the key / value record 112, the record chain 110 is appended to the hash bucket 124 of the hash table 120.

[0023] In one embodiment, the hash table 120 is a data structure that enables a time-lapse lookup of the record chain 110 based on a given key. That is, when a key is received, the hash table 120 is indexed by applying the hash function 122 to that key, generating an appropriate index value for the hash bucket 124 corresponding to the key. The record chain 110 can then be retrieved by directly referencing a pointer in the hash bucket 124. The ability to perform time-lapse lookups can significantly reduce the time spent reading key / value records 112, writing records 112, or performing key probes (i.e., determining whether the key has a key / value record 112 that exists in the buffer data structure 106).

[0024] As described above, in various embodiments, each hash bucket 124 includes a latch 126 that controls access to this record chain 110. Therefore, when a transaction attempts to read or write a value associated with a particular key, it can use that key to index the hash table 120 and acquire the latch 126 corresponding to the key associated with the hash bucket 124 before performing the read or write. If a latch 126 cannot be acquired for a database transaction 102, processing of the database transaction 102 may be delayed until the latch 126 is released. In some embodiments, the latch 126 may have one of three possible states: available, shared acquired, and exclusively acquired. If transaction 102 is not currently accessing the record chain 110, its latch 126 is available for acquisition. If transaction 102 is performing a read operation on key / value record 112, latch 126 can be acquired in a shared state, meaning that other transactions 102 can also acquire latch 126 as long as they are performing a read operation (i.e., not attempting to modify record 112 while reading). However, if transaction 102 is performing a write operation, latch 126 is acquired exclusively for transaction 102, meaning that other transactions 102 cannot acquire latch 116 until it is released. Therefore, if two transactions 102 are attempting to perform write operations on the same key, the later transaction will be delayed until the former completes its write operation and releases latch 12. If transaction 102 is attempting to access multiple key / value pairs, latch 126 can be acquired in ascending order of buckets 124 to prevent deadlocks.While the acquisition of latch 126 is primarily described in relation to read and write operations, latch 126 may also be acquired when performing other operations such as defragmentation, garbage collection, and flushing record 112 to the persistent store of database 108. In some embodiments, latch 126 may also function as a concurrency control mechanism for the active transaction list 130 and the skip list 140.

[0025] The active transaction list 130 is, in one embodiment, a data structure that tracks various metadata about an active transaction 102. In various embodiments, the metadata for a given transaction 102 includes the transaction identifier of transaction 102 and one or more pointers that can be used to access records 112 associated with transaction 102. In this way, the list 130 allows for the identification of records 112 of transaction 102 based on the transaction identifier, which may be useful, for example, when determining which records 112 should be deleted if transaction 102 is rolled back. The metadata may also include an indication of whether the transaction is active or committed, which may be useful in determining whether its records 112 can be marked for flushing to database 108. Therefore It can be used for this purpose.

[0026] In one embodiment, the skip list 140 is a data structure that maintains the order of keys within record 112, enabling forward and reverse scanning of keys. In some embodiments, the database 108 is configured so that records 11 are flushed for committed transactions 102 in ascending order of keys (as well as version order), and the skip list 140 may allow this ordering to be determined quickly and easily. As will be described in more detail below with reference to Figure 3, in some embodiments, the skip list 140 includes an indirect pointer for accessing record 112 in the skip list 140. That is, rather than having a direct pointer between records 112 (i.e., a pointer record specifying a memory address), the skip list 140 includes an indirect pointer to a hash bucket 124, which includes a direct pointer to chain 110. Advantageously, when a new record 112 is added to the record chain 110, the direct pointer in the hash bucket 124 is updated, rather than the indirect pointer in the skip list 140. The use of indirect pointers also allows the skip list 140 to utilize a hash bucket latch 126 when manipulating record 112 with respect to the list 140. Therefore, if record 112 of transaction 102 is accessed via the skip list 140 for modification or deletion, a latch 126 may be acquired for the key 112 to prevent other modifications from being performed. As described above, although shown separately from record 112 in Figure 1, a portion of the skip list 140 may reside in record 112 in some embodiments, as will be described below with respect to Figures 2 and 3.

[0027] When record 112 is inserted into buffer data structure 106, a scan of skip list 140 may be performed to determine where in skip list 140 the record should be inserted. Once record 112 is inserted, an additional skip list scan may be performed to find the specific record 112 in skip list 140. In the illustrated embodiment, scan engine 150 is a component of transaction manager 104 that can scan skip list 140 and may implement a fast scan algorithm, which will be described in more detail below and begin in Figure 5. Thus, as described above, scan engine 150 may divide skip list 140 into sections (hereinafter referred to as "anchor ranges") based on key prefixes such that a given section contains keys having the same prefix. To facilitate the lookup of a specific section, in various embodiments, scan engine 150 maintains an index that can be referenced to determine the relevant section for a given key prefix and to initiate a scan within that section. As described below, in some embodiments, this index is integrated into a hash table 120 by adding a pointer to record 112 within each section (called an "anchor record") in a hash bucket 124, thereby allowing a given section to be quickly found for a given key prefix. Once the relevant section for a given key prefix is ​​determined, the scan engine 150 may sequentially scan the records by implementing a technique hereafter referred to as "climbing the mountain." In various embodiments, the scan engine 150 may also be responsible for maintaining the skip list 140, including performing record insertions (and record deletions) for the skip list 140. As described, once the location for record insertion is determined from the scan, record 112 is inserted, and a pointer to be included in the skip list tower for record 112 is determined.The scan engine 150 may "walk backwards" from its current position to determine which pointers in other records 112 should be updated. In some embodiments, the scan engine 150 manages a thread pool of running threads to implement parallel high-speed scanning.

[0028] The contents of record 112, including those used to implement skip list 140, are described in more detail here to facilitate a good understanding of the fast scan algorithm, which will be described in detail below.

[0029] Next, moving to Figure 2, a block diagram of record chain 110 is shown. As shown, record chain 110 may include a group of key / value records 112A-112C, a collision record 220, and a lock record 230. Record 112 may further include a key 212, a value 213, a transaction identifier 214, a commit identifier 215, a purge flag 216, a lock 217, a skip list pointer 218, and a record chain pointer 219. In some embodiments, chain 110 may include more (or fewer) records 112, 220, or 230 than those shown, and a given record 112 may include more (or fewer) elements 212-219 than those shown.

[0030] In the illustrated embodiment, the record chain 110 is implemented using a linked list such that each key / value record 112 contains a pointer 219 that identifies the next record 112 in the chain 110. When a record 112 is added, it is inserted into the head, identified by a direct pointer 202 in the hash bucket 124, or appended to a collision record 220, as described later. The added record 112 may then contain a pointer 219 to a record that was previously in the head. As a record 112 becomes old, it moves towards the tail (record 112B or lock record 230 in Figure 2) until transaction 102 is committed. It can then be flushed to persistent storage in database 108 and deleted. The transaction identifier 214 of a given record 112 can identify the transaction 102 to which the record 112 is associated, as well as indicate the order in which the transaction 102 was received. Thus, since record 112B is further from the head than record 112A, transaction ID 214B may correspond to a transaction 102 prior to transaction ID 214A. When rolling back transaction 102 corresponding to transaction ID 214B, the transaction manager 104 can find record 112B by directly referring to pointer 202 to identify the head of chain 110 and traversing records 112A and 220 until record 112B, which has the corresponding transaction ID 214B, is found. Subsequently, record 112B is deleted, and pointer 222A is modified to have the same address as pointer 219B. In some embodiments, when transaction 102 commits, the commit identifier 215 of record 112 is set to reflect the commit, marking record 112 as ready to be flushed to persistent storage in database 108. Records 112 are later scanned by the transaction manager 104 process to identify which records 112 have the commit identifier 215 and determine which records 112 can be flushed to database 108.

[0031] Once the key / value record 112 is successfully flushed to persistent storage, in some embodiments, the transaction manager 104 sets a purge flag 216 to indicate that the record 112 is ready to be erased from the buffer data structure 106. In some embodiments, the purge engine may read this flag 216 to determine whether the record 112 should be erased from the buffer data structure 106.

[0032] In some embodiments, when two different keys (e.g., keys 212A and 213C) produce the same hash value (i.e., a hash collision occurs) and therefore share the same hash bucket 124, the collision record 220 is used to append record 112 to chain 110. In various embodiments, the size of the hash table 120 is chosen to have a sufficient number of hash buckets 124 to minimize the likelihood of collisions. However, if a hash collision occurs, record 220 is inserted to include a pointer 222 to record 112 having different keys 212. Often, the hash bucket latch 126 is specific to a single key 212, but in such cases, the hash bucket latch 126 is associated with multiple different keys 212.

[0033] As described above, in some embodiments, individual records 112 also include their own respective locks 217, providing additional coherency control. In some embodiments, individual lock records 230 are also inserted into the record chain 110, creating locks that are associated with a specific key when no corresponding value exists.

[0034] The skip list pointer 218 is, in one embodiment, a pointer that forms a skip list 140. As illustrated in Figure 3, the pointer 218 within a given record 112 forms a pointer tower that implements a linked hierarchy of data record sequences, in which case each consecutive sequence skips fewer records 112 than the previous sequence. In some embodiments, the pointer 218 may also be implemented using an indirect pointer, which links the key / value records 112 together in the skip list 140 without using a direct pointer to the physical address of the record 112. Instead, the pointer 218 refers to a hash bucket 124 that points to the record chain 110 containing the record 112. In various embodiments, using an indirect pointer greatly simplifies pointer management because only one direct pointer needs to be maintained for a given record 112. That is, since the location of the hash bucket 124 remains the same, the indirect pointer is not updated if the record 112 is moved to a later location in the record chain 110, for example.

[0035] Next, moving to Figure 3, a block diagram of the skip list 140 is shown. As described above, in various embodiments, the skip list 140 may be used to maintain the order of keys 212 stored in record 112, which may be used to flush the records 112 of committed transactions 102 in ascending order of keys. In the illustrated embodiment, the skip list pointer 218 in record 112 forms a tower 300 that points to towers 300 in other records 112.

[0036] When a specific key 212 is being searched for in the skip list 140, the traversal of the skip list 140 may begin in the illustrated embodiment at the top of the leftmost tower 300 (the location corresponding to bucket ID 312A1 in the illustrated embodiment), where the key 212 in record 112 is compared with the key being searched for. If a match is found, the searched record 112 is found. If no match is found, the traversal proceeds along the path from the forward pointer 314A to another key record 112 having another key 212 to compare with. If that key 212 is smaller than the key 212 being searched for, the traversal returns to the previous tower 300 and descends to the next level within the tower 300 (the location of bucket ID 312A2 in Figure 3). However, if the key 212 being searched for is larger than the other key 212, the traversal proceeds forward along another pointer 314. This process then continues forward until a match is identified for the searched record 112. An example of this traversal is explained below, along with Figure 4.

[0037] The forward pointer 314 is shown in Figure 3 (and subsequent figures) for ease of understanding, while the skip list pointer 218 is implemented using indirect pointers in some embodiments. In the illustrated embodiment, the skip list pointer 218 is implemented using, in particular, a bucket identifier 312, which points to bucket 124 in the hash table 120, and then to record 112 via pointer 202. Thus, moving along pointer 314C involves following pointer 218 of bucket ID 312A3 to bucket 124, and then moving along pointer 202A to record chain 110 containing record 112 using pointer 218 of bucket 312B1. Although not illustrated, in some embodiments the skip list 140 also includes a set of reverse pointers, which can be implemented in a similar manner and will be described in more detail below.

[0038] Before describing high-speed scanning algorithms, it is useful to consider how low-efficiency scanning algorithms are implemented.

[0039] Moving on to Figure 4, a block diagram of the slow scan 400 is shown. As will be explained, the slow scan 400 is a less efficient algorithm than the fast scan algorithms described herein because it may use far more memory accesses to scan a particular key 212. As shown in Figure 4, the exemplary skip list 140 may consist of records 112 sorted in the order of keys 212A to W. The skip list 140 includes eight levels of forward pointers 314 (indicated as levels 1 to 8) that allow movement in ascending order of keys, and another level of reverse pointers (indicated as level -1) that allows movement in descending order of keys. Sentinel towers 300 are located at both ends of the skip list 140 and do not correspond to records 112 (and are therefore shown to have keys -∞ and ∞). Furthermore, below each key 212 in Figure 4 is the bucket identifier 312 of the bucket 124, which contains a direct pointer 202 to its record 112 (or its record chain 110). Thus, as shown, the bucket 124 having the bucket identifier 312 19 contains a pointer 202 to the record 112 having the key 212 A.

[0040] In the example shown in Figure 4, a slow scan 400 is being performed to find record 112 (or simply "record S") with key 212, namely S. As illustrated, scan 400 begins with a skip list traversal at the top of the left sentinel tower 300, where the first memory access is performed to read a skip list pointer 218 at level 8 containing bucket ID 312, namely 20. A second memory access is then performed to read record 112, pointed to by bucket #20, which is record 112 with key K. Since key S is higher in key order than key K, the traversal continues along level 8, and in this case, record W is read during the third memory access. Since key S is lower in key order than key W, the traversal returns to record K on the fourth memory access, reading a skip list pointer 218 at level 7, and descending to the next level.

[0041] As can be understood, this process continues with 20 more memory accesses until record R is identified, which has a pointer 218 to bucket #17 of the record, but this does not include additional memory accesses for using indirect pointers or multiple accesses for moving down the record chain 110. Furthermore, the slow scan 400 may be performed multiple times to insert multiple records 112 associated with a given transaction. In addition, in some embodiments, the skip list 140 may include even higher skip list towers 300 (e.g., one with 33 levels) and may be considerably wider. All of these memory accesses can impact system performance. Often, the fast scan algorithms described below use far fewer memory accesses.

[0042] Moving on to Figure 5, the fast scan 500 is shown. As described above, the fast scan 500 is an algorithm used by the scan engine 150 to scan the skip list 140 more quickly in response to receiving a request to identify a relevant location for a given key. Although illustrated as including steps 510-540, the fast scan 500 may include more (or fewer) steps than those illustrated. For example, if the scan 500 is performed for a purpose other than record insertion, step 540 may be omitted.

[0043] In the illustrated embodiment, the high-speed scan 500 begins in step 510, in which the scan engine 150 identifies an anchor range in the skip list 140 that includes the relevant locations of the particular key, using a specific key prefix. As described above, an “anchor range” is a set / range of records in the skip list that have the same key prefix and is used to anchor where the skip list scan begins. As illustrated below with reference to Figures 6A and 6B, the scan engine 150 may use key prefixes of different lengths to generate anchor ranges of varying widths so that the scan can begin from the narrowest available anchor range. As illustrated below with reference to Figures 7A and 7B, the scan engine 150 identifies an anchor range for a particular key prefix using an index that associates the key prefix with the anchor range by including a pointer to each of those ranges in the “anchor record”. do In some embodiments, the anchor record for a particular anchor range is record 112, which has the lowest key 212 for that anchor range but still has the same prefix. Once anchor record 112 is found, the scan engine 150 can use it as a starting point to begin scanning from anchor record 112 to find the relevant location.

[0044] In step 520, the scan begins with the scan engine 150 "climbing mountain" from the anchor record to record 112, which has a skip list tower 300 that overshoots the relevant location. As illustrated below with reference to Figure 8, the anchor record may not contain a tower that is tall enough to allow the relevant location to be determined using only skip list traversal. Therefore, the scan engine 150 may scan forward from the anchor record using the highest available tower pointer until a key / value record 112 containing a skip list tower 300 that overshoots the location is identified using the pointer 314 (i.e., the key / value record 112 pointed to by the pointer 314 has a key that is later in the key order than a particular key). In some cases, the relevant location may be identified as part of this forward scan without performing step 530.

[0045] In step 530, once the overshooting tower 300 is identified, the scan engine 150 performs a local skip list traversal from the overshooting tower 300 to identify the relevant location of a particular key. In various embodiments, this local skip list scan may be implemented in a similar manner to the slow scan 400, but does not need to be initiated at the sentinel tower. If the scan 500 is being performed to access record 112, the scan 500 may terminate after the traversal has identified its location and the scan engine 150 has accessed the contents of record 112. If record 112 has been inserted at that location, the scan 500 may proceed to step 540 to determine how the skip list 140 should be modified to facilitate the insertion.

[0046] In step 540, the scan engine 150 walks backward from the identified location to identify the associated skip list pointer 314 for record insertion. As illustrated with Figures 9A and 9B, the scan engine 150 sequentially scans backward until it identifies an overshooting pointer for each level in the tower 300 of the record 112 being inserted. Once all associated pointers 314 have been identified, the scan engine 150 then inserts the new record 112 at the determined location, and the new record 112 may include towers 300 with the identified pointers 314. The scan engine 150 then updates the associated pointers 314 in the other towers 300 to point to the newly inserted record 112.

[0047] In many cases, a fast scan 500 will have far fewer memory accesses than a slow scan 400 because the skip list traversal is performed only on a small subset of the total 112 records in the skip list 140. While some additional memory accesses may be performed to facilitate this local skip list traversal, techniques such as climbing mountain or reverse walking for the relevant pointer identification ensure that the number of additional memory accesses is often far less than that required to perform a slow scan 400.

[0048] Here are various examples of how to divide the key space of the skip list 140 using key prefixes.

[0049] Moving to Figure 6A, a block diagram of the key space partitioning 600A is shown. As illustrated and previously described, the skip list 140 contains a set of records 112 that are accessible from buckets 124, each having a key 212 and various bucket identifiers 302. The records 112 are also arranged in key order and each has a skip list tower 300. In the illustrated embodiment, the key space partitioning 600A partitions the key space using two different key prefixes 610, namely a short key prefix 610A and a long key prefix 610B. As recognized, the short key prefix 610A becomes a wider anchor range 620A with a larger number of records 112. In the illustrated example, since key 212 "DEBHQTT" is the lowest key of the "DE" key prefix 610A, the range 620A has anchor record 622A in bucket 19. The long key prefix 610B has fewer records 112 and a narrower anchor range 620B, as key 212 "DEJKEVM" is the lowest-order key of the "DEJK" key prefix 610B. Although anchor ranges 620A and 620B are illustrated as having separate anchor records 622A and 622B, it is worth noting that anchor record 622 can participate in multiple anchor ranges 620 based on different length prefixes 610 of key 212 of record 112. For example, anchor record 622A is also the anchor record for the longer "DEBH" key prefix 610.

[0050] In the example shown in Figure 6A, the scan engine 150 performs a scan for position 624 corresponding to key 212 "DEJKSBC" of record 112 inserted in the skip list 140. In various embodiments, the scan engine 150 teethFirst, the longest available prefix 610 for a particular key 212 is selected to determine if a corresponding anchor range 620 exists in the skip list 140, as this may result in fewer records being scanned. In the illustrated example, the long key prefix 610B "DEJK" does not have a corresponding anchor range 620B, and therefore the scan engine 150 can begin scanning from anchor record 622B. However, if the long key prefix 610B is "DEJA", in this example, there is no corresponding anchor range 620 because no other records 112 have this prefix. In various embodiments, in response to determining that no corresponding anchor range 620 exists in the skip list 140 for the longest prefix 610, the scan engine 150 selects the next longest prefix 610 for the particular key 212. In this example, since "DE" is still a component of key 212, the scan engine 150 can fall back and use a shorter key prefix 610A. Scanning from anchor record 622A via a wider anchor range 620A will scan more records than scanning from a narrower anchor range 620B, but the number of records scanned is still often far less than that scanned when performing a slow scan 400. If a corresponding anchor range does not exist (for example, the key has the prefix 610 "DFAC"), the scan engine 150 may fall back further and use a slow scan 400 to identify location 624. However, once this record 112 is inserted, it can become anchor record 622, which in the future will enable fast scanning of both DF prefix 610 and DFAC prefix 610.

[0051] While segmentation 600A uses the same key prefix length across the entire key space, in some embodiments, different key prefix lengths may be used for different parts of the key space, as will be discussed below.

[0052] Moving to Figure 6B, a block diagram of another key space partitioning 600B is shown. In the illustrated embodiment, different parts of the key space may use different anchor plans 650 to partition the key space using different numbers of prefixes 610. For example, as shown, anchor plan 650A using three different prefixes 610 is used for keys 212 having initials A through L, anchor plan 650B using four different prefixes 610 is used for keys 212 having initials M through S, and anchor plan 650C using two different prefixes 610 is used for keys 212 having initials T through Z. In such an embodiment, the scan engine 150 may first access the anchor plans 650 to determine what length of prefixes should be used for a given key 212. For example, if a scan is being performed on key 212 which has the initial letter G, the scan engine 150 may determine from anchor plan 650A that it should use a 10-character initial prefix 610, followed by an 8-character prefix, and then a 3-character prefix.

[0053] In various embodiments, different anchor plans 650 may be selected based on the underlying distribution of keys 212 in the skip list 140. For example, portions of the key space with a greater concentration of keys 212 may use an additional prefix 610. In some embodiments where the database system 10 implements a multitenant database, the scan engine 150 may use different anchor plans 650 for different tenants. Furthermore, prefixes 610 may be selected to group records 112 that have common attributes (e.g., are included in the same database table) into the same anchor range 620. In some embodiments, the scan engine 150 may also change the anchor plan 650 over time to improve performance. For example, in one embodiment, the scan engine 150 may use a machine learning algorithm to determine the appropriate anchor plan 650 for various partitions of the key space. However, such a change may not be disruptive to the database system 10 because the scan engine 150 may initially fall back to using a slow scan 400 to insert records 112 until a sufficient number of records 112 have been inserted based on the new anchor plan 650 and the fast scan 500 becomes executable again. Since the temporary performance degradation due to changing the anchor plan is likely to affect only a small number of scans, such a change may not be a major problem.

[0054] Moving to Figure 7A, a block diagram of the anchor record identifier 700 is shown. As described above, in various embodiments, the scan engine 150 maintains an index that associates the key prefix 610 with key / value records 112 within each anchor range 620 having the same key prefix, and accesses the index to identify the relevant anchor record 622 in which the scan should begin. In the illustrated embodiment, this index is implemented by including the anchor entries 710 in the hash bucket 124. Each anchor entry 710 further includes the bucket identifier 312 of the anchor record 622 and its corresponding key prefix 610. In other embodiments, the index may be implemented differently. For example, the anchor entries 710 may be located elsewhere so as to be appended to the record chain 110, the anchor entries 710 may contain more (or fewer) components, the key prefix 610 may be replaced with a hash value generated from the key prefix 610 (as shown in Figure 7B), and the index may be implemented using a data structure other than the hash table 120, etc.

[0055] In the illustrated embodiment, the scan engine 150 implements anchor record identification 700 by applying a hash function 122 to a given prefix 610 to determine the bucket identifier 312 of the associated hash bucket 124 (e.g., bucket 124N). If a corresponding anchor entry 710 exists in bucket 124 (meaning that a corresponding anchor range 620 exists for its prefix 610), the scan engine 150 reads the bucket identifier 312 containing the anchor record 622 to determine the hash bucket 124 associated with the anchor record 622. The scan engine 150 then uses this bucket identifier 312 to access the hash bucket 124 of the anchor record 612 (e.g., bucket 124A). The scan engine 150 then traverses the pointer in bucket 124 to the anchor chain 110, which contains the anchor record 622. However, if no anchor entry 710 exists in the hash bucket 124 (meaning the corresponding anchor range 620 does not yet exist), the scan engine 150 proceeds with the next largest key prefix 610 for a given key 212, repeating this process until it can identify the associated anchor record 622, or, if it cannot identify an anchor entry 710 for any key prefix 610 of key 212, it proceeds to perform a slow scan 400.

[0056] In various embodiments, the scan engine 150 creates an anchor entry 710 for an anchor range 620 when an initial record 112 is added to the range 620—when the initial record becomes an anchor record 622. When a record 112 with a lower key 212 is added to the range 620 (or when a record 112 with a lower key 212 is deleted), the scan engine 150 updates the anchor record bucket identifier 312 in the anchor entry 710 to point to the hash bucket 124 for these new anchor records 622 (or the next lower key 212 record 112). If each record 112 in a particular anchor range 620 is deleted, the scan engine 150 may remove the corresponding anchor entry 710 from the hash bucket 124. In some cases, if the corresponding record 112 becomes an anchor record 622 for multiple prefixes 610, multiple anchor entries 710 may be modified for a given key 212.

[0057] In some rare cases, a hash collision may cause two prefixes 610 to map to the same hash bucket 124. Therefore, to verify that the anchor entry 710 is correct for the prefix 610 being hashed, the scan engine 150 may compare the prefix 610 being hashed with the key prefix 610 in the anchor entry 710. In some embodiments, if two prefixes 610 are associated with the same hash bucket 124, a given hash bucket 124 may contain multiple anchor entries 710. In another embodiment, for conflicting prefixes 610, the anchor entry 710 may be stored in a conflict record 220 pointed to by the hash bucket 124. In yet another embodiment, other techniques may be used to deal with the collision, such as using the next longest prefix 610 for a particular key 212 being scanned.

[0058] Next, we move to Figure 7B, which shows a block diagram illustrating an example of an anchor record identifier 700. Continuing from the example presented in Figure 6A, the scan engine 150 again attempts to determine the relevant location 624 for the key 212 "DEJKSBC". In this example, the scan engine 150 may first apply the hash function 122 to the long prefix 610B "DEJK" to obtain the bucket identifier 312 "31". Then, the scan engine 150 may compare the hash value of "DEJK" with the hash value 702 "739813" to determine if the anchor entry 710 is correct. If so, the scan engine 150 may read the bucket identifier 312 "6" and then access the corresponding bucket 124, in which case the scan engine 150 can access the anchor record 622 in the narrower range 620 and begin scanning. However, if a corresponding entry 720 does not exist (or the hash value 702 does not match), the scan engine 150 may apply the hash function 122 to the short prefix 610A "DE" to access the corresponding entry 710 to determine the anchor record 622 of the wider anchor range 620A, in which case the scan can be initiated as described below.

[0059] Moving to Figure 8, a block diagram of the "climbing mountain" 800 is shown. Once the scan engine 150 determines an anchor record 622 for a given prefix 610, the scan engine 150 may read the tower 300 in that record 121 to determine one or more pointers 314 in order to scan forward within the anchor range 620 for the relevant position 624. However, in many cases, the tower 300 is not high enough to perform a skip list traversal (i.e., it does not have a pointer 314 pointing to a record 112 located after position 624). The scan engine 150 could sequentially scan the records 112 forward, but this approach is inefficient and does not take advantage of the logarithmic scan provided by the skip list. Therefore, in the illustrated embodiment, the scan engine 150 uses the climbing mountain 800.

[0060] In various embodiments, the technique begins with the scan engine 150 performing a “climb up mountain,” meaning that the scan engine 150 scans forward from anchor record 622 along the highest available pointer 314 in tower 300 of the skip list 140 until it can identify an overshooting tower 300 (i.e., a tower 300 with a pointer 314 that points past position / overshoots position 624). Thus, in the example shown in Figure 8, the scan engine 150 may traverse pointer 314 “17” in anchor record 622, pointer 314 “29” in the next record 112, and pointer 314 “42” in subsequent record 112, because these are the highest available pointers in tower 300 for these records 112. The advantage of “climbing up mountain” is that any intermediate record 112 with a short tower 300 can be skipped, and as a result, a logarithmic scan is performed in an upward climb. In some cases, position 624 may be one of the records 112 being accessed in this upward climb.

[0061] Once an overshooting tower 300 is identified, the scan engine 150 may perform a “climb down the mountain” using a local skip-list traversal 810. In various embodiments, the local traversal 810 is implemented in the same way as the skip-list traversal of the slow scan 400, but the traverse 810 is a “local” performance, meaning it starts from the overshooting tower 300 rather than the sentinel tower 300. Thus, in the illustrated example, in response to determining that pointer 314 “42” points past position 624, the scan engine 150 traverses the next highest pointer 314 “15”, then pointer 314 “9”, and then determines position 624 after the overshoot. Since the local scan 810 also enables logarithmic scanning, the climbing up the mountain and climbing down the mountain processes are efficient approaches to determining the relevant position 624 after the relevant anchor range 620 has been identified.

[0062] If scan 500 is performed to find an existing record 112, scan 500 may terminate with scan engine 150 providing the contents of record 112. However, if scan 500 is performed to insert record 112, scan engine 150 may need to perform additional work to determine which pointers 314 should be included in record 112 and which pointers 314 should be modified in other records 112, as described below.

[0063] Moving to Figure 9A, a block diagram of the "backward walking" 900 is shown. In the slow scan 400, relevant pointers 314 may be determined to facilitate record insertion while the scan 400 is being performed. Since the scan engine 150 cannot access all relevant pointers 314 while performing the climbing mountain 800, the scan engine 150 may take this into consideration when performing the backward walking 900.

[0064] In the illustrated embodiment, the scan engine 150 implements backward walking 900 by sequentially scanning backward along the lowest level in the skip list 140 to identify pointers 314 in other records 112 related to key / value records 112 inserted into the skip list 140. In various embodiments, the scan engine 150 may first determine the height of the new tower 300 for the records 112 being inserted in order to know how many levels there are in the new tower 300 and therefore how many related pointers 314 to insert into the tower 300. In some embodiments, the scan engine 150 determines the height of the tower 300 using a power of 2 for the backoff, meaning that the tower heights of the skip list are allocated logarithmically, such that 1 / 2 is one height (not including the lowest inverse level), 1 / 4 is two heights, and so on. In one embodiment, the scan engine 150 may determine the height of the tower 300 by generating a random number and counting consecutive seconds (or 0 seconds) from the most significant bit (or least significant bit). Once the tower height is determined, the scan engine 150 may sequentially scan backward until it identifies an overshoot pointer 314 for each level in the new tower 300. Thus, in the example shown in Figure 9A, the tower height of four forward pointers 314 is used. Thus, the scan engine 150 may sequentially scan backward until it has pointers 314 "12", "9", "42", and "42" at each of the four levels. Once the relevant pointers 314 are identified, the scan engine 150 may pin the record by recording those relevant pointers 314 and the corresponding bucket identifier 312 of the record 112.

[0065] In some cases, the scan engine 150 may not perform backward walking 900 if the inserted record 112 has a tower 300 of sufficient height (for example, more than 8 levels in some embodiments, but this is rare) and making backward walking 900 a less efficient approach than performing a slow scan 400 to determine a set of associated pointers 314. Thus, in some embodiments, the scan engine 150 may determine the number of pointers 314 contained in the tower 300 of the new record 112 before sequentially scanning in reverse. Based on the determined number, the scan engine 150 decides whether to use a slow scan 400 to traverse the skip list 140 from the sentinel node or to use walking backward 900 to sequentially scan in reverse from the identified location 624.

[0066] Once all relevant pointers 314 have been identified, the scan engine 150 may perform record insertion as described below.

[0067] Next, moving to Figure 9B, a block diagram of record insertion 950 is shown. As illustrated, this involves the scan engine 150 placing the previously pinned associated pointer 314 at position 624 in a new tower 300 within the inserted record 112, so that the new record 112 points to the same previously linked record 112 as the associated pointer 314 in the other record 112. Since the new tower 300 obstructs the pointer 314 in the other record 112, the scan engine 150 may update these pointers 314 to point to the bucket identifier 312 of the inserted record 112, as shown in Figure 9B.

[0068] Since record chain 110 and / or record 112 may be modified during backward walking 900 and record insertion 950, the scan engine 150 may also acquire latches 126 and / or locks 217 of the determined record 112 with associated pointers 314 to prevent modification during walking 900 and insertion 950. Once each of the associated pointers 314 is identified, the scan engine 150 may further perform verification of the pointers 314 before performing code insertion 950 to ensure that simultaneous record modification or another record insertion does not interfere with the record insertion 950.

[0069] As described above, in various embodiments, the scan engine 150 may also need to add or update one or more anchor entries 710 in response to the inserted record 112 becoming the initial record 112 of the anchor range 620, or having one or more new lowest keys 212 of the anchor range 620.

[0070] The following describes various methods that use one or more of the techniques mentioned above.

[0071] Next, moving to Figure 10A, a flowchart of Method 1010 for high-speed skip list scanning is shown. Method 1010 is one embodiment of a method performed by a computing system such as a database system 10, which is running a scan engine 150. In some cases, Method 1010 can improve the performance of scanning skip lists.

[0072] In step 1015, the skip list (e.g., skip list 140) is stored containing multiple key / value records (e.g., key / value record 112) which include one or more pointers (e.g., skip list pointers 218 / 314) to other key / value records among the multiple key / value records. In some embodiments, the skip list maintains the order of the keys of the key / value records, which are stored in a buffer data structure that stores data for the active database transaction.

[0073] In step 1020, the skip list is scanned for positions (position 624) associated with a specific key.

[0074] In substep 1022, the scanning step includes identifying a specific portion of the skip list (e.g., anchor range 620) using a specific key prefix (e.g., key prefix 610), where the specific portion contains key / value records having the same prefix. In various embodiments, an index is maintained that associates the key prefix with the key / value records within each portion having the same key prefix, and using the prefix includes accessing the index to identify the initial key / value record (e.g., anchor record 612) within the specific portion from which the scan should begin. In some embodiments, accessing the index includes applying a hash function (e.g., hash function 122) to the prefix to identify a pointer to the initial key / value record within the specific portion (e.g., bucket identifier 312) and traversing the pointer to the initial key / value record within the specific portion (e.g., local traverse 810). In some embodiments, a first prefix (e.g., a short key prefix 610A) and a second longer prefix (e.g., a long key prefix 610B) of a particular key are selected, and it is determined whether relevant portions of the skip list (e.g., a wider anchor range 620A and a narrower anchor range 620B) exist for the first and second prefixes, and whether relevant portions exist within the skip list for the second longer prefix. do not In response to this determination, the process includes selecting a portion related to the second prefix for use in scanning for location. In some embodiments, the length of the prefix used is determined by accessing a set of anchor plans (e.g., anchor plan 650) that specify the prefix length to be used based on a particular key portion.

[0075] In substep 1024, the scanning step includes initiating a scan for a location within an identified portion. In various embodiments, the scan includes scanning forward for key / value records within a particular portion along the highest available pointer in the skip list tower of key / value records (e.g., Climbing Mountain 800) until a key / value record is identified that includes a pointer to another key / value record located after a location associated with a particular key in key order (e.g., within the Overshooting Tower 300 in Figure 8), and traversing the skip list from the identified key / value record (e.g., Local Traversal 810).

[0076] In some embodiments, method 1010 further includes the step of inserting a key / value record (e.g., inserted record 112 in Figure 9B) into the skip list at a location associated with a particular key, in response to the scan identifying a location. In some embodiments, method 1010 further includes the step of determining a set of pointers to be included in the inserted key / value leocode (e.g., for the new tower 300 in Figure 9B) by sequentially scanning backward from the identified location (e.g., walking backward 900) and identifying key / value records containing pointers that point to key / value records located after the identified location in key order. In some embodiments, the number of pointers in the set (e.g., the height of the new tower 300) is determined before sequentially scanning backward. Based on the determined number, it is determined whether to traverse the skip list from the sentinel node (e.g., the sentinel tower 300 in Figure 4) or sequentially scan backward from the identified location.

[0077] Next, moving to Figure 10B, a flowchart of another method 1030 for high-speed skip list scanning is shown. Method 1030 is another embodiment of the method performed by a computer system such as a database system 10, which may be running a scan engine 150. In some cases, performing method 1030 may reduce the burden of scanning the skip list.

[0078] In step 1035, a skip list (e.g., skip list 140) is stored that maintains the order of the keys (e.g., key 212) of key / value records (e.g., record 112) in a database (e.g., database 108). In various embodiments, the skip list maintains the order of the keys for key / value records of database transactions that are awaiting commit by the database.

[0079] In step 1040, a location in the skip list associated with a particular key (e.g., location 624) is determined. In substep 1042, the determining step includes identifying a range in the skip list (e.g., anchor range 620) using a prefix of the particular key (e.g., key prefix 610), where the range includes key / value records having the same prefix key. In substep 1044, the determining step includes starting a scan of locations within the identified range. In some embodiments, the determining step includes scanning the identified range forward using a pointer that overshoots locations until a key / value record containing a skip list tower (e.g., overshoot tower 300 in Figure 8) is identified, and traversing the skip list from the skip list tower (e.g., local traverse 810). In some embodiments, the determination step includes, in response to determining the location, sequentially scanning backward from the location in the skip list (e.g., backward walking 900) to identify a pointer (e.g., the relevant skip list pointer 218 in Figure 9A) that should be included in a tower of key / value records being inserted into the skip list (e.g., the new tower 300 in Figure 9B), and inserting the key / value record at the determined location using the tower containing the identified pointer. In some embodiments, the determination step includes applying a hash function (e.g., hash function 122) to a prefix to identify a hash table (e.g., hash table 120) containing a pointer (e.g., bucket identifier 312) to an initial key / value record in the range (e.g., anchor record 622), and starting the scan from the initial key / value record in the range.

[0080] Next, moving to Figure 10C, a flowchart of method 1060 for high-speed skip list scanning is shown. Method 1060 is another embodiment of the method performed by a computer system such as database system 10, which may be running a scan engine 150. In some cases, performing method 1060 may reduce the time required to scan the skip list and / or insert records into the skip list.

[0081] In step 1065, a skip list (e.g., skip list 140) is maintained to store the order of keys of a plurality of key / value records (e.g., record 112). In some embodiments, the first key / value record of the plurality of key / value records in the skip list indirectly points to the second key / value record of the plurality of key / value records by including a first pointer (e.g., bucket identifier 312) in a bucket of a hash table (e.g., hash table 120), where the bucket contains a second pointer to the second key / value record.

[0082] In step 1070, the skip list is divided into sections (e.g., anchor range 620) based on the key prefix (e.g., prefix 610) (e.g., key space partitioning 600). In some embodiments, the partitioning includes maintaining a hash table (e.g., hash table 120) that associates the prefix with a pointer to a section in the skip list (e.g., anchor record bucket identifier 312).

[0083] In step 1075, the location associated with a particular key (e.g., location 624) is determined by initiating a scan of locations within the section corresponding to the prefix of the particular key. In various embodiments, the determining step includes scanning forward within the section using a pointer that points beyond the location until a key / value record containing a skip list tower (e.g., the overshooting tower 300 in Figure 8) is identified (e.g., climbing up in Figure 8), and then performing a skip list traversal (e.g., local traversal 810) from the identified key / value record to determine the location.

[0084] In some embodiments, method 1060 further includes scanning backward along the lowest level of the skip list (e.g., backward walking 900) for key / value records to be inserted into the skip list tower (e.g., the new tower 300 in Figure 9B), and inserting the key / value records into the skip list using the skip list tower.

[0085] Exemplary multitenant database system Moving to Figure 11, an exemplary multi-tenant database system (MTS) 1100 is shown, which can implement functionality as a database system 10. In the illustrated embodiment, the MTS 1100 includes a database platform 1110, an application platform 1120, and a network interface 1130 connected to a network 1140. The database platform 1110 includes data storage 1112 and a set of database servers 1114A-N that interact with the data storage 1112, and the application platform 1120 includes a set of application servers 1122A-N, each having its own environment 1124. In the illustrated embodiment, the MTS 1100 is connected to various user systems 1150A-N via the network 1140. In other embodiments, the technology of this disclosure is implemented in non-multi-tenant environments such as client / server environments, cloud computing environments, clustered computers, etc.

[0086] In various embodiments, the MTS 1100 is a set of computer systems that together provide various services to users (alternatively also called “tenants”) who interact with the MTS 1100. In some embodiments, the MTS 1100 implements a customer relationship management (CRM) system that provides a mechanism for tenants (e.g., businesses, government agencies, etc.) to manage relationships and interactions with customers and potential customers. For example, the MTS 1100 may enable tenants to store customer contact information (e.g., customer websites, email addresses, telephone numbers, and social media data), identify sales opportunities, record service issues, and manage marketing campaigns. Furthermore, the MTS 1100 may enable these tenants to identify how customers have communicated, what customers have purchased, when customers last purchased goods, and what customers have paid for. To provide CRM system services and / or other services, the MTS 1100 includes a database platform 1110 and an application platform 1120, as illustrated.

[0087] The database platform 1110 is, in various embodiments, a combination of hardware elements and software routines that implement a database service for storing and managing data of the MTS 1100, including tenant data. As shown in the figure, the database platform 1110 includes data storage 1112. In various embodiments, the data storage 1112 includes a set of storage devices (e.g., solid-state drives, hard disk drives, etc.) that are connected together over a network (e.g., a storage-attached network (SAN)) and configured to store data redundantly to prevent data loss. In various embodiments, the data storage 1112 is used to implement a database 108 that includes a collection of information organized to enable access, storage, and manipulation of the information. The data storage 1112 may implement a single database, a distributed database, a collection of distributed databases, a database with redundant online / offline backups or other redundancy, etc. As part of implementing the database, the data storage 1112 may store one or more database records 112, each having a data payload (e.g., the values ​​of fields in a database table) and metadata (e.g., key values, timestamps, table identifiers for tables associated with the record, tenant identifiers for tenants associated with the record, etc.).

[0088] In various embodiments, database records 112 may correspond to rows in a table. A table typically contains one or more data categories, logically arranged as columns or fields in a displayable schema. Thus, each record in a table may contain an instance of data for each category defined by its fields. For example, a database may include a table describing customers, with fields for basic contact information such as name, address, telephone number, fax number, etc. Thus, a record in that table may contain the value of each field in the table (e.g., the name in the name field). Another table may describe purchase orders, with fields for information such as customer, product, selling price, date, etc. In various embodiments, standard entity tables are provided for use by all tenants, such as tables for account, contact, lead, and opportunity data, each containing predefined fields. The MTS 1100 may store database records for one or more tenants within the same table, i.e., tenants may share a table. Thus, in various embodiments, a database record may include a tenant identifier indicating the owner of the database record. As a result, a tenant's data is kept secure and separate from other tenants' data, so that a tenant does not have access to another tenant's data unless such data is explicitly shared.

[0089] In some embodiments, the data stored in data storage 1112 includes a buffer data structure 106 and persistent storage organized as part of a log structured merge tree (LSM tree). As described above, the database server 1114 may first write database records to the local in-memory buffer data structure 106 before later flushing those records to persistent storage (e.g., within data storage 1112). As part of flushing database records, the database server 1114 may write the database records 112 to a new file located at the "top" level of the LSM tree. Over time, as database records are moved to lower levels of the LSM tree, the database records may be rewritten by the database server 1114 to a new file located at a lower level. In various implementations, as database records become old and are moved down the LSM tree, they are moved to a slower storage device in data storage 1112 (e.g., from a solid-state drive to a hard disk drive).

[0090] When database server 1114 wants to access a database record for a particular key, database server 1114 may traverse different levels of the LSM tree for a file that potentially contains the database record for that particular key. If database server 1114 determines that a file may contain the relevant database record, database server 1114 may fetch the file from data storage 1112 into database server 1114's memory. Database server 1114 may then check the fetched file for a database record 112 having a particular key 212. In various embodiments, the database record 112 is immutable once written to data storage 1112. Therefore, if database server 1114 wants to modify the value of a table row (which can be identified from the accessed database record), database server 1114 writes the new database record 112 to buffer data structure 106, which is erased at the top level of the LSM tree. Over time, the database record 112 is merged into lower levels of the LSM tree. Therefore, the LSM tree may store various database records 112 for a database key 212, in which case older database records 112 for that key 212 are located at a lower level in the LSM tree than newer database records.

[0091] In various embodiments, the database server 1114 is a hardware element, software routine, or a combination thereof that can provide database services such as data storage, data retrieval, and / or data manipulation. Such database services may be provided by the database server 1114 to components within the MTS 1100 (e.g., application server 1122) and to components outside the MTS 1100. For example, the database server 1114 may receive a database transaction request from application server 1122 requesting to write data to or read data from data storage 1112. The database transaction request may specify an SQL SELECT command to select one or more rows from one or more database tables. The contents of the rows may be defined in database records, and the database server 1114 may find and return one or more database records corresponding to the selected one or more table rows. In various cases, the database transaction request may instruct the database server 1114 to write one or more database records for an LSM tree, which the database server 1114 maintains in the database platform 1110. In some embodiments, the database server 1114 implements a relational database management system (RDMS) or an object-oriented database management system (OODBMS) to facilitate the storage and retrieval of information to and from the data storage 1112. In various cases, the database servers 1114 may communicate with each other to facilitate the processing of transactions. For example, database server 1114A may communicate with database server 1114N to determine whether database server 1114N has written a database record for a particular key to its in-memory buffer.

[0092] The application platform 1120 is, in various embodiments, a combination of hardware elements and software routines that implements and runs CRM software applications and provides relevant data, codes, forms, web pages, and other information to and from the user system 1150, and provides relevant data, objects, web page content, and other tenant information via the database platform 1110. To facilitate these services, in various embodiments, the application platform 1120 communicates with the database platform 1110 to store, access, and manipulate data. In some cases, the application platform 1120 may communicate with the database platform 1110 over different network connections. For example, one application server 1122 may be connected over a local area network, and another application server 1122 may be connected over a direct network link. The Transport Control Protocol and Internet Protocol (TCP / IP) are exemplary protocols for communication between the application platform 1120 and the database platform 1110, but it will be apparent to those skilled in the art that other transport protocols may be used depending on the network interconnection used.

[0093] The application server 1122 is, in various embodiments, a hardware element, software routine, or a combination thereof that can provide services to the application platform 1120, including processing requests received from tenants of the MTS 1100. In various embodiments, the application server 1122 can give rise to environments 1124 that can be used for various purposes, such as providing functionality for developers to develop, run, and manage applications. Data can be transferred to environment 1124 from another environment 1124 and / or from the database platform 1110. In some cases, environment 1124 cannot access data from other environments 1124 unless the data is explicitly shared. In some embodiments, multiple environments 1124 can be associated with a single tenant.

[0094] The application platform 1120 may provide the user system 1150 with access to multiple different hosted (standard and / or custom) applications, including CRM applications and / or applications developed by tenants. In various embodiments, the application platform 1120 may manage application creation, application testing, application storage in database objects in data storage 1112, application execution in the environment 1124 (e.g., virtual machines in process space), or any combination thereof. In some embodiments, the application platform 1120 may add and remove application servers 1122 from the server pool at any time for any reason, even if there is no server affinity with a particular application 1122 for the user and / or organization. In some embodiments, an interface system (not shown) implementing load balancing functionality (e.g., an F5 Big-IP load balancer) may be placed between the application server 1122 and the user system 1150 and configured to distribute requests to the application server 1122. In some embodiments, the load balancer routes user requests to the application server 1122 using a least connection algorithm. Other examples of load balancing algorithms, such as round-robin or observed response time, can also be used. For example, in one embodiment, three consecutive requests from the same user may be directed to three different servers 1122, and three requests from different users may be directed to the same server 1122.

[0095] In some embodiments, the MTS 1100 uses security mechanisms such as encryption to maintain each tenant's data individually unless the data is shared. If two or more servers 1114 or 1122 are used, they may be located in close proximity to each other (e.g., in a server farm located within a single building or campus), or they may be distributed at distant locations (e.g., one or more servers 1114 located in City A and one or more servers 1122 located in City B). Thus, the MTS 1100 may include one or more logically and / or physically connected servers distributed locally or across one or more geographical locations.

[0096] One or more users (for example, via user system 1150) may interact with MTS 1100 via network 1140. User system 1150 may correspond to, for example, a tenant of MTS 1100, an MTS 1100 provider (e.g., an administrator), or a third party. Each user system 1150 may be a desktop personal computer, workstation, laptop, PDA, mobile phone, or any Wireless Access Protocol (WAP) enabled device, or any other computing device that can interface directly or indirectly to the Internet or other network connectivity. User system 1150 may include dedicated hardware configured to interface with MTS 1100 via network 1140. User system 1150 may run a graphical user interface (GUI) that corresponds to MTS 1100, an HTTP client (e.g., a browsing program such as Microsoft® Internet Explorer® browser, Netscape Navigator® browser, Opera browser, or a WAP-enabled browser for mobile phones, PDAs, or other wireless devices), or both, enabling users of user system 1150 (e.g., subscribers to a CRM system) to access, process, and display information and pages available from MTS 1100 via network 1140. Each user system 1150 may include one or more user interface devices, such as a keyboard, mouse, touchscreen, or pen, for interacting with the graphical user interface (GUI) provided by the browser on a display monitor screen, LCD display, etc., along with pages, forms, and other information provided by MTS 1100 or other systems or servers. As stated above, the disclosed embodiments are suitable for use with the Internet, which refers to a specific global internet network.However, please understand that other networks may be used instead of the internet, such as intranets, extranets, virtual private networks (VPNs), non-TCP / IP based networks, and any LAN or WAN.

[0097] Because users of user system 1150 may have different levels of capability, the capability of a particular user system 1150 may be determined by one or more permission levels associated with the current user. For example, when a sales representative uses a particular user system 1150 to interact with MTS 1100, that user system 1150 may have the capabilities assigned to that sales representative (e.g., user privileges). However, when an administrator uses the same user system 1150 to interact with MTS 1100, user system 1150 may have the capabilities assigned to that administrator (e.g., administrative privileges). In a system using a hierarchical role model, a user with a certain level of access permission may have access to applications, data, and database information accessible by users with lower access permission levels, but may not have access to certain applications, database information, and data accessible by users with higher access permission levels. Therefore, different users may have different capabilities regarding access to and modification of applications and database information, depending on their security or permission level. Additionally, there may be several data structures assigned at the tenant level and managed by the MTS 1100, while other data structures are managed at the user level.

[0098] In some embodiments, the user system 1150 and its components can be configured using a browser-like application that includes computer code executable on one or more processing elements. Similarly, in some embodiments, the MTS 1100 (and any additional instances of the MTS, if two or more MTS exist) and their components can be operator-configured using an application that includes computer code executable on processing elements. Thus, the various operations described herein can be performed by executing program instructions stored on a non-temporary computer-readable medium and executed by the processing elements. The program instructions may be stored on a non-volatile medium such as a hard disk, on any other known volatile or non-volatile storage medium or device such as ROM or RAM, or provided on any medium from which program code can be initiated, such as a compact disc (CD), a digital versatile disc (DVD), a floppy disk, etc. In addition, the entire program code or any part thereof may be transmitted and downloaded from the software source, for example, via the Internet, or from another server as is well known, or transmitted, as is well known, via any other conventional network connection (e.g., extranet, VPN, LAN, etc.) using any communication medium and protocol (e.g., TCP / IP, HTTP, HTTPS, Ethernet®, etc.). It will also be understood that computer code for implementing aspects of the disclosed embodiments may be implemented in any programming language executable on a server or server system, such as C, C++, HTML, Java®, JavaScript®, or any other scripting language such as VBScript.

[0099] Network 1140 may be a LAN (Local Area Network), a WAN (Wide Area Network), a wireless network, a point-to-point network, a star network, a token ring network, a hub network, or any other suitable configuration. The global internetwork of the network is often referred to as "Internet," using a capital "I," and is an example of a TCP / IP (Transportation Control Protocol and Internet Protocol) network. However, it should be understood that the disclosed embodiments may utilize any of various other types of networks.

[0100] User system 1150 may communicate with MTS 1100 using TCP / IP, and at higher network levels, it may communicate using other common internet protocols such as HTTP, FTP, AFS, WAP, etc. For example, if HTTP is used, user system 1150 may include an HTTP client, commonly called a "browser," in MTS 1100 to send and receive HTTP messages from an HTTP server. Such a server may be implemented as a single network interface between MTS 1100 and network 1140, although other techniques may be used similarly or alternatively. In some implementations, the interface between MTS 1100 and network 1140 includes load-sharing functionality, such as a round-robin HTTP request distributor, to balance the load across multiple servers and evenly distribute incoming HTTP requests.

[0101] In various embodiments, the user system 1150 communicates with the application server 1122 to request and update system-level and tenant-level data from the MTS 1100, which can request one or more queries to the data storage 1112. In some embodiments, the MTS 1100 automatically generates one or more SQL statements (SQL queries) designed to access the desired information. In some cases, the user system 1150 may generate requests having a specific format corresponding to at least a portion of the MTS 1100. For example, the user system 1150 may request to move data objects to a particular environment 1124 using object notation that describes object relation mappings (e.g., JavaScript object notation mappings) of a specified set of objects.

[0102] Example computer system Next, moving to Figure 12, a block diagram of an exemplary computer system 1200 is shown, which may implement the functionalities described herein, such as a database system 10, a part of the database system 10, or a client interacting with the database system 10. The computer system 1200 includes a processor subsystem 1280 which is coupled to system memory 1220 and an I / O interface 1240 via an interconnect 1260 (e.g., a system bus). The I / O interface 1240 is coupled to one or more I / O devices 1250. The computer system 1200 may be any of various types of devices, including, but is not limited to, a server system, a personal computer system, a desktop computer, a laptop or notebook computer, a mainframe computer system, a tablet computer, a handheld computer, a workstation, a network computer, a mobile phone or music player, or a consumer device such as a personal data assistant (PDA). For convenience, a single computer system 1200 is illustrated in Figure 12, but system 1200 may also be implemented as two or more computer systems operating together in a cluster.

[0103] The processor subsystem 1280 may include one or more processors or processing units. In various embodiments of the computer system 1200, multiple instances of the processor subsystem 1280 may be coupled to the interconnect 1260. In various embodiments, the processor subsystem 1280 (or each processor unit within 1280) may include a cache or other form of onboard memory.

[0104] System memory 1220 is available for storing program instructions that can be executed by the processor subsystem 1280 in order to cause the system 1200 to perform various operations described herein. System memory 1220 can be implemented using different physical non-temporary memory media, such as hard disk storage, floppy disk storage, removable disk storage, flash memory, random access memory (RAM-SRAM, EDO RAM, SDRAM, DDR SDRAM, RAMBUS RAM, etc.), read-only memory (PROM, EEPROM, etc.), etc. Memory within the computer system 1200 is not limited to primary storage such as memory 1220. Rather, the computer system 1200 may also include other forms of storage, such as cache memory within the processor subsystem 1280 and secondary storage on the I / O device 1250 (e.g., hard drives, storage arrays, etc.). In some embodiments, these other forms of storage may also store program instructions that can be executed by the processor subsystem 1280 in order to cause the system 1200 to perform operations described herein. In some embodiments, the memory 1220 may include a transaction manager 104, a scan engine 150, a buffer data structure 106, and / or a portion of the database 108.

[0105] The I / O interface 1240 may be any of various types of interfaces configured to couple with and communicate with other devices, according to various embodiments. In one embodiment, the I / O interface 1240 is a bridge chip (e.g., a southbridge) from a front-side bus to one or more back-side buses. The I / O interface 1240 is coupled with one or more I / O devices 1250 via one or more corresponding buses or other interfaces. Examples of I / O devices 1250 include storage devices (hard drives, optical drives, removable flash drives, storage arrays, SANs, or controllers associated therewith), network interface devices (to local or wide area networks), or other devices (graphics, user interface devices, etc.). In one embodiment, the computer system 1200 is coupled with a network via the network interface device 1250 (e.g., configured to communicate via WiFi, Bluetooth®, Ethernet®, etc.).

[0106] While specific embodiments have been described above, these embodiments are not intended to limit the scope of the disclosure, even if only a single embodiment is described with respect to a particular feature. Examples of features provided in this disclosure are intended to be illustrative, not restrictive, unless otherwise specified. The above description is intended to cover alternatives, modifications, and equivalents that would be obvious to those skilled in the art who are interested in this disclosure.

[0107] The scope of this disclosure includes any feature or combination of features disclosed herein (whether express or implicit), or any generalization thereof, whether or not it mitigates any or all of the issues addressed herein. Therefore, new claims may be created for any such combination of features during the proceedings of this application (or any application claiming priority thereto). In particular, with reference to the appended claims, features from dependent claims may be combined with features from independent claims, and features from each independent claim may be combined in any suitable manner, not only in the specific combinations enumerated in the appended claims.

[0108] This disclosure includes references to “one embodiment” or a group of “embodiments” (e.g., “several embodiments” or “various embodiments”). Embodiments are different implementations or instances of the disclosed concepts. References to “embodiments,” “one embodiment,” “a particular embodiment,” etc., do not necessarily refer to the same embodiment. Numerous possible embodiments, including those specifically disclosed, and modifications or substitutions within the spirit or scope of this disclosure are intended.

[0109] This disclosure may discuss potential benefits that may arise from the disclosed embodiments. Not all implementations of these embodiments will necessarily reveal any or all of the potential benefits. Whether a benefit is realized for a particular embodiment depends on many factors, some of which are outside the scope of this disclosure. In fact, there are several reasons why an implementation within the claims may not present some or all of any of the disclosed benefits. For example, a particular implementation may include other circuits outside the scope of the disclosure that negate or diminish one or more of the disclosed benefits, along with one of the disclosed embodiments. Furthermore, suboptimal design execution of a particular implementation (e.g., implementation technique or tools) may also negate or diminish the disclosed benefits. Even assuming a skilled implementation, the realization of benefits may still depend on other factors, such as the environmental conditions in which the implementation is deployed. For example, the inputs supplied to a particular implementation may prevent one or more of the problems addressed in this disclosure from occurring on a particular occasion, and as a result, the benefits of the solution may not be realized. Given the existence of possible factors outside the disclosure, it is expressly intended that any potential benefit described herein should not be construed as a limitation of claims that must be satisfied to prove infringement. Rather, the identification of such potential benefits is intended to describe the types of improvements available to designers who benefit from the benefits of this disclosure. The fact that such benefits are described in an acceptable manner (e.g., stating that certain benefits "may occur") is not intended to raise questions about whether such benefits can actually be realized, but rather to acknowledge the technical reality that the realization of such benefits often depends on additional factors.

[0110] Unless otherwise specified, the embodiments are non-limiting. That is, even if only a single embodiment describes a particular feature, the disclosed embodiments are not intended to limit the scope of the claims drafted based on this disclosure. The disclosed embodiments are intended to be illustrative, not limiting, and there is nothing to the contrary in the disclosure. Accordingly, this application is intended to allow claims covering the disclosed embodiments, as well as alternatives, modifications, and equivalents that would be obvious to those skilled in the art who are interested in this disclosure.

[0111] For example, the features in this application may be combined in any appropriate manner. Therefore, new claims may be created for any such combination of features during the proceedings of this application (or an application claiming priority thereto). In particular, with respect to the attached claims, features from dependent claims may be combined with features of other dependent claims, including claims dependent on other independent claims, where appropriate. Similarly, features from each independent claim may be combined where appropriate.

[0112] Accordingly, each of the attached dependent claims may be drafted to depend on a single other claim, but additional dependencies are also contemplated. Any combination of features in the dependent claims that are consistent with the present disclosure may be contemplated and claimed in this application or another application. In short, the combinations are not limited to those specifically enumerated in the attached claims.

[0113] Where appropriate, a claim drafted in one format or legal type (e.g., apparatus) may also be intended to support a corresponding claim in another format or legal type (e.g., method).

[0114] As this disclosure is a legal document, various terms and phrases may be subject to administrative and judicial interpretation. The following paragraphs, and the definitions provided throughout the disclosure, should be used in determining how to interpret the claims drafted based on this disclosure.

[0115] A reference to the singular form of an item (i.e., a noun or noun phrase preceded by "a," "an," or "the") is intended to mean "one or more" unless the context clearly indicates otherwise. Therefore, a reference to "an item" in a claim does not, without context, exclude additional instances of that item. "Multiple" items refer to a set of two or more items.

[0116] The term "may" is used here in a permissive sense (i.e., it has the possibility, it can) and not in a compulsory sense (i.e., it must).

[0117] The terms "to include" and "to have" and their forms are open-ended and mean "to include, but not to be limited by."

[0118] In this disclosure, where the term "or" is used in relation to a list of choices, it will generally be understood to be used in an inclusive sense unless the context provides otherwise. Thus, the statement "x or y" is equivalent to "x or y or both," and therefore covers 1) x but not y, 2) y but not x, and 3) both x and y. On the other hand, phrases such as "either x or y, but not both" clarify that "or" is used in an exclusive sense.

[0119] The phrases "w, x, y, or z, or any combination thereof" or "at least one of w, x, y, and z" are intended to cover all possibilities, ranging from a single element to the total number of elements in the set. For example, given the set [w, x, y, z], these phrases cover any single element in the set (e.g., w but not x, y, or z), any two elements (e.g., w and x but not y or z), any three elements (e.g., w, x, and y but not z), and all four elements. Thus, the phrase "at least one of w, x, y, and z" refers to at least one element in the set [w, x, y, z], thereby covering all possible combinations in this list of elements. This phrase should not be interpreted as requiring the existence of at least one instance of w, at least one instance of x, at least one instance of y, and at least one instance of z.

[0120] In this disclosure, various “labels” may precede nouns or noun phrases. Unless the context provides otherwise, different labels used for features (e.g., “First Circuit,” “Second Circuit,” “Specific Circuit,” “Given Circuit,” etc.) refer to different instances of the feature. In addition, when applied to features, the labels “First,” “Second,” and “Third” do not imply any type of order (e.g., spatial, temporal, logical, etc.) unless otherwise specified.

[0121] The phrase "based on" is used to describe one or more factors that influence a decision. This term does not preclude the possibility that additional factors may influence the decision; that is, the decision may be based on the specified factors alone, or on the specified factors and other unspecified factors. Consider the phrase "decide A based on B." This phrase specifies that B is a factor used to determine A or that influences the decision of A. This phrase does not preclude the possibility that the decision of A may also be based on several other factors, such as C. This phrase is also intended to cover embodiments in which A is determined based solely on B. When used herein, the phrase "based on" is synonymous with the phrase "based at least in part."

[0122] The phrases “in response to” and “respond to” describe one or more factors that trigger an effect. This phrase does not preclude the possibility that additional factors may influence or otherwise trigger an effect, either together with or independently of the specified factors. That is, the effect may simply be in response to those factors, or it may be in response to the specified factors as well as other unspecified factors. Consider the phrase “perform A in response to B.” This phrase specifies that B is a factor that triggers the performance of A, or a factor that triggers a particular outcome of A. This phrase does not preclude the possibility that performing A may also be in response to other factors such as C. Nor does this phrase preclude the possibility that performing A may be performed jointly in response to B and C. This phrase is also intended to cover embodiments in which A is performed simply in response to B. As used herein, the phrase “in response to” is synonymous with the phrase “in response to at least partially.” Similarly, the phrase “in response to” is synonymous with the phrase “in response to at least partially.”

[0123] In this disclosure, different entities (which may be referred to in various ways, such as “units,” “circuits,” or other components) are described or claimed as “configured” to perform one or more tasks or operations. This formulation—[the entity is configured] to perform one or more tasks—is used here to refer to a structure (i.e., something physical). More specifically, this formulation is used to indicate that the structure is arranged to perform one or more tasks during operation. A structure can be said to be “configured” to perform some task even when it is not currently operating. Thus, an entity described or explained as “configured” to perform some task refers to something physical, such as a device, a circuit, a system having a processor unit and memory that stores executable program instructions to implement the task. This phrase is not used here to refer to intangible things.

[0124] In some cases, various units / circuits / components may be described herein as performing a set of tasks or operations. These entities are understood to be “configured” to perform these tasks / operations, even if not specifically stated otherwise.

[0125] The term "configured to" is not intended to mean "configurable to." For example, an unprogrammed FPGA cannot be considered "configured to" perform a particular function. However, this unprogrammed FPGA can be "configurable to" perform that function. After proper programming, an FPGA can be said to be "configured to" perform a particular function.

[0126] For the purposes of a U.S. patent application based on this disclosure, claiming that a structure is “configured to perform one or more tasks” does not expressly intend to invoke 35 U.S.C §112(f) with respect to that claim element. If an applicant wishes to invoke §112(f) during the proceedings of a U.S. patent application based on this disclosure, the applicant shall describe the claim element using the structure “means for performing a function.”

Claims

1. In the computing system, A step of storing a skip list containing multiple key / value records, wherein the multiple key / value records include one or more pointers to other key / value records among the multiple key / value records; A step of scanning the skip list for positions associated with a specific key, wherein the scanning step is: Steps include identifying a range of the skip list, including the position associated with the particular key, using the key prefix of the particular key, wherein the range is a range that includes key / value records in the skip list having the same key prefix; The steps include: starting a scan for the position within the identified range; The steps of scanning key / value records forward within the range along a pointer in a skip list tower of key / value records until a key / value record is identified that contains a pointer to another key / value record located after the position associated with the particular key in key order, The steps include: traversing the skip list from the identified key / value record to identify the position associated with the particular key; The scanning step includes, A non-temporary computer-readable medium that stores program instructions capable of implementing actions including those described above.

2. The aforementioned scan, A step of scanning key / value records forward within the range along the highest available pointer in the skip list tower of the key / value records, A computer-readable medium according to claim 1, including the following:

3. The aforementioned operation is, In response to identifying the aforementioned location, the step of inserting a key / value record into the skip list at the location associated with the specific key, The computer-readable medium according to claim 1, further comprising:

4. The aforementioned operation is, The step of determining the set of pointers to include in the inserted key / value record by sequentially scanning in reverse order from the identified position and identifying a key / value record that includes a pointer that points to a key / value record located after the identified position in key order, The computer-readable medium according to claim 3, further comprising:

5. The aforementioned operation is, Before sequentially scanning in the reverse direction, the steps include determining the number of pointers in the set, The steps include determining, based on the number determined, whether to traverse the skip list from the sentinel node or sequentially scan in reverse from the identified location, The computer-readable medium according to claim 4, further comprising:

6. The aforementioned operation is, The step of maintaining an index that associates a key prefix with key / value records within each part that has the same key prefix, Furthermore, the use of the aforementioned key prefix means This includes accessing the aforementioned index and identifying the initial key / value record within the range to initiate the scan. The computer-readable medium according to claim 1.

7. The aforementioned index is a hash table, Accessing the aforementioned index means The steps include: applying a hash function to the key prefix to identify a hash bucket containing a pointer to the initial key / value record within the range; The steps include: traversing the pointer to the initial key / value record within the range; A computer-readable medium according to claim 6, including the following:

8. The aforementioned scanning step is, A step of selecting a first key prefix and a second key prefix of the specified key, and determining whether there is a relevant range of the skip list for the first key prefix and the second key prefix, wherein the second key prefix is ​​longer than the first key prefix. In response to determining that no range associated with the second key prefix exists in the skip list, the steps include selecting a range associated with the second key prefix for use in the scan for the position, A computer-readable medium according to claim 1, including the following:

9. The aforementioned operation is, The process further includes the step of determining the length of the key prefix to be used by accessing a set of anchor plans, the anchor plans specifying the key prefix length to be used based on the particular key portion. The computer-readable medium according to claim 1.

10. The skip list maintains the order of keys in key / value records stored in a buffer data structure that stores data for active database transactions. The computer-readable medium according to claim 1.

11. A computing system stores a skip list containing multiple key / value records, wherein each of the multiple key / value records includes one or more pointers to other key / value records among the multiple key / value records. The computing system scans the skip list for positions associated with a particular key, and the scanning step is: Steps include identifying a range in the skip list, including the position associated with the particular key, using the key prefix of the particular key, wherein the range is a range that includes key / value records in the skip list having keys of the same key prefix; The steps include: starting a scan for the location within the identified range; The computing system performs the steps of scanning key / value records forward within the range along a pointer in a skip list tower of key / value records until it identifies a key / value record containing a pointer to another key / value record located after the position associated with the particular key in key order, The steps include: traversing the skip list from the identified key / value record to identify the position associated with the particular key; The scanning step includes, Methods that include...

12. In response to identifying the location, insert a key / value record into the skip list at the location associated with the specific key, The method according to claim 11, further comprising:

13. A step of determining a set of pointers to be included in the key / value record to be inserted, by sequentially scanning in reverse order from the identified position and identifying a key / value record that includes a pointer that points to a key / value record located after the identified position in key order. The method according to claim 12, further comprising:

14. The step of determining the number of pointers in the set before sequentially scanning in the reverse direction, The steps include determining, based on the number determined, whether to traverse the skip list from the sentinel node or sequentially scan in reverse from the identified location, The method according to claim 13, further comprising:

15. A step of maintaining an index that associates a key prefix with key / value records in each portion having the same key prefix, Furthermore, the use of the aforementioned key prefix means This includes accessing the aforementioned index and identifying the initial key / value record within the range to initiate the scan. The method according to claim 11.

16. The index is a hash table, Accessing the aforementioned index means The steps include: applying a hash function to the key prefix to identify a hash bucket containing a pointer to the initial key / value record within the range; The steps include: traversing the pointer to the initial key / value record within the range; The method according to claim 15, including the method described in claim 15.

17. The scanning step is: A step of selecting a first key prefix and a second key prefix of the specified key, and determining whether there is a relevant range of the skip list for the first key prefix and the second key prefix, wherein the second key prefix is ​​longer than the first key prefix. In response to determining that no range associated with the second key prefix exists in the skip list, the steps include selecting a range associated with the second key prefix for use in the scan for the position, The method according to claim 11, including the method described in claim 11.

18. The further step of determining the length of the key prefix to be used by accessing a set of anchor plans, the anchor plans specifying the key prefix length to be used based on the particular key portion. The method according to claim 11.

19. The skip list maintains the order of keys in key / value records stored in a buffer data structure that stores data for database transactions. The method according to claim 11.

20. A computing system, One or more processors, In the computing system, A step of storing a skip list containing multiple key / value records, wherein the multiple key / value records include one or more pointers to other key / value records among the multiple key / value records; A step of scanning the skip list for positions associated with a specific key, wherein the scanning step is: Steps include identifying a range of the skip list, including the position associated with the particular key, using the key prefix of the particular key, wherein the range is a range that includes key / value records in the skip list having the same key prefix; The steps include: starting a scan for the position within the identified range; The steps of scanning key / value records forward within the range along a pointer in a skip list tower of key / value records until a key / value record is identified that contains a pointer to another key / value record located after the position associated with the particular key in key order, The steps include: traversing the skip list from the identified key / value record to identify the position associated with the particular key; The scanning step includes, A memory containing program instructions that can implement operations including, A computing system equipped with [the following features].

Images