key space reference

Abstract

Techniques related to tracking record writes across a key space spanning a set of database nodes are disclosed. A first database node of a database system can receive a request to perform a database transaction that includes writing a particular record for a key included by a key space. The first database node can access a key space reference directory that stores a plurality of indications of when database nodes of the database system have made writes to the key space. In response to determining that a second database node has written a record for the key space within a particular time period, the first database node can send a request to the second database node for information indicating whether the second database node has written a record for the key. Based on a response received from the second database node, the first database node can determine whether to write the particular record.

Description

background Technical Field

[0002] The publicly available text generally pertains to database systems, and more specifically, to tracking record writes across a set of database nodes targeting a key space.

[0003] Related technical descriptions

[0004] Modern database systems typically implement management systems that enable users to store collections of information in an organized manner, allowing for efficient access and manipulation of those collections. In some cases, these management systems maintain a Log Structure Merge Tree (LSM tree) with multiple levels, each storing information as key-value pairs in database records. An LSM tree typically includes two high-level components: a memory cache and persistent storage. During operation, the database system receives transaction requests to process transactions, which involves writing database records to persistent storage. The database system first writes the database records to the memory cache and then flushes them to persistent storage. Attached Figure Description

[0005] Figure 1 This is a block diagram illustrating exemplary elements of a system according to some implementations, which is capable of using keyspace references to track record writes across a set of database nodes' keyspaces.

[0006] Figure 2A This is a block diagram illustrating exemplary elements of key space licensing according to some implementation schemes.

[0007] Figure 2B This is a block diagram illustrating exemplary elements of key space references according to some implementation schemes.

[0008] Figure 3 This is a block diagram illustrating exemplary elements related to a database node that uses keyspace references to process active database transactions, according to some implementation schemes.

[0009] Figure 4 This is a block diagram illustrating exemplary elements related to updating a database node referenced in the key space after a database transaction has been committed, according to some implementation schemes.

[0010] Figure 5 and Figure 6 This is a flowchart illustrating an exemplary method related to handling database transactions using keyspace references according to some implementation schemes.

[0011] Figure 7 This is a block diagram illustrating the components of a multi-tenant system according to some implementation schemes.

[0012] Figure 8This is a block diagram illustrating the elements of a computer system for implementing the various systems described in the public text, according to some implementation schemes. Detailed Implementation

[0013] As mentioned above, modern database systems can maintain an LSM tree consisting of database records with key-value pairs. In many cases, a database system comprises a single active database node responsible for writing database records to the permanent storage component of the LSM tree. However, in some cases, a database system comprises multiple database nodes that write database records to the LSM tree. To prevent multiple database nodes from writing database records to the same database key at approximately the same time, key permissions can be provided to the database nodes. These permission identifiers specify which database nodes are allowed to write to the corresponding database record's key.

[0014] However, in various situations, it may be desirable to redeliver key permissions from a previous owner database node to a new owner database node. For example, an administrator might want to perform a rolling upgrade of a database node to a new software version. To prevent delays in database transactions associated with the keys and key permissions provided to that database node, it might be desirable to redeliver those key permissions to another database node so that those database transactions can be executed while the previous database node is offline for the upgrade. However, in many cases, at the time of redelivery, the previous database node still has ongoing database transactions associated with the key permissions being redelivered. Restarting those ongoing database transactions is not advisable because some of these transactions may take a long time to execute (e.g., an hour). The published text specifically addresses the technical problem of how to handle ongoing transactions when redelivering key permissions to a new owner database node.

[0015] More specifically, the public text describes various techniques for enabling ongoing transactions in a key space to be committed or rolled back on the previous owner database node, while allowing new transactions in the same key space to begin on the new owner database node. However, since transactions can run on two or more separate database nodes that commit database records for the same key space, a mechanism is needed to ensure that their writes do not lead to data corruption within the database. Therefore, the public text further addresses the technical problem of ensuring consistency within a database system when multiple transactions are executed on multiple database nodes for the same key space.

[0016] In the various implementations described below, keyspace references are used to track where database record writes for a specific keyspace are about to occur and where database record writes for that keyspace have previously occurred. A first keyspace reference can be created when write permission for a keyspace is initially granted to the first database node of the database system. This reference indicates that database record writes for that keyspace will only occur on the first database node. Consequently, the database nodes of the database system direct write requests to the first database node based on the first keyspace reference, which can be included in a reference directory that stores keyspace references for various keyspaces of the database system.

[0017] At some point during the operation of the database system, a specific keyspace may be re-provided from a first database node to a second database node. In some implementations, when this keyspace is provided to the second database node, a second keyspace reference is created with an "open" state, indicating that database record writes for that keyspace will occur on the second database node. The first keyspace reference associated with the first database node can be set to a "closed" state, indicating that database record writes for that keyspace will no longer occur on the first database node. As a result, the database nodes of the database system will direct write requests to the second database node instead of the first database node. Therefore, database record writes for that keyspace may continue to occur even if the first database node is offline. When new database record writes are directed to the second database node, the first database node may still have active transactions for records in the keyspace that have been written but not committed. As a result, a situation may arise where the second database node writes and commits records for the same keyspace before records on the first database node, leading to inconsistent databases because it appears that the later record was written before the earlier record, even if the earlier record was written before the later record.

[0018] To avoid this situation, in various implementations, when a second database node wishes to write a database record for a key in a specific keyspace, it checks the keyspace references in the reference directory. Based on the keyspace references, the second database node can determine whether its write to the keyspace might potentially conflict with a write to another database node within a certain timeframe. For example, the second database node might determine that there are ongoing database transactions at the first database node that have already written database records for the keyspace. In various implementations, if the second database node determines that at least one database record will be written to the keyspace, it sends a request to each database node that may have already written the database record for the key for which the second database node intends to write the database record. For example, the second database node might send a request to the first database node to determine whether a database record for writing to the keyspace is associated with the key. If no record is written for that key, the second database node will write the record it wants for the key. If a record exists for the key written by another database node, the second database node can terminate its transaction or ensure that its record is committed after the record on the other database node. Because it can see what other database nodes are writing to a database record for a given key, a database node can prevent itself from writing and committing database records to the same key in a way that could corrupt the database.

[0019] In some cases, the keyspace may move around several database nodes. As a result, there may be several "read" keyspace references (identifying the location where a database record write previously occurred in the keyspace) and one "write" keyspace reference (identifying the location where a database record write to the keyspace is to occur). Therefore, when a database node wants to read or write a database record for a key, it may have to check multiple keyspace references corresponding to that key and their associated database nodes.

[0020] Public text techniques may be superior to existing methods because they provide mechanisms that allow keyspace redistribution across database nodes while ensuring consistency in the database system by preventing those nodes from committing records in incorrect chronological order. In particular, these techniques allow keyspace redistribution without restarting ongoing transactions. By avoiding restarting ongoing transactions, database resources are not wasted, and time is saved, as database transactions can take a long time to execute. Furthermore, the cost of performing updates on database nodes is lower because the cost of restarting an ongoing transaction as part of a keyspace transfer is not incurred. Therefore, the overall operation of the database system is improved. Exemplary applications of public text techniques will now be discussed, starting from the reference... Figure 1 start.

[0021] Turn now Figure 1A block diagram of system 100 is shown. System 100 includes a set of components that can be implemented by hardware or a combination of hardware and software routines. In the illustrated embodiment, system 100 includes a database 110 (with an LSM file 115), database nodes 120A and 120B, and a directory manager node 140. As further shown, directory manager node 140 includes a storage directory 145 with keyspace permissions 124 and keyspace references 126, one or more of which are stored in database node 120. Also as shown, database node 120 includes a corresponding memory cache 130 that stores database records 132 for key 134. In some embodiments, system 100 is implemented differently than shown. For example, there may be no directory manager node 140; instead, storage directory 145 may be stored in a storage area shared by database nodes 120, such that database nodes 120 maintain keyspace permissions 124 and key references 126. Furthermore, although the techniques for open text have been discussed regarding LSM trees, these techniques can be applied to other types of database implementations where multiple nodes are writing and committing records to the database.

[0022] In various implementations, system 100 implements a platform service (e.g., a customer relationship management (CRM) platform service) that allows users of the service to develop, run, and manage applications. System 100 may be a multi-tenant system providing various functionalities to multiple users / tenants hosted by a multi-tenant system. Therefore, system 100 can execute software routines from various different users (e.g., system 100's providers and tenants), and provide code, web pages, and other data to users, databases, and other entities associated with system 100. As illustrated, for example, system 100 includes database nodes 120 that can store, manipulate, and retrieve data from LSM files 115 of database 110 on behalf of users of system 100.

[0023] In various implementations, database 110 is a collection of information organized in a manner that allows access, storage, and manipulation of information. Therefore, database 110 may include supporting software that allows database nodes 120 to perform operations (e.g., access, storage, etc.) on the information stored in database 110. In some implementations, database 110 is implemented by one or more storage devices connected together on a network (e.g., a storage attached network (SAN)) and configured to redundantly store information to prevent data loss. Since the storage devices can persistently store data, database 110 can be used as a persistent storage device. In various implementations, database records 132 written to LSM file 115 by one database node 120 can be accessed by other database nodes 120. LSM file 115 may be stored as part of a log structure merge tree (LSM tree) implemented at database 110.

[0024] In various implementations, the LSM tree is a data structure that stores LSM files 115 in an organized manner using a level-based scheme. The LSM tree may include two high-level components: a memory component implemented at memory cache 130 and an on-disk component implemented at database 110. In some implementations, memory cache 130 is considered to be separate from the LSM tree. Database nodes 120 may initially write database records 132 to their memory cache 130. As cache 130 becomes full and / or at a specific point in time, database nodes 120 may flush their database records 132 to database 110. As part of flushing those database records 132, in various implementations, database nodes 120 write the database records 132 to a new set of LSM files 115 at database 110.

[0025] In various implementations, LSM file 115 is a set of database records 132. Database record 132 may be a key-value pair including data and a corresponding database key 134 that can be used to look up the database record. For example, database record 132 may correspond to a data row in a database table, wherein database record 132 specifies the value of one or more attributes associated with the database table. In various implementations, file 115 is associated with one or more database key ranges defined by the keys 134 of the database records 132 included in the LSM file 115. Consider one embodiment in which file 115 stores three database records 132 associated with keys 134 “XYA”, “XYW”, and “XYZ”, respectively. These three keys 134 span the database key range XYA→XYZ, therefore, LSM file 115 is associated with this database key range.

[0026] In various implementations, database node 120 is hardware, software, or a combination thereof capable of providing database services, such as data storage, data retrieval, and / or data manipulation. These database services may be provided to other components within system 100 or to components outside system 100. For example, database node 120A may receive a request from an application server to execute database transaction 122. In various implementations, database transaction 122 is a logical unit of work (e.g., a specified set of database operations) to be performed relative to database 110. For example, processing database transaction 122 may include executing an SQLSELECT command to select one or more rows from one or more database tables. The content of the rows may be specified in database record 132, so database node 120 may return one or more database records 132 corresponding to the selected one or more table rows. In various cases, executing database transaction 122 may include database node 120 writing one or more database records 132 to an LSM tree. In various implementations, database node 120 first writes those database records 132 to its memory cache 130 and then flushes them to database 110.

[0027] In various implementations, memory cache 130 is a buffer that stores data in the memory (e.g., random access memory) of database node 120. HBase™ Memstore is one embodiment of memory cache 130. As described above, database node 120 can initially write database records 132 to its memory cache 130. In some cases, the latest / latest version of rows in a database table can be found in the database records 132 stored in memory cache 130. However, in some implementations, the database records 132 written to the memory cache 130 of database node 120 are not visible to other database nodes 120. That is, other database nodes 120 do not know what information is stored in the memory cache 130 of database node 120 without being asked. To prevent database record conflicts, since one database node 120 may not know about database records 132 written by another database node 120, in various implementations, database node 120 is equipped with keyspace permissions 124, which control which database records 132 a given database node 120 can write. Therefore, key space permission 124 can prevent two or more database nodes 120 from writing database record 132 to the same database key 134 within a specific time interval, thereby preventing database node 120 from flushing conflicting database record 132 to database 110.

[0028] In various implementations, keyspace permission 124 is information identifying a keyspace and its corresponding owner. For example, as shown, keyspace permission 124 is provided to database node 120B, thus allowing database node 120B to write its corresponding key 134 to its memory cache 130, which falls within the keyspace associated with keyspace permission 124, as database record 132. In various implementations, keyspace permission 124 is provided to at most one database node 120 at any given time. Therefore, although keyspace permission 124 is provided to database node 120B, database node 120A is not allowed to write its corresponding key 134 to the keyspace associated with keyspace permission 124, as database record 132. To allow writing to database record 132 for a specific key 134, in various implementations, database node 120 may issue a permission request specifying key 134 to directory manager node 140. In some cases, the permission request may specify multiple keys 134 (keyspaces).

[0029] In various implementations, directory manager node 140 facilitates the management and distribution of keyspace licenses 124 and keyspace references 126 among database nodes 120. As part of facilitating the management and distribution of keyspace licenses 124, in various implementations, directory manager node 140 updates and distributes keyspace licenses 124 in response to a request received from database node 120. For example, directory manager node 140 may receive a request from database node 120B that allows writing to record 132 of keyspace “XY”. In response, directory manager node 140 may determine whether keyspace licenses 124 for that keyspace have already been provided to database node 120. If not, directory manager node 140 may update keyspace licenses 124 to provide the keyspace to database node 120B, and then may notify all database nodes 120 (including database node 120B) to provide the keyspace. If a license for the requested key space has already been granted, in various implementations, directory manager node 140 identifies the owning database node 120 and sends a request to that database node 120 to relinquish the requested key space. Database node 120 can send a response indicating that the key space has been relinquished, and then directory manager node 140 can update key space license 124 to provide the key space to database node 120B, and then notify all database nodes 120 to re-provide the key space. In various implementations, when the key space is provided to database node 120, a key space reference 126 is created.

[0030] In various implementations, key space reference 126 includes information identifying a time period during which a specified database node 120 writes a database record 132 with a key 134 falling within a specified key space. For example, key space reference 126 could instruct database node 120A to write a database record 132 belonging to key space “XY” during a time period defined from a first transaction commit number (XCN) to a later second XCN. In various implementations, a key space reference 126 for a key space could identify a database node 120 currently permitted to write to database records 132 belonging to that key space. The key space reference 126 identifying a database node 120 permitted to write to a key space is referred to herein as a “write” key space reference 126. This contrasts with a “read” key space reference 126, which identifies a database node 120 that previously wrote to a key space but is no longer permitted to write to that key space (unless the associated key space permission 124 is re-granted to the database node 120). Therefore, in various implementations, the directory manager node 140 can store multiple key space references 126 for the same key space, one of which is a write key space reference 126, and the others are read key space references 126. When the key space is re-provided to another database node 120, the current write key space reference 126 can be converted to a read key space reference 126, and the newly created key space reference 126 can become a write key space reference 126.

[0031] In various implementations, when database node 120 wishes to write database record 132 for a certain key 134, database node 120 sends a directory request 142 to directory manager node 140 for one or more keyspace references 126. In various cases, directory manager node 140 may return write keyspace references 126 as part of a directory response 144 to requesting database node 120. Requesting database node 120 may then send a record write request to database node 120, which is identified by write keyspace references 126 as allowing writes to the keyspace containing the appropriate key 134. Consider an embodiment where database node 120A wishes to write database record 132 for key 134 "X", but a keyspace permission 124 containing key 134 "X" has already been provided to database node 120B. To write database record 132, database node 120A can obtain a write keyspace reference 126 from storage directory 145, which indicates that database node 120B is allowed to write database record 132 to the keyspace containing key 134 "X". As a result, database node 120A can send a record write request to database node 120B to write the specific database record 132.

[0032] In various implementations, when database node 120 wants to write database record 132 to its own memory cache 130 for key 134, database node 120 can send a directory request 142 to directory manager node 140 for one or more keyspace references 126. Directory manager node 140 can then return one or more read keyspace references 126 as part of a directory response 144 to the requesting database node 120. In various implementations, the database node 120 queries which read keyspace references 126 to determine if another database node 120 has written to the associated keyspace within a specific time period, which could potentially lead to commit conflicts between these database nodes 120. For example, database node 120B might want to write database record 132B with key 134B. Database node 120B can determine from a set of read keyspace references 126 that database node 120A has written record 132 to a keyspace including key 134B, but has not yet committed record 132. As shown in the figure, database node 120A has written database record 132A with key 134A, which belongs to the key space of this embodiment.

[0033] In various implementations, in response to determining that another database node 120 has written to the key space within a specific time period, database node 120 sends a record request 136 to that database node 120 to determine whether database node 120 has specifically written a record 132 for a specific key 134. Continuing with the previous embodiments, database node 120B can send a record request 136 to database node 120A to determine whether a database record 132 has been written for key 134B. Database node 120A can send a record response 138 indicating whether a database record 132 has been written for key 134B. If a database record 132 has been written for key 134B, database node 120B can abort the database transaction 122 associated with its record write, or can delay the record write until database node 120A commits another database record 132. If no database record 132 has been written for key 134B, database node 120B can write and commit the database record 132 for key 134B.

[0034] By maintaining the keyspace permissions 124 and keyspace references 126 of system 100, database node 120 can determine where database records 132 will be written and where previously written but committed database records 132 can be found. Thus, when a user redistributes one or more keyspaces to other database nodes 120 as part of updating a specific database node 120, all database nodes 120 of system 100 can determine, based on keyspace permissions 124 and keyspace references 126, which database nodes 120 are allowed to write database records 132 to the redistributed keyspaces. As a result, when updating a specific database node 120, writes to database records in those keyspaces can continue to occur. Furthermore, ongoing database transactions 122 on the specific database node 120 being updated can be committed without causing conflicts because the other database nodes 120 that provided those keyspaces can understand what database record writes have occurred at the specific database node 120. Therefore, those database nodes 120 can prevent themselves from writing and committing database records 132 that would conflict with those written at the specific database node 120.

[0035] Turn now Figure 2A A block diagram illustrating exemplary elements of keyspace license 124 is shown. In the illustrated embodiment, keyspace license 124 designates keyspace 210 and node indicator 220. In some embodiments, system 100 does not include keyspace license 124, but uses keyspace reference 126 to fulfill their roles. In some embodiments, keyspace license 124 is implemented differently than shown. For example, keyspace license 124 may specify an identifier that distinguishes it from other keyspace licenses 124 and allow keyspace license 124 to be looked up in storage directory 145.

[0036] In various implementations, key space 210 corresponds to, for example, Figure 1The key space 210 is a series of keys 134 defined by a minimum key 134 and a maximum key 134. For example, key space 210 may correspond to a range of keys 134 from “AAAAA” to “EEEEE”. In some embodiments, key space 210 corresponds to multiple key ranges (e.g., from “AAAAA” to “BBBBB” and from “CCCCC” to “EEEEE”). In some cases, the key range of key space 210 may be specified by a single key prefix instead of a minimum key 134 and a maximum key 134. For example, key space 210 may specify “XY”, containing all keys 134 with the prefix “XY”. In some embodiments, for a given key space 210, there exists a single key space permission 124, thereby allowing at most one database node 120 to write database records 132 to that key space 210. Therefore, when not all database nodes 120 wish to write database record 132 to a specific key space 210, not all database nodes 120 can request ownership of the key space 210, or issue a request to all database nodes 120 to write database record 132. If ownership of the key space 210 is to be transferred, in various implementations, the node indication 220 of the corresponding key space license 124 is updated to reflect the new ownership of all database nodes 120.

[0037] In various implementations, node indicator 220 indicates the database node 120 associated with keyspace license 124. In some implementations, database node 120 is assigned a log window that defines a list of log files to which database node 120 is allowed to write log information. Node indicator 220 may specify an identifier for the log window, thereby associating it with database node 120 via the log window. Node indicator 220 may be updated in response to the occurrence of various events. For example, directory manager node 140 may update the node indicator 220 for a specific keyspace license 124 after receiving a request from database node 120 for ownership of keyspace 210 corresponding to a specific keyspace license 124. As another example, ownership of keyspace 210 may be transferred from database node 120 that is receiving software updates to its database applications, thus node indicator 220 may be updated to remove ownership of keyspace 210 from database node 120.

[0038] In various embodiments, key space license 124 can be divided into multiple key space licenses 124. For example, a key space license 124 specifying key space 210 "XY" can be divided into two key space licenses 124: one specifying key space 210 "XYA-XYM", and the other specifying key space 210 "XYN-XYZ". In various embodiments, multiple key space licenses 124 can be combined into a single key space license 124. For example, two key space licenses 124 from the previous embodiment can be combined into a single key space license 124 specifying key space 210 "XY".

[0039] Turn now Figure 2B The diagram illustrates a block diagram of exemplary elements of key space reference 126. In the illustrated embodiment, key space reference 126 specifies key space 210, node indication 220, epoch range 230, and state 240. In some embodiments, key space reference 126 is implemented differently than shown; for example, key space reference 126 may not specify state 240.

[0040] As described above, a key space reference 126 can be created when a write permission to key space 210 is granted to database node 120. In various implementations, when key space reference 126 is created, the key space 210 of that key space reference 126 is set to identify the granted key space, and the node indicator 220 is set to identify the associated database node 120. Various information included in key space reference 126 can be added to and updated over time, including after a write permission to the identified key space has been granted to another database node 120. For example, the epoch range 230 of key space reference 126 (described below) may not specify a full range (e.g., an upper limit) until all active database transactions 122 that write to key space 210 have been committed.

[0041] In various implementations, epoch range 230 identifies the time period during which database record 132 is committed to the corresponding keyspace 210. When database transaction 122 is committed, the database record 132 written for that transaction can be marked with a transaction commit number (XCN). Database records 132 committed earlier in time can be marked with an XCN whose value is less than the XCN of database records 132 committed later. In various implementations, committed database records 132 are retained in the memory cache 130 of database node 120 until these records are flushed to database 110 in response to a triggering event (e.g., the memory cache 130 stores a threshold amount of data). When database node 120 flushes its memory cache 130, one or more database records 132 can be flushed to a specific XCN (referred to as "refresh XCN"). In various implementations, epoch range 230 defines the time period by specifying a minimum XCN and a maximum XCN. The minimum XCN can identify the most recent refresh XCN when keyspace reference 126 was created. For example, database node 120B can refresh all database records 132 with XCN less than 600. If a key space reference 126 associated with database node 120B is subsequently created, the lower bound of the epoch range 230 of that key space reference 126 can be set to 600.

[0042] The maximum XCN can identify the XCN associated with the last database transaction 122 written to key space 210 before being granted to another database node 120. That is, when database node 120 owns a specific key space 210, multiple database transactions 122 can be executed that write database records 132 to that key space 210. A specific key space 210 may be granted to another database node 120 at some point; however, those database transactions 122 may still be active. In various implementations, these database transactions 122 are allowed to complete and not terminate prematurely. The epoch range 230 of the key space reference 126 corresponding to those database transactions 122 can be updated to specify the XCN of the last committed transaction among those transactions 122 as the maximum XCN of epoch range 230. Since the maximum XCN may not be set until the last commit of those transactions 122, in various implementations, epoch range 230 is initially assigned a null value for the maximum XCN. As a result, when the maximum XCN is set to a null value, the time period indicated by epoch range 230 may have a beginning but no end.

[0043] In various implementations, state 240 identifies a stage in the lifecycle of keyspace reference 126. These states can include "open," "closed," and "inactive." In various implementations, the "open" state indicates that writes to records of the corresponding keyspace 210 are permitted at the database node 120 indicated by the corresponding node indicator 220. When keyspace reference 126 is initially created, its state 240 can be set to "open." In various implementations, the "closed" state indicates that: 1) writes to records of keyspace 210 are not permitted at the corresponding database node 120, and 2) there is still an active database transaction 122 at that database node 120. In various implementations, active database transaction 122 refers to an ongoing database transaction 122 for which database node 120 is writing database record 132 to its memory cache 130. When database record 132 of database transaction 122 has been committed, active database transaction 122 can become committed database transaction 122. When keyspace 210 of keyspace reference 126 has been provided to another database node 120, the state 240 of keyspace reference 126 can be set to "closed". In various embodiments, the "inactive" state indicates that the active database transaction 122 associated with keyspace 210 has been committed on the corresponding database node 120. In various embodiments, keyspace reference 126 can be deleted after the committed database record 132 associated with keyspace reference 126 has been flushed from the memory cache 130 of the corresponding database node 120 to persistent storage (e.g., database 110).

[0044] Turn now Figure 3 This is a block diagram of an embodiment layout relating to a database node 120 that uses keyspace reference 126 to process active database transactions 122. In the illustrated embodiment, database node 120B includes keyspace reference 126, a memory cache 130, and a database application 300. As shown, database application 300 is assigned keyspace license 124 "XY" and is processing an active database transaction 122 with an associated snapshot transaction commit number (snapshot XCN) 310 "445". As further shown, there are three keyspace references 126, each associated with a different database node 120, but corresponding to the same keyspace 210 "XY". In some embodiments, database node 120 is implemented differently than shown. For example, database application 300 may handle multiple active transactions 122 and multiple committed transactions 122.

[0045] In various implementations, database application 300 is a set of executable program instructions for managing database 110, including managing the LSM tree built around database 110. Thus, database application 300 can receive requests to execute database transaction 122, which involves reading and / or writing database records 132 of database 110. For example, database node 120B can receive transaction requests from application nodes to execute a set of SQL statements identified by the application node. Upon receiving a transaction request, database application 300 can initiate an active database transaction 122 based on the received transaction request. In various implementations, active database transaction 122 is associated with snapshot XCN 310. In various implementations, snapshot XCN 310 identifies the latest XCN, whose corresponding database record 132 can be read by active database transaction 122. For example, the active database transaction 122 shown is associated with snapshot XCN 310 "445". As a result, active database transaction 122 can read committed database records 132 assigned with XCNs less than or equal to "445". In some cases, only database records with an XCN less than "445" can be read (132).

[0046] The following discussion will use an embodiment to provide a deeper understanding of the concepts discussed throughout the disclosed text. Consider an embodiment in which database node 120B wishes to write database record 132 with key 134 "XYZ" for the illustrated active database transaction 122. Before writing database record 132, in various embodiments, database node 120B considers key space reference 126 to determine whether another database node 120 has written key space 210 "XY" within a time period including snapshot XCN 310 "445". As disclosed, key space reference 126 can be accessed from directory manager node 140 via directory request 142 and subsequent directory response 144. When active database transaction 122 is initiated, database node 120B may issue directory request 142. In some cases, directory manager node 140 may only provide key space references 126 that have an epoch range 230 containing snapshot XCN 310 of the initiated database transaction 122.

[0047] As shown in the figure, key space reference 126A identifies database node 120B and has an open state 240, indicating that a record write will occur on database node 120B for key space 210 "XY". Therefore, key space reference 126A is considered a "write" key space reference 126, while key space references 126B and 126C are considered "read" key space reference 126 because they identify locations where record writes to key space 210 "XY" were previously permitted. In some cases, database node 120B may have previously written database record 132 for a key 134 "XYZ" that has not yet been committed. Thus, in some implementations, database node 120B initially searches its memory cache 130 for a database record 132 with key 134 "XYZ". If the database record 132 with key 134 "XYZ" is located, database node 120B can write a new database record 132 relative to that located database record 132. If such a database record 132 is not present in the memory cache 130 of database node 120B, then database node 120B may consider other keyspace references 126.

[0048] As further illustrated, keyspace reference 126B identifies database node 120A, has an open epoch range 230 (without a defined upper limit XCN), and is in a closed state 240, indicating that there are still active database transactions 122 that may have written database record 132 for the uncommitted keyspace 210 "XY". Database node 120B can initially determine whether snapshot XCN 310 "445" falls within the epoch range 230 of keyspace reference 126B. Because snapshot XCN 310 "445" falls within the epoch range 230 "390 - NULL", and not all database transactions 122 associated with keyspace 210 "XY" have been committed, there is a possibility that database node 120A has written database record 132 with a key 134 "XYZ" unknown to database node 120B. As a result, in various implementations, database node 120B determines whether database record 132 has already been written to database node 120A for key 134 "XYZ". To do this, database node 120B may send a record request 136 to database node 120A, requesting indication of whether database node 120A has written such database record 132. Database node 120A may return a record response 138. If the record response 138 indicates that database node 120A has written database record 132 for key 134 "XYZ", then database node 120B may terminate the active database transaction 122 (or a sub-transaction portion), or wait until other database records 132 have been committed before writing its database record 132 for key 134 "XYZ". In some implementations, the record response 138 includes the database record 132 written by database node 120. If record response 138 indicates that database node 120A has not written a database record 132 for key 134 "XYZ", then database node 120B may consider other keyspace references 126.

[0049] As shown in the figure, key space reference 126B identifies database node 120C, has a closed epoch range 230, and an inactive state 240, indicating that all active database transactions 122 at database node 120C have been committed. Although those database transactions 122 have been committed, in some cases, the corresponding database record 132 has not yet been flushed to database 110, but is instead held in the memory cache 130 of database node 120C. Therefore, database node 120B can send a record request 136 to database node 120C to indicate whether database node 120C has written database record 132 for key 134 "XYZ". Based on the record response 138 received from database node 120, database node 120B can terminate the active database transaction 122, or wait to write its database record 132 after another database record 132 has been flushed. If record response 138 indicates that database node 120C has not yet written record 132 for key 134 "XYZ", then database node 120C may consider other key space references 126 if there are more key space references 126 associated with key 134 "XYZ" and the epoch range 230 of the key space reference 126 contains snapshot XCN 310 "445".

[0050] Turn now Figure 4 A block diagram of an embodiment layout related to database node 120 is shown, which prompts an update of key space reference 126 in response to the commit of active database transaction 122. In the illustrated embodiment, database node 120A includes key space reference 126, memory cache 130, and database application 300. As shown, database application 300 has committed database transaction 122 with XCN 410 "600".

[0051] Although the following discussion is conducted with reference to database node 120A, it applies to other database nodes 120, such as database node 120B. During operation, database node 120A may obtain keyspace permission 124 to write database records 132 to keyspace 210 "XY". In various cases, while database node 120A holds the keyspace permission 124, it may initiate multiple active database transactions 122 that write database records 132 to keyspace 210 "XY". While database node 120A is processing those database transactions 122, it may receive a request from directory manager node 140 to relinquish part or all of keyspace 210 "XY". For example, database node 120A may be requested to relinquish keyspace 210 "XYZ". In various implementations, database node 120A relinquishes the requested keyspace 210 but allows the commit of active database transactions 122 associated with that keyspace 210. Although at least one active database transaction 122 is associated with the key space 210, database node 120A may not update the corresponding key space reference 126 (e.g., key space reference 126B for key space 210 “XY”) to define the upper limit of the epoch range 230 of the key space reference 126.

[0052] After processing active database transaction 122, database node 120A can commit database transaction 122, resulting in a committed database transaction 122. As part of the commit process, in some implementations, database node 120A marks each database record 132 of database transaction 122 with XCN 410. As shown, for example, committed transaction 122 has XCN 410 "600". Therefore, each record 132 associated with committed transaction 122 may include metadata identifying XCN 410 "600". After processing the last active transaction associated with abandoned keyspace 120, in various implementations, database node 120A updates the epoch range 230 of the corresponding keyspace reference 126B with the XCN 410 of database transaction 122. Consider one embodiment where the illustrated committed database transaction 122 is the last active transaction 122 for keyspace 210 "XY" at database node 120A. Therefore, database node 120A can update the epoch range 230 of key space reference 126B to specify "XCN 390-600" and state 240 as "inactive". Database node 120A can send a reference update request 404 to directory manager node 140 to distribute the new version of key space reference 126B to other database nodes 120 of system 100.

[0053] Turn now Figure 5A flowchart of method 500 is shown. Method 500 is an embodiment of a method executed by a first database node (e.g., database node 120B) of a database system (e.g., system 100) as part of processing a database transaction (e.g., database transaction 122). In some cases, method 500 can be executed by executing program instructions stored on a non-transitory computer-readable medium (e.g., memory 820). In some embodiments, method 500 includes more or fewer steps than shown. For example, method 500 may include the step of the first database node returning a response to the transaction requester.

[0054] Method 500 begins at step 510, where a first database node receives a request to execute a database transaction that includes writing a specific record (e.g., database record 132) to a key (e.g., key 134) included in a key space (e.g., key space 210). Before receiving the request to execute the database transaction, the first database node may issue a permission request to the database system (e.g., to directory manager node 140) to approve the write to the key space. Thus, the first database node may receive permission to write to the key space (e.g., key space permission 124). In some cases, permission to write to the key space may be re-provided to the first database node from a second database node (e.g., database node 120A). In some implementations, after the first database node receives permission to write to the key space, an active transaction on the second database node that includes writing to the key space is allowed to be committed. In some cases, at least one active transaction may have already prompted a write to the key space record before the first database node receives permission.

[0055] In step 520, the first database node accesses a keyspace reference directory (e.g., storage directory 145) that stores multiple indications (e.g., keyspace reference 126) of when database nodes in the database system have made writes to the keyspace. These multiple indications may include a set of indications specific to the keyspace. One of these indications may identify a database node permitted to write records to that keyspace, and two or more of these indications may identify database nodes that read records written to that keyspace. Thus, a specific indication can indicate that all writes to the keyspace will be performed by the first database node when permission is granted to it. The first database node may receive write requests from the second database node to write specific records to the keyspace as part of an active transaction on the second database node. Therefore, the first database node can grant permission to write specific records to the second database node. The second database node can then use that permission to complete its writes.

[0056] In some cases, a specific indication may identify an epoch range of the key space (e.g., epoch range 230) and be associated with a second database node. A first database node may determine that an epoch corresponding to a database transaction falls within the epoch range. This determination may indicate that the second database node may have written records to that key space within a specific time period. In some cases, the epoch range of a specific indication may be modified in response to the commit of the last active transaction linked to the specific indication. Before modifying the specific indication, the epoch range may or may not have an upper limit defined for the epoch range. The specific indication may be deleted after all records written to the key space by the second database node have been stored in the persistent database of the database system (e.g., database 110). In various implementations, the indication is maintained only for uncommitted work or transactions in main memory and has not yet been flushed to persistent storage.

[0057] In step 530, in response to determining that the second database node may write records to the key space within a specific time period, the first database node sends a request (e.g., record request 136) to the second database node for information indicating whether the second database node has already written a record for the key. In step 540, based on the response received from the second database node (e.g., record response 138), the first database node determines whether to write the specific record. In some cases, in response to determining that the second database node has already written a record for the key, the first database node may terminate at least a portion of the database transaction involving writing the specific record. In some cases, in response to determining that the second database node has already written a record for the key, the first database node may wait before writing the specific record until the record written by the second database node has been committed.

[0058] Turn now Figure 6 A flowchart of method 600 is shown. Method 600 is one embodiment of a method executed by a database system (e.g., system 100) as part of processing a database transaction (e.g., database transaction 122). In some cases, method 600 can be executed by executing program instructions stored on a non-transitory computer-readable medium (e.g., memory 820). In some embodiments, method 600 includes more or fewer steps than shown. For example, method 600 may include the step of a first database node of the database system (e.g., database node 120B) returning a response to the transaction requester.

[0059] Method 600 begins at step 610, where the database system maintains a key space reference directory (e.g., storage directory 145) that stores multiple indications (e.g., key space reference 126) related to a key space (e.g., key space 210). In step 620, the database system allocates key space to a first database node. In various cases, a first specific indication among the multiple indications identifies a first time period (e.g., epoch range 230) of the database system where key space was previously allocated and a second database node (e.g., database node 120A), such that the second database node is permitted to write records whose keys fall within the key space at the second database node. The database system may add a second specific indication to the key space reference directory, specifying an open state (e.g., open state 240), indicating that all record writes to the key space will occur at the first database node. The database system may update the first specific indication to specify a closed state (e.g., closed state 240), indicating that no record writes to the key space will occur at the second database node. In some cases, the database system receives an upgrade request to perform a rolling upgrade at the second database node and performs the allocation in response to receiving the upgrade request.

[0060] In step 630, a transaction is executed by the first database node, the transaction involving key write records for the key space. This execution includes, in step 632, the first database node determining, based on a first specific indication, that a first time period overlaps with a second time period associated with the transaction. This execution includes, in step 634, in response to the determination, the first database node sending a request (e.g., record request 136) to the second database node regarding information indicating whether a record has been written for that key by the second database node but not committed. In response to determining that the second database node has not written a record for that key, the first database node may write that specific record.

[0061] Exemplary multi-tenant database system

[0062] Turn now Figure 7 An exemplary multi-tenant database system (MTS) 700 is shown, which can implement various technologies for public text. For example, system 100 may be an MTS 700. Figure 7In this embodiment, MTS 700 includes a database platform 710, an application platform 720, and a network interface 730 connected to a network 740. Also as shown, the database platform 710 includes a data storage 712 and a set of database servers 714A-N that interact with the data storage 712, and the application platform 720 includes a set of application servers 722A-N with corresponding environments 724. In the illustrated embodiment, MTS 700 is connected to various user systems 750A-N via network 740. The disclosed multi-tenant systems are included for illustrative purposes and are not intended to limit the scope of the disclosed text. In other embodiments, the technologies described in the disclosed text are implemented in non-multi-tenant environments, such as client / server environments, cloud computing environments, clustered computers, etc.

[0063] In various implementations, the MTS 700 is a group of computer systems that together provide various services to users (or “tenants”) interacting with the MTS 700. In some implementations, the MTS 700 implements a customer relationship management (CRM) system that provides a mechanism for tenants (e.g., companies, government agencies, etc.) to manage their relationships and interactions with customers and prospects. For example, the MTS 700 enables tenants to store customer contact information (e.g., customer websites, email addresses, phone numbers, and social media data), identify sales opportunities, record service issues, and manage marketing campaigns. Furthermore, the MTS 700 enables tenants to identify how they communicate with customers, what customers have purchased, when customers last purchased items, and how much customers paid. To provide CRM system services and / or other services, as shown in the figure, the MTS 700 includes a database platform 710 and an application platform 720.

[0064] In various embodiments, database platform 710 is a combination of hardware components and software routines that implement database services for storing and managing data (including tenant data) of MTS 700. As shown, database platform 710 includes data storage 712. In various embodiments, data storage 712 includes a set of storage devices (e.g., solid-state drives, hard disk drives, etc.) connected together on a network (e.g., a storage attached network (SAN)) and configured to redundantly store data to prevent data loss. In various embodiments, data storage 712 is used to implement a database (e.g., database 110) that includes a collection of information organized in a manner that allows access, storage, and manipulation of information. Data storage 712 can implement a single database, a distributed database, a collection of distributed databases, a database with redundant online or offline backups, or other redundancies. As part of implementing the database, data storage 712 can store files (e.g., file 115) that include one or more database records with corresponding data payloads (e.g., values ​​of fields in a database table) and metadata (e.g., key values, timestamps, table identifiers of the tables associated with the records, tenant identifiers of the tenants associated with the records, etc.).

[0065] In various implementations, database records may correspond to rows of tables. Tables typically contain one or more data categories, which are logically set as columns or fields in a visual schema. Therefore, each record in a table may contain a data instance for each category defined by its fields. For example, a database may include a table describing customers, with fields for basic contact information such as name, address, phone number, and fax number. Therefore, records in this table may include values ​​for each field in the table (e.g., the name of the name field). Another table might describe purchase orders, including fields for information such as customer, product, sales price, and date. In various implementations, standard entity tables are provided for use by all tenants, such as tables for account, contact, lead, and opportunity data, each containing predefined fields. The MTS 700 may store database records for one or more tenants in the same table; that is, tenants may share a single table. Therefore, in various implementations, database records include a tenant identifier indicating the owner of the database record. As a result, one tenant's data remains secure and separate from other tenants' data, preventing that tenant from accessing another tenant's data unless such data is explicitly shared.

[0066] In some implementations, the data stored in data storage 712 is organized as part of a Log Structure Merge Tree (LSM tree). An LSM tree typically includes two high-level components: a memory cache and persistent storage. In operation, database server 714 may first write database records to the local memory cache and then flush these records to persistent storage (e.g., data storage device 712). As part of flushing database records, database server 714 may write database records to a new file included at the “top level” of the LSM tree. Over time, as database records move down the LSM tree levels, they can be rewritten by database server 714 to new files included at lower levels. In various implementations, as database records age and move down the LSM tree, they move to increasingly slower storage devices (e.g., from solid-state drives to hard disk drives) of data storage 712.

[0067] When database server 714 wants to access a database record for a specific key, it can traverse different levels of the LSM tree to search for files that might contain the database record for that specific key. If database server 714 determines that a file can include the relevant database record, it can retrieve the file from data storage 712 into its own storage. Database server 714 can then examine the retrieved file for the database record with the specific key. In various implementations, database records are immutable once written to data storage 712. Therefore, if database server 714 wants to modify the value of a row in a table (which can be identified from the accessed database record), it writes a new database record to the top level of the LSM tree. Over time, this database record is merged down the LSM tree. Thus, the LSM tree can store various database records for a database key, where older records for that key are located at lower levels of the LSM tree compared to newer records.

[0068] In various implementations, database server 714 is a hardware element, software routine, or a combination thereof capable of providing database services (e.g., data storage, data retrieval, and / or data manipulation). Database server 714 may correspond to database node 120. Such database services can be provided by database server 714 to components within MTS 700 (e.g., application server 722) and components outside MTS 700. For example, database server 714 may receive database transaction requests from application server 722, requesting to write data to or read data from data storage 712. Database transaction requests may specify SQL SELECT commands to select one or more rows from one or more database tables. The content of a row can be defined in a database record, so database server 714 can locate and return one or more database records corresponding to the selected one or more table rows. In various cases, database transaction requests may instruct database server 714 to write one or more database records against an LSM tree, which database server 714 maintains, implemented on database platform 710. In some implementations, database server 714 implements a relational database management system (RDMS) or an object-oriented database management system (OODBMS), which facilitates the storage and retrieval of information for data storage 712. In various cases, database servers 714 can communicate with each other to facilitate transaction processing. For example, database server 714A can communicate with database server 714N to determine whether database server 714N has written a database record to its memory cache for a specific key.

[0069] In various implementations, application platform 720 is a combination of hardware components and software routines that implement and execute CRM software applications, and provide relevant data, code, forms, web pages, and other information to and from user system 750, and store relevant data, objects, web page content, and other tenant information via database platform 710. To facilitate these services, in various implementations, application platform 720 communicates with database platform 710 to store, access, and manipulate data. In some cases, application platform 720 may communicate with database platform 710 via different network connections. For example, one application server 722 may be connected via a local area network, and another application server 722 may be connected via a direct network link. Transmission Control Protocol (TCP / IP) and Internet Protocol (TCP / IP) are exemplary protocols for communication between application platform 770 and database platform 710; however, it will be apparent to those skilled in the art that other transport protocols may be used depending on the network interconnection used.

[0070] In various implementations, application server 722 is a hardware element, software routine, or combination thereof capable of providing services to application platform 720, including processing requests received from tenants of MTS 700. In various implementations, application server 722 can create environment 724 that can be used for various purposes, such as providing developers with the ability to develop, execute, and manage applications (e.g., business logic). Data can be transferred from another environment 724 and / or from database platform 710 to environment 724. In some cases, environment 724 cannot access data from other environments 724 unless such data is explicitly shared. In some implementations, multiple environments 724 may be associated with a single tenant.

[0071] Application platform 720 can provide user system 750 with access to multiple different hosted (standard and / or custom) applications, including CRM applications and / or applications developed by tenants. In various implementations, application platform 720 can manage application creation, application testing, application storage in database objects at data storage 712, application execution in environment 724 (e.g., virtual machines in process space), or any combination thereof. In some implementations, application platform 720 can add and remove application server 722 from the server pool at any time for any reason, even if there is no server affinity between a user and / or organization and a particular application server 722. In some implementations, an interface system (not shown) implementing load balancing functionality (e.g., an F5 Big-IP load balancer) is located between application server 722 and user system 750 and is operable to distribute requests to application server 722. In some implementations, the load balancer uses a least-connections algorithm to route user requests to application server 722. Other embodiments of load balancing algorithms, such as round-robin and observational response time, may also be used. For example, in some implementations, three consecutive requests from the same user may hit three different servers 722, and three requests from different users may hit the same server 722.

[0072] In some implementations, the MTS 700 provides security mechanisms (e.g., encryption) to keep each tenant's data separate unless the data is shared. If more than one server 714 or 722 is used, the servers can be located very close to each other (e.g., a cluster of servers in a single building or campus) or distributed in locations far apart from each other (e.g., one or more servers 714 located in city A and one or more servers 722 located in city B). Therefore, the MTS 700 can include one or more logically and / or physically connected servers that are distributed locally or across one or more geographical locations.

[0073] One or more users (e.g., via user system 750) can interact with MTS 700 via network 740. User system 750 may correspond to, for example, a tenant of MTS 700, a provider of MTS 700 (e.g., an administrator), or a third party. Each user system 750 may be a desktop PC, workstation, laptop, PDA, mobile phone, or any device that supports Wireless Access Protocol (WAP) or any other computing device capable of directly or indirectly engaging with the Internet or other network connections. User system 750 may include dedicated hardware configured to engage with MTS 700 via network 740. User system 750 may execute a graphical user interface (GUI) corresponding to MTS 700, an HTTP client (e.g., a browser, such as Microsoft's Internet Explorer™ browser, Netscape's Navigator™ browser, Opera's browser, or a WAP-enabled browser in the case of cellular phones, PDAs, or other wireless devices), or both, thereby allowing users of user system 750 (e.g., subscribers of a CRM system) to access, process, and view information and pages available to them through MTS 700. Each user system 750 may include one or more user interface devices, such as a keyboard, mouse, touchscreen, pen, etc., for interacting with a graphical user interface (GUI) provided by a browser on a display monitor screen, LCD monitor, etc., in conjunction with pages, forms, and other information provided by the MTS 700 or other systems or servers. As described above, the disclosed embodiments are applicable to the Internet, which refers to a specific global network interconnection. However, it should be understood that other networks can be used instead of the Internet, such as intranets, extranets, virtual private networks (VPNs), non-TCP / IP based networks, any LAN or WAN, etc.

[0074] Because users of User System 750 can have varying capabilities, the capabilities of a specific User System 750 may be determined by one or more permission levels associated with the current user. For example, when a salesperson interacts with MTS 700 using a specific User System 750, that User System 750 may have the capabilities assigned to that salesperson (e.g., user privileges). However, when an administrator interacts with MTS 700 using the same User System 750, that User System 750 may have the capabilities assigned to that administrator (e.g., administrative privileges). In systems with a hierarchical role model, a user at one permission level can access applications, data, and database information accessible to users at lower permission levels, but cannot access certain applications, database information, and data accessible to users at higher permission levels. Therefore, depending on the user's security or permission level, different users may have different capabilities in accessing and modifying application and database information. It is also possible that some data structures managed by MTS 700 are assigned at the tenant level, while other data structures are managed at the user level.

[0075] In some embodiments, the user system 750 and its components can be configured using an application (e.g., a browser) that includes computer code executable on one or more processing elements. Similarly, in some embodiments, the MTS 700 (and additional instances of the MTS, where more than one instance exists) and its components are configured by an operator using an application that includes computer code executable on a processing element. Therefore, the various operations described herein can be performed by executing program instructions stored on a non-transitory computer-readable medium and executed by a processing element. The program instructions can be stored on a non-volatile medium (e.g., a hard disk), or in any other well-known volatile or non-volatile storage medium or device, such as ROM or RAM, or provided on any medium capable of booting the program code, such as an optical disc (CD) medium, a digital versatile disc (DVD) medium, a floppy disk, etc. Furthermore, it is well known that the entire program code or portions thereof can be transferred and downloaded from a software source (e.g., via the Internet) or from another server, or transmitted via any other well-known conventional network connection (e.g., extranet, VPN, LAN, etc.) using any well-known communication medium and protocol (e.g., TCP / IP, HTTP, HTTPS, Ethernet, etc.). It is also understood that the computer code used to implement aspects of the disclosed implementation scheme can be implemented in any programming language capable of executing on a server or server system, such as C, C++, HTML, Java, JavaScript, or any other scripting language, such as VBScript.

[0076] Network 740 can be a LAN (Local Area Network), WAN (Wide Area Network), wireless network, point-to-point network, star network, token ring network, hub network, or any other suitable configuration. The global interconnected network, often referred to as the "Internet" with a capital "I," is an embodiment of TCP / IP (Transmission Control Protocol and Internet Protocol) networking. However, it should be understood that the disclosed implementation can utilize any and many other types of networks.

[0077] User system 750 can communicate with MTS 700 using TCP / IP, and at a higher network level, it can communicate using other common Internet protocols such as HTTP, FTP, AFS, and WAP. For example, in the case of using HTTP, user system 750 may include an HTTP client, commonly referred to as a "browser," for sending and receiving HTTP messages from the HTTP server at MTS 700. This server may be implemented as the sole network interface between MTS 700 and network 740, but other technologies can be used or alternatively. In some implementations, the interface between MTS 700 and network 740 includes load-sharing functionality, such as a round-robin HTTP request distributor, to balance the load and distribute incoming HTTP requests evenly across multiple servers.

[0078] In various implementations, user system 750 communicates with application server 722 to request and update system-level and tenant-level data from MTS 700, which may require one or more queries to data storage 712. In some implementations, MTS 700 automatically generates one or more SQL statements (SQL queries) designed to access the required information. In some cases, user system 750 may generate requests with a specific format corresponding to at least a portion of MTS 700. For example, user system 750 may request the movement of data objects into a specific environment 724 using object notation describing an object-relational mapping (e.g., a JavaScript object notation mapping) of multiple specified objects.

[0079] Exemplary computer system

[0080] Turn now Figure 8 A block diagram of an exemplary computer system 800 is described, which may implement system 100, database 110, database node 120, MTS 700, and / or user system 750. Computer system 800 includes a processor subsystem 880 connected to system memory 820 and I / O interface 840 via interconnect 860 (e.g., system bus). I / O interface 840 is connected to one or more I / O devices 850. Although for convenience, in Figure 8The diagram shows a single computer system 800, but system 800 can also be implemented as two or more computer systems operating together.

[0081] Processor subsystem 880 may include one or more processors or processing units. In various embodiments of computer system 800, multiple instances of processor subsystem 880 may be coupled to interconnect 860. In various embodiments, processor subsystem 880 (or each processor unit within 880) may include cache or other forms of on-board memory.

[0082] System memory 820 can be used to store program instructions executable by processor subsystem 880 to enable system 800 to perform the various operations described herein. System memory 820 can be implemented using different physical storage media, such as hard disk storage, floppy disk storage, removable disk storage, flash memory, random access memory (RAM-SRAM, EDO RAM, SDRAM, DDRSDRAM, RAMBUS RAM, etc.), read-only memory (PROM, EEPROM, etc.), etc. The memory in computer system 800 is not limited to main memory, such as memory 820. Instead, computer system 800 may also include other forms of memory, such as cache memory in processor subsystem 880 and auxiliary memory on I / O devices 850 (e.g., hard disk drives, storage arrays, etc.). In some embodiments, these other forms of memory may also store program instructions executable by processor subsystem 880. In some embodiments, program instructions implementing database application 300 when executed may be included / stored in system memory 820.

[0083] According to various implementations, I / O interface 840 can be any of various types of interfaces configured to connect to and communicate with other devices. In one implementation, I / O interface 840 is a bridge chip (e.g., a southbridge) from a front end to one or more back end buses. I / O interface 840 can be connected to one or more I / O devices 850 via one or more corresponding buses or other interfaces. Embodiments of I / O device 850 include storage devices (hard disk drives, optical drives, removable flash drives, storage arrays, SANs, or their associated controllers), network interface devices (e.g., to a local area network or wide area network), or other devices (e.g., graphics, user interface devices, etc.). In one implementation, computer system 800 is connected to a network via network interface device 850 (e.g., configured to communicate via WiFi, Bluetooth, Ethernet, etc.).

[0084] The implementation of the subject matter of this application includes, but is not limited to, the following embodiments 1 to 20.

[0085] 1. A method comprising:

[0086] The first database node of the database system receives a request to execute a database transaction, which includes writing specific records for the keys included in the key space;

[0087] The first database node accesses the key space reference directory, which stores multiple indications of when the database nodes of the database system have written to the key space;

[0088] In response to determining that the second database node has written records to the key space within a specific time period, the first database node sends a request to the second database node for information indicating whether the second database node has written records to the key.

[0089] Based on the response received from the second database node, the first database node determines whether to write a specific record.

[0090] 2. The method according to Embodiment 1 further includes:

[0091] Before receiving a request to execute a database transaction, the first database node:

[0092] Send a permission request to the database system to obtain approval to write records to the key space; and

[0093] Receive permission to write records to the key space, wherein the permission is transmitted from the second database node to the first database node.

[0094] 3. The method according to embodiment 2, wherein after the first database node receives permission to write records to the key space, an active transaction on the second database node is allowed to be committed, the active transaction including writing records to the key space, and wherein, before the first database node receives permission, at least one active transaction results in a record write to the key space.

[0095] 4. The method according to embodiment 2, wherein a specific one of a plurality of instructions: at the same time as permission is granted to the first database node, the first database node will perform all record writes to the key space identified by the specific instruction.

[0096] 5. The method according to embodiment 4 further includes:

[0097] The first database node receives a relinquishment request to the second database node, allowing the second database node to write records to the key space as part of an active transaction on the second database node; and

[0098] In response to the abandonment request, the first database node abandons the license.

[0099] 6. The method according to embodiment 1, wherein a specific epoch range of an identifier key space is identified among a plurality of indications and associated with a second database node, and wherein the method further comprises:

[0100] The first database node determines that the epoch corresponding to the database transaction falls within the epoch range, wherein this determination indicates that the second database node has written records to the key space within a specific time period.

[0101] 7. The method according to Example 6 further includes:

[0102] In response to the commit of the last active transaction linked to a specific instruction, modify the epoch range, where no epoch was defined for the upper limit of the epoch range prior to the modification.

[0103] 8. The method according to embodiment 6 further includes:

[0104] After all records written to the keyspace at the second database node are stored in the persistent database of the database system, a specific instruction is deleted.

[0105] 9. The method according to embodiment 1, wherein the plurality of indications includes a set of indications for a key space, and wherein one of the indications in the set identifies a database node that is permitted to write records for the key space, and wherein two or more of the indications in the set identify a database node that reads records written for the key space.

[0106] 10. The method according to Embodiment 1 further includes:

[0107] In response to determining that the second database node has written a record for the key, the first database node terminates at least a portion of the database transaction involving the writing of the specific record.

[0108] 11. A non-transitory computer-readable medium having stored thereon program instructions executable by a first database node of a database system to cause the first database node to perform an operation, the operation comprising:

[0109] Receive requests to execute database transactions, which include writing specific records for the keys included in the key space;

[0110] Access the key space reference directory, which stores multiple indicators of when database nodes of the database system have written to the key space;

[0111] In response to determining that the second database node has written records to the key space within a specific time period, a request is sent to the second database node for information indicating whether the second database node has written records to the key.

[0112] Based on the response received from the second database node, determine whether to write a specific record.

[0113] 12. The medium according to embodiment 11, wherein a particular one of the plurality of indications corresponding to the second database node identifies the key space by a minimum key and a maximum key, and wherein the particular indication specifies a time period containing a specific time period.

[0114] 13. The medium according to Example 11, wherein the operation further includes:

[0115] Request approval to write records to the keyspace at the first database node;

[0116] Receive permission to write records to the key space; and

[0117] This allows a specific instruction to be stored in the keyspace reference directory, where the specific instruction indicates that writes to all records in that keyspace will occur at the first database node.

[0118] 14. The medium according to Example 13, wherein the operation further includes:

[0119] Execute another database transaction, which includes writing another specific record to the keys included in the key space;

[0120] The keyspace reference directory is used to determine that permission to write records to the keyspace has been transferred to the third database node; and

[0121] Send a write request to the third database node to write another specific record.

[0122] 15. The medium according to Example 11, wherein the operation further includes:

[0123] In response to determining that the second database node has written a record for the key, wait before writing the specific record until the record written by the second database node has been committed.

[0124] 16. A method comprising:

[0125] The database system maintains a keyspace reference directory that stores multiple indicators related to the keyspace;

[0126] The database system allocates key space to the first database node of the database system, wherein the first specific indicator among a plurality of indicators identifies the first time period and the second database node of the database system that previously allocated key space, such that the second database node is allowed to write records whose keys fall into the key space at the second database node;

[0127] The first database node executes a transaction involving key write records targeting the key space, wherein the execution includes:

[0128] Based on a first specific instruction, it is determined that the first time period overlaps with a second time period associated with the transaction; and

[0129] In response to this determination, a request is sent to the second database node to indicate whether a record has been written by the second database node but not committed by the second database node.

[0130] 17. The method according to embodiment 16 further includes:

[0131] The database system receives an upgrade request to perform a rolling upgrade on the second database node, wherein, in response to receiving the upgrade request, an allocation is performed.

[0132] 18. The method according to embodiment 16, wherein the allocation includes:

[0133] The database system adds a second specific indicator to the keyspace reference directory, wherein the second specific indicator specifies an open state, indicating that a write to all records in the keyspace will occur at the first database node; and

[0134] The database system updates a first specific indicator to specify a closed state, which indicates that no keyspace record writes will occur at the second database node.

[0135] 19. The method according to embodiment 16, wherein the execution includes:

[0136] Based on the keyspace reference directory, determine that at least two of a plurality of instructions belong to the keyspace, wherein the at least two instructions include a first specific instruction; and

[0137] In response to determining that the time period identified by one of at least two indications, which is different from the first specific indication, overlaps with the second time period, a request is sent to the third database node to indicate whether the information indicating the record has been written to the third database node for that key but has not been committed by the third database node.

[0138] 20. The method according to Example 16, wherein,

[0139] In response to the determination that the second database node has not written a record for that key, the first database node writes that specific record.

[0140] The published text includes references to “implementations,” which are non-limiting ways of realizing the disclosed concepts. References to “an implementation,” “one implementation,” “a specific implementation,” “some implementations,” “various implementations,” etc., do not necessarily refer to the same implementation. A large number of possible implementations are envisioned, including specific implementations described in detail, as well as modifications or substitutions falling within the spirit or scope of the published text. Not all implementations necessarily exhibit any or all of the potential advantages described herein.

[0141] Public texts include references to “one” implementation or a set of “implementations” (e.g., “some implementations” or “various implementations”). An implementation is a different way of realizing or being implemented of the disclosed concepts. References to “implementation,” “one implementation,” “specific implementation,” etc., do not necessarily refer to the same implementation. A large number of possible implementations are envisioned, including those specifically disclosed implementations as well as modifications or substitutions that fall within the spirit or scope of the public text.

[0142] The disclosure may discuss potential advantages that may arise from the disclosed embodiments. Not all implementations of these embodiments necessarily exhibit any or all of the potential advantages. Whether an advantage is realized for a particular implementation depends on many factors, some of which are beyond the scope of the disclosure. In fact, there are many reasons why an implementation falling within the scope of the claims may not exhibit some or all of the disclosed advantages. For example, a particular implementation may include circuitry outside the scope of the disclosure that, when combined with one of the disclosed embodiments, negates or diminishes one or more of the disclosed advantages. Furthermore, suboptimal design execution of a particular implementation (e.g., the implementation technique or tool) may also negate or diminish the disclosed advantages. Even assuming skillful implementation, the realization of an advantage may still depend on other factors, such as the environmental circumstances in which the implementation is deployed. For example, the inputs provided to a particular implementation may prevent one or more problems addressed in the disclosure from occurring in a particular context, and as a result, the benefits of its solution may not be realized. Given the possibility of factors beyond the disclosure, it is expressly stated that any potential advantages described herein should not be construed as a limitation of the claims that must be satisfied to prove infringement. Rather, the identification of such potential advantages is intended to illustrate the type of improvement available to a designer who benefits from the disclosure. Describing these advantages permissively (e.g., stating that a particular advantage "may occur") is not intended to convey doubt about whether these advantages can actually be realized, but rather to acknowledge that the realization of these advantages often depends on the technological reality of additional factors.

[0143] Unless otherwise stated, the embodiments are non-limiting. That is, the disclosed embodiments are not intended to limit the scope of the claims drafted based on the disclosure, even if only a single embodiment is described for a particular feature. The disclosed embodiments are intended to be illustrative rather than limiting, and there is no statement to the contrary in the disclosure. Therefore, this application is intended to allow for claims covering the disclosed embodiments, as well as substitutions, modifications, and equivalents that will be apparent to those skilled in the art who benefit from the disclosure.

[0144] For example, features in this application can be combined in any suitable manner. Therefore, during the examination of this application (or an application claiming priority thereto), new claims can be made for any such combination of features. In particular, referring to the appended claims, features of dependent claims can be combined with features of other dependent claims, including claims dependent on other independent claims, where appropriate. Similarly, features from various independent claims can be combined where appropriate.

[0145] Therefore, while appended dependent claims can be drafted such that each dependent claim depends on a single other claim, additional dependent claims may also be considered. Any combination of features in dependent claims consistent with the published text is contemplated and may be claimed in this application or another application. In short, combinations are not limited to those specifically listed in the appended claims.

[0146] Where appropriate, it is also anticipated that claims drafted in one format or statutory type (e.g., apparatus) are intended to support corresponding claims in another format or statutory type (e.g., method).

[0147] Because the published text is a legal document, various terms and phrases may be subject to administrative and judicial interpretation. Therefore, it is hereby announced that the following paragraphs, as well as the definitions provided throughout the publication, will be used to determine how to interpret the claims drafted based on the published text.

[0148] Unless the context clearly specifies otherwise, references to the singular form of an item (i.e., a noun or noun phrase preceded by "a," "an," or "the") are intended to mean "one or more." Therefore, without accompanying context, a reference to "item" in a claim does not exclude additional instances of that item. "A plurality of" items refers to a group of two or more items.

[0149] The word "may" used in this article is in a permissive sense (i.e., possible, able) rather than in a mandatory sense (i.e., mandatory).

[0150] The terms “including” and “comprise” and their forms are open-ended, meaning “including but not limited to”.

[0151] When the term “or” is used in public texts concerning a list of options, it is generally understood to be used in an inclusive sense unless the context otherwise specifies. Therefore, a statement of “x or y” is equivalent to “x or y or both,” thus encompassing 1) x but not y, 2) y but not x, and 3) both x and y. On the other hand, phrases such as “x or y, but not both” indicate that “or” is used in an exclusive sense.

[0152] The statements “w, x, y, or z or any combination thereof” or “...at least one of w, x, y, and z” are intended to cover all possibilities involving a single element up to the total number of elements in the set. For example, given the set [w, x, y, z], these terms cover any single element of the set (e.g., w, but excluding x, y, or z), any two elements (e.g., w and x, but excluding y or z), any three elements (e.g., w, x, and y, but excluding z), and all four elements. The phrase “...at least one of w, x, y, and z” therefore refers to at least one element in the set [w, x, y, z], thus covering all possible combinations in the list of elements. This phrase should not be interpreted as requiring the existence of at least one instance of w, at least one instance of x, at least one instance of y, and at least one instance of z.

[0153] In public text, various “labels” may precede nouns or noun phrases. Unless the context otherwise specifies, different labels used for a feature (e.g., “first circuit,” “second circuit,” “specific circuit,” “given circuit,” etc.) refer to different instances of that feature. Furthermore, when applied to features, the labels “first,” “second,” and “third” do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) unless otherwise stated.

[0154] The phrase "based on" is used to describe one or more factors that influence the determination. This term does not exclude the possibility that additional factors might influence the determination. That is, the determination can be based solely on the specified factors or on the specified factors along with other unspecified factors. Consider the phrase "A is determined based on B." This phrase specifies that B is a factor used to determine A or a factor influencing the determination of A. This phrase does not exclude the possibility that the determination of A could also be based on another factor, such as C. This phrase is also intended to cover implementations where A is determined solely based on B. As used herein, the phrase "based on" is synonymous with the phrase "at least partially based on."

[0155] The phrases “responding to” and “responsive” describe one or more factors that trigger an effect. This phrase does not exclude the possibility that additional factors may influence or otherwise trigger the effect, whether in conjunction with or independently of a specific factor. That is, an effect may respond only to those factors, or it may respond to the specified factor as well as other unspecified factors. Consider the phrase “responding to B to execute A.” This phrase specifies that B is a factor that triggers the execution of A or triggers a specific result of A. This phrase does not exclude that the execution of A may also respond to some other factor, such as C. This phrase also does not exclude that the execution of A may jointly respond to B and C. This phrase is also intended to cover implementations where A is executed only in response to B. As used herein, the phrase “responding to” is synonymous with the phrase “at least partially responding to.” Similarly, the phrase “responding to” is synonymous with the phrase “at least partially responding to.”

[0156] In public texts, different entities (which may be referred to differently as “units,” “circuits,” other components, etc.) may be described or required to be “configured” to perform one or more tasks or operations. This expression—[entity] configured to perform one or more tasks—is used in this document to refer to a structure (i.e., something physical). More specifically, this expression is used to indicate that the structure is set up to perform one or more tasks during operation. It can be said that a structure is “configured” to perform some tasks, even if the structure is not currently being operated. Therefore, an entity described or stated as “configured to” perform a task refers to something physical, such as a device, circuit, system with processor units and memory storing program instructions executable to perform that task, etc. This phrase is not used in this document to refer to intangible things.

[0157] In some cases, various units / circuits / components may be described herein as performing a set of tasks or operations. It should be understood that these entities are "configured" to perform those tasks / operations, even if not specifically indicated otherwise.

[0158] The term "configured as" does not mean "configurable as." For example, an unprogrammed FPGA is not considered "configured as" to perform a specific function. However, such an unprogrammed FPGA can be "configurable as" to perform that function. After proper programming, an FPGA can be said to be "configured as" to perform a specific function.

[0159] For the purposes of a U.S. patent application based on a published text, a statement in the claims describing a structure “configured” to perform one or more tasks is not expressly intended to invoke 35 USC § 112(f) for that claim element. If the applicant wishes to invoke section 112(f) during the proceedings of a U.S. patent application based on a published text, a “component” structure for [performing a function] will be used to state the claim element.

[0160] Different “circuits” can be described in public texts. These circuits or “circuit systems” constitute hardware comprising various types of circuit elements, such as combinational logic, clock-controlled storage devices (e.g., flip-flops, registers, latches, etc.), finite state machines, memories (e.g., random access memory, embedded dynamic random access memory), programmable logic arrays, etc. Circuits can be custom-designed or taken from standard libraries. In various implementations, circuits may appropriately include digital components, analog components, or a combination of both. Certain types of circuits are often referred to as “units” (e.g., decoding units, arithmetic logic units (ALUs), functional units, memory management units (MMUs), etc.). Such units also refer to circuits or circuit systems.

[0161] Therefore, the circuits / units / components and other elements disclosed in the accompanying drawings and described herein include hardware elements, such as those described in the preceding paragraphs. In many cases, the internal arrangement of hardware elements in a particular circuit can be specified by describing the function of the circuit. For example, a particular "decoding unit" can be described as having the function of executing "the opcode of a processing instruction and routing that instruction to one or more of a plurality of functional units," meaning that the decoding unit is "configured" to perform that function. To those skilled in the art of computers, such a functional description is sufficient to suggest a set of possible structures for the circuit.

[0162] In various implementations, as discussed in the preceding paragraphs, the arrangement of circuits, cells, and other elements defined by the functions or operations they are configured to perform, relative to each other, and the way such circuits / cells / components interact form the microarchitecture definition of hardware that is ultimately manufactured in an integrated circuit or programmed into an FPGA to form a physical implementation of the microarchitecture definition. Therefore, a microarchitecture definition is considered by those skilled in the art to be a structure from which many physical implementations can be derived, all falling within the broader structure described by the microarchitecture definition. That is, a person skilled in the art with a microarchitecture definition provided in the published text can implement the structure by encoding the description of the circuits / cells / components in a Hardware Description Language (HDL) (e.g., Verilog or VHDL) without excessive experimentation and the application of ordinary skills. HDL descriptions are often expressed in a seemingly functional manner. However, to those skilled in the art, such HDL descriptions are a way to translate the structure of a circuit, cell, or component into the next level of implementation details. Such HDL descriptions can take the form of behavioral code (typically non-synthesizable), Register Transfer Language (RTL) code (as opposed to behavioral code, typically synthesizable), or structural code (e.g., a netlist specifying logic gates and their connections). HDL descriptions can then be synthesized against a cell library designed for a given integrated circuit manufacturing technology and can be modified for timing, power, and other reasons to produce a final design database, which is then transferred to a foundry to generate masks and ultimately manufacture integrated circuits. Some hardware circuitry or portions thereof can also be custom-designed in a schematic editor and captured into the integrated circuit design along with the synthesized circuitry. Integrated circuits can include transistors and other circuit elements (e.g., passive elements such as capacitors, resistors, inductors, etc.) and interconnections between transistors and circuit elements. Some implementations may realize multiple integrated circuits interconnected to implement hardware circuitry, and / or discrete elements may be used in some implementations. Alternatively, HDL designs can be synthesized into a programmable logic array such as a field-programmable gate array (FPGA) and implemented in the FPGA. This decoupling between the design of a set of circuits and their subsequent low-level implementations often results in a situation where the circuit or logic designer never specifies a particular set of structures for the low-level implementation beyond describing what the circuit configuration does, as this process is performed at different stages of the circuit implementation process.

[0163] The fact that many different low-level combinations of circuit elements can be used to implement the same circuit specifications results in a large number of equivalent circuit structures. As mentioned above, these low-level circuit implementations can vary depending on the manufacturing technology, the foundry chosen to manufacture the integrated circuits, the cell library provided for a particular project, and so on. In many cases, the choice made by different design tools or methods to produce these different implementations can be arbitrary.

[0164] Furthermore, for a given implementation, a single implementation of the circuit's specific functional specifications typically involves a large number of devices (e.g., millions of transistors). Therefore, the sheer volume of such information makes it impractical to provide a complete description of the low-level structure used to implement a single implementation, let alone a multitude of equivalent possible implementations. For this reason, the published text uses functional abbreviations commonly used in the industry to describe the circuit's structure.

Claims

1. A method comprising: Information is received by the first database node of the database system, indicating that the key space has been provided to the first database node for writing records with keys falling into the key space, wherein the key space was previously provided to the second database node of the database system; The first database node receives a request to execute a database transaction, which includes writing a specific record for a key included in the key space. The first database node accesses the key space reference directory, which stores multiple indications of when the database nodes of the database system have written to the key space; In response to determining that the second database node has written at least one record to the key space within a specific time period, the first database node sends a request to the second database node to obtain information indicating whether the second database node has written a record that conflicts with the specific record to the same key. and Based on the response indicating that the second database node has not written a record that conflicts with the specific record, the first database node writes the specific record.

2. The method according to claim 1, further comprising: Before receiving the request to execute the database transaction, the first database node issues a permission request to the database system to obtain approval to write records to the key space, wherein the information indicating that the key space has been provided to the first database node is received in response to issuing the permission request.

3. The method of claim 2, wherein after the key space is provided to the first database node, an active transaction on the second database node including writing records to the key space is permitted to be committed, and wherein at least one of the active transactions has resulted in a record write to the key space before the permission is provided to the first database node.

4. The method of claim 2, wherein a particular one of the plurality of indications indicates that, while the key space is provided to the first database node, the first database node will perform writes to all records in the key space identified by the particular indication.

5. The method according to claim 4, further comprising: The first database node receives a request to relinquish the key space to the second database node, thereby allowing the second database node to write records to the key space as part of an active transaction on the second database node. and In response to the abandonment request, the first database node abandons the key space.

6. The method of any one of claims 1 to 5, wherein a particular one of the plurality of indications identifies an epoch range of the key space and is associated with the second database node, and wherein the method further comprises: The first database node determines that the epoch corresponding to the database transaction falls within the epoch range, wherein the determination indicates that the second database node has written records to the key space within the specific time period. The epoch range refers to the time period during which a database transaction is committed to the corresponding key space, defined by specifying the minimum and maximum transaction commit numbers. The minimum transaction commit number identifies the most recent refresh transaction commit number when the keyspace reference was created, and the maximum transaction commit number identifies the transaction commit number associated with the last database transaction written to the keyspace before being provided to another database node.

7. The method according to claim 6, further comprising: In response to the commit of the last active transaction linked to the specific indication, the epoch range is modified, wherein no epoch was defined for the upper limit of the epoch range prior to the modification.

8. The method according to claim 6, further comprising: The specific instruction is deleted after all records written to the keyspace at the second database node are stored in the persistent database of the database system.

9. The method of claim 1, wherein the plurality of indications includes a set of indications for the key space, and wherein one of the set of indications identifies a database node that is allowed to write records for the key space, and wherein two or more of the set of indications identify the database node that reads records written for the key space at the database node.

10. A computer-readable medium having program instructions stored thereon, the program instructions being executable by a first database node of a database system to cause the first database node to perform operations including: Receive information indicating that the key space has been provided to the first database node for writing records with keys falling into the key space, wherein the key space was previously provided to the second database node of the database system; Receive a request to execute a database transaction, the database transaction including writing a specific record for a key included in the key space; Access the key space reference directory, which stores multiple indications of when database nodes of the database system have written to the key space; In response to determining that the second database node has written at least one record to the key space within a specific time period, a request is sent to the second database node to obtain information indicating whether the second database node has written a record that conflicts with the specific record to the same key; and Based on the response indicating that the second database node has not written a record that conflicts with the specific record, the specific record is written.

11. The medium of claim 10, wherein a particular one of the plurality of indications corresponding to the second database node identifies the key space by a minimum key and a maximum key, and wherein the particular indication specifies a time period containing the particular time period.

12. The medium of claim 10, wherein the operation further comprises: A request for approval to write a record to the key space at the first database node, wherein the information indicating that the key space has been provided to the first database node is received in response to the request for approval; and A specific instruction is stored at the key space reference directory, wherein the specific instruction indicates that a write to all records in the key space will occur at the first database node.

13. The medium of claim 12, wherein the operation further comprises: Execute another database transaction, which includes writing another specific record for the keys included in the key space; The keyspace reference directory is used to determine that permission to write records to the keyspace has been transferred to the third database node; and Send a write request to the third database node to write the other specific record.

14. A computer system, comprising: At least one processor; and A memory having stored program instructions executable by the at least one processor to perform the method according to any one of claims 1 to 9.

Images