A database system and method supporting parallel updates

Abstract

This invention provides a database system and method supporting parallel updates, relating to the field of database technology. The system includes: a concurrency control protocol that caches the maximum read timestamp, maximum write timestamp, and maximum normal write timestamp for each data item in the database; a write-ahead log module that records normal logs and incremental logs; a caching module that caches the normal record with the highest version number of the data item and all subsequent incremental records; a persistence module that saves the cached records to a persistent medium; a write flow control module that controls write operations, and if the transaction timestamp is greater than the maximum read timestamp, performs normal writes and incremental writes respectively, generating normal records and incremental records, and updating the maximum write timestamp and maximum normal write timestamp; a read flow control module that maintains a dependency graph based on the dependencies between read and write transactions; and executes read transactions after all write transactions in the dependency graph have been completed. This invention supports parallel updates, guarantees ACID and serializable isolation levels, and eliminates cascading rollbacks.

Description

Technical Field

[0001] This invention relates to the field of database technology, and in particular to a database system and method that supports parallel updates. Background Technology

[0002] Concurrency control is a mechanism used to protect database integrity and ensure timely correction of errors caused by concurrent operations when multiple users execute write transactions simultaneously. The basic unit of concurrency control is the transaction. Incorrect concurrency mechanisms can lead to problems such as dirty reads, phantom reads, and non-repeatable reads. The purpose of concurrency control is to ensure that the work of one user does not unreasonably affect the work of another user. In some cases, these measures guarantee that when a user operates with other users, the result is the same as the result obtained when the user operates alone.

[0003] Traditionally, database systems use the following four schemes to serialize conflicting concurrent transactions: (1) two-phase locking, (2) timestamp-based protocols, typically represented by timestamp ordering, (3) optimistic concurrency control, and (4) allowing dirty reads, i.e. allowing cascading rollbacks (e.g., RubatoDB, Hyper). The first three methods only support concurrent updates and cannot support parallel updates. Although the fourth method supports parallel updates, it involves cascading scheduling, i.e., cascading rollbacks are usually a bad design because the rollback of one transaction may cause hundreds or thousands of subsequent transactions to roll back, greatly affecting the system's performance and causing severe performance jitter.

[0004] Concurrent updates involve assigning concurrent transactions to the processor at different times, meaning that concurrent transactions do not run simultaneously at the same time. Parallel updates, on the other hand, mean that conflicting update transactions can be performed independently of each other, meaning that conflicting transactions can run simultaneously at the same time.

[0005] In 2018, Microsoft proposed FasterKV. Although it can support parallel updates, it cannot guarantee ACID. ACID refers to the four properties that a database management system (DBMS) must possess to ensure that transactions are correct and reliable during the writing or updating of data: atomicity (or indivisibility), consistency, isolation (or independence), and durability. Therefore, FasterKV is not a database system in the strict sense, let alone a reliable database system that supports parallel updates. Summary of the Invention

[0006] To address the issues of traditional solutions failing to support parallel updates in concurrent transactions within database systems, and employing dirty reads that allow cascading rollbacks for parallel updates, which severely impact database performance, and the inability of the new FasterKV solution to guarantee ACID, this invention provides a database system and method that supports parallel updates. This system supports parallel updates while guaranteeing ACID and serializable isolation levels, ensuring no cascading between schedules, eliminating the need for cascading rollbacks, and supporting distributed transactions.

[0007] To achieve the above objectives, the present invention provides a database system that supports parallel updates, including: a concurrency control protocol, a write-ahead log module, a caching module, a persistence module, a write flow control module, and a read flow control module;

[0008] The concurrency control protocol is used for:

[0009] At the start of each transaction, a timestamp is obtained from the timestamp service and used as the version number for all subsequent records of that transaction.

[0010] For each data item Q read in the database, the timestamp of the largest transaction that read that data item is cached in memory as the maximum read timestamp; for range queries in the database, the range is decomposed into non-overlapping intervals, and the timestamp of the largest transaction that reads these intervals is cached in memory as the maximum inter-interval read timestamp.

[0011] For each data item that has been written in the database, the timestamp of the largest transaction that has written to that data item and the timestamp of the largest transaction that has written to that data item are cached in memory respectively, and used as the maximum write timestamp and the maximum ordinary write timestamp.

[0012] The write-ahead log module is used for:

[0013] Record logs in two formats: regular logs and incremental logs. Write-ahead logs need to be saved synchronously to persistent media.

[0014] The cache module is used for:

[0015] The system caches the highest version number of each data item's ordinary record and all subsequent incremental records in memory. Ordinary and incremental records written by transactions are also stored in the cache module, with a separate Writer object generated for each ordinary or incremental record. Ordinary records record specific, definite values, while incremental records only record the incremental operation itself. Incremental operations are limited to functions that can be represented as functions that only accept the current data item as an argument. Update operations that cannot be represented as functions that only accept the current data item as an argument need to be converted into read-modify-write operations and generate ordinary records. Read-modify-write operations do not support parallel updates. The cache module supports fast lookup of all cached records by data item primary key and version number.

[0016] The persistence module is used for:

[0017] Records in the cache are asynchronously saved to the persistent medium. The persistent module also supports fast lookup of database records by data item primary key and version number.

[0018] The write process control module is used for:

[0019] Determine the timestamp of the transaction writing the data item. If it is less than the maximum read timestamp of that data item cached in memory, roll back the transaction; otherwise, perform write operations separately for regular write operations and incremental update operations.

[0020] For a regular write operation on a data item, further determine whether the timestamp of the transaction that writes the data item is less than the maximum write timestamp of the data item cached in memory. If it is less, roll back the transaction; otherwise, update the maximum write timestamp and the maximum regular write timestamp of the data item, and generate a regular log and a regular record.

[0021] For incremental update operations on data items, further determine whether the timestamp of the transaction that writes the data item is less than the maximum normal write timestamp of the data item cached in memory. If it is less, roll back the transaction; otherwise, update the maximum write timestamp of the data item and generate incremental logs and incremental records.

[0022] The read flow control module is used for:

[0023] A Reader object is generated based on the transaction of the read data item;

[0024] For the data items that need to be read in this transaction, retrieve the last ordinary record with a timestamp less than the transaction's timestamp and all subsequent incremental records from the cache in memory; if the cache is not hit, load the last ordinary record with a timestamp less than the transaction's timestamp and all subsequent records from the persistence module into the cache.

[0025] Based on the dependencies between transactions that read data items and transactions that write data items, a dependency graph of a Reader object is maintained based on the last normal record and all subsequent incremental records.

[0026] After the Writer objects corresponding to the ordinary records and all subsequent incremental records in the dependency graph have finished executing, the Reader object is executed.

[0027] The Reader object merges the values ​​of all the Writer objects it depends on from the dependency graph to obtain the value of the data item to be read, and atomically updates the maximum read timestamp.

[0028] As a further improvement of the present invention, the step of maintaining a dependency graph based on the dependency relationship between transactions of reading data items and transactions of writing data items, and based on the last ordinary record and all subsequent incremental records, includes:

[0029] Each of the above ordinary records or incremental records constitutes a Writer object. The Writer corresponding to an ordinary record is called an ordinary Writer, and the Writer corresponding to an incremental record is called an incremental Writer. The value of the Writer is the corresponding ordinary or incremental record.

[0030] All edges in the dependency graph originate from a Writer object and end at a Reader object.

[0031] As a further improvement to the present invention, a dependency graph of a Reader object is maintained based on the last ordinary record and all subsequent incremental records; including:

[0032] Once the dependency graph of a Reader object is established, if a transaction arrives with a new write data item corresponding to the data item that the Reader object needs to read, the dependency graph of the Reader object needs to be updated.

[0033] When the transaction that wrote to the database corresponding to the last ordinary record or any incremental record is rolled back, the dependency graph of all data items written by that transaction also needs to be updated.

[0034] As a further improvement of the present invention, a dependency graph optimization module is also included for periodically and quantitatively fusing incremental records, including:

[0035] When the transactions corresponding to the regular record of a data item and the subsequent incremental records have been committed, all the above records are merged to generate a new regular record. The new regular record uses the version number of the last incremental record and updates the maximum regular write timestamp. At the same time, the merged record is saved to the persistence module and deleted from the cache.

[0036] As a further improvement of the present invention, a dependency graph optimization module is also included, for:

[0037] When a transaction that reads a data item generates a Reader object r, a corresponding placeholder transaction is generated. The placeholder transaction contains a Reader / Writer pair, where the Writer is a regular Writer, and the timestamp of the placeholder transaction is set to the version number of the last Writer object less than r + 1. A dependency graph is generated for the Reader of the placeholder transaction. Subsequent transactions that read the same data item and whose timestamp is greater than the Reader version number in the placeholder transaction only need to depend on the Writer of the placeholder transaction. When the Reader object corresponding to the placeholder transaction finishes execution, the value of the Writer corresponding to the placeholder transaction is replaced with the read value, and the placeholder transaction is committed.

[0038] As a further improvement of the present invention, it also includes a constraint update and condition update module;

[0039] The constraint update and condition update module is used for:

[0040] When updating data items with constraints, if the transaction writing the data item is an incremental update operation, the value range of the data item after the current incremental update operation is calculated, and it is determined whether there is an intersection with the constraint range. If there is no intersection, it is proven that the incremental update operation is illegal and the transaction needs to be rolled back; otherwise, it is further determined whether it is a subset of the constraint range. If it is a subset of the constraint range, it is proven that the operation is definitely legal, and the transaction continues to be executed to complete the incremental update of the data item with constraints; otherwise, the operation will be converted into a read-modify-write operation and parallel updates are no longer allowed.

[0041] When updating data items with conditions, if the transaction writing the data item is an incremental update operation, the domain range of the data item before the current incremental update operation is calculated, and it is determined whether there is an intersection with the update conditions. If there is no intersection, it proves that the incremental update operation is invalid and the transaction needs to be rolled back; otherwise, it is further determined whether it is a subset of the update conditions. If it is, it proves that the operation must be valid and the transaction can continue to be executed; otherwise, the operation will be transformed into a read-modify-write operation and parallel updates can no longer be performed.

[0042] As a further improvement to the present invention, the method for defining and updating the domain range and the value range includes:

[0043] Assuming the initial committed value of a data item is v, then the range of values ​​(value range) for the data item is [v, v], denoted as range. key The domain is also [v, v], denoted as domain. key ;

[0044] For each incremental update operation DeltaWriterw, where wf represents the function corresponding to the w update operation, then:

[0045] range key =wfmap(domain) key )

[0046] domain key =domain key ∪range key

[0047] For updates to the value range and domain range, non-commutative update operations are performed in order of transaction timestamp size; for update operations that are independent of order, out-of-order updates can be performed.

[0048] As a further improvement of the present invention, an index update module is also included;

[0049] The index update module is used for:

[0050] When a unique index exists on the column of the updated data item, the update operation needs to be transformed into a read-modify-write operation and parallel updates are no longer possible.

[0051] Even when a non-unique index exists on the column being updated in the data item being updated, parallel updates can still be performed using the following method:

[0052] When updating data items, non-unique index items are created. Non-unique index items include:

[0053] The primary key column is encoded as: Index Name - Index Value - Primary Key Value of the Indexed Data Item - Transaction Version Number, where the index value is the value of the index column and the transaction version number is the version number of the transaction that created the index; non-primary key columns record the metadata information of the index items and are called index metadata columns.

[0054] For incremental record data items, the index value is set to an ambiguous value, and the value in the index metadata column is the value range of that data item; for index items of ordinary records, the index value is a definite value, and the index metadata column is empty.

[0055] For a query using an index, indexed_column∈C, let the timestamp of the query transaction be TS. qFirst, update the query condition C∪{undefined} maximum gap read timestamp. Then, scan index items whose index values ​​are not undefined and add them to the result set R if they meet the query condition and transaction visibility. Next, scan index items whose index values ​​are undefined. First, determine if the visibility is satisfied. If not, skip the index item. Otherwise, continue to determine the relationship between the value range of the index item and the query condition set. If the value range of the index item and the query condition do not intersect, skip the data item. Otherwise, if the value range of the index item is a subset of the query condition set, add the index item to the result set R. Otherwise, register a Reader object for the data item corresponding to the primary key value of the indexed data item. Set the timestamp of the Reader object to the version number of the index item. Execute the read process for the Reader. After execution, the index value of the index item is the value read by the Reader. It can be compared with the query condition. If it does not meet the condition, skip it. Otherwise, add it to the result set R.

[0056] As a further improvement of the present invention, the process of converting the update operation into a read-modify-write operation is as follows: First, register a Reader and a placeholder Writer using the current transaction timestamp as the version number, execute the read process to obtain the value of the current data item, apply the update operation, and then replace the value in the placeholder Writer with the updated value and execute the write process.

[0057] As a further improvement of the present invention, the index update module is also used to clean up ambiguous indexes, including:

[0058] A large number of ambiguous indexes in the system can significantly slow down query performance. It is necessary to clean up ambiguous indexes regularly. The cleanup methods are as follows:

[0059] Scan for ambiguous indexes. For each ambiguous index, register a Reader object for the data item corresponding to the primary key value of the indexed data item, and wait for all Writer objects that the Reader object depends on to commit. At this point, the value of the index item becomes a definite value. Create a new index item, and use the definite value for the index value. After creation, delete the corresponding ambiguous index item.

[0060] The present invention also provides a method for a database system that supports parallel updates, including: a read transaction concurrency control method and a write transaction concurrency control method;

[0061] The write transaction concurrency control method includes:

[0062] At the start of each transaction, a timestamp is obtained from the timestamp service and used as the version number for subsequent records of each transaction;

[0063] If the transaction's timestamp is less than the maximum read timestamp, rollback is performed; otherwise, write operations are performed separately for regular write operations and incremental update operations.

[0064] For a regular write operation on a data item, further determine whether the timestamp of the transaction that writes the data item is less than the maximum write timestamp of the data item cached in memory. If it is less, roll back the transaction; otherwise, update the maximum write timestamp and the maximum regular write timestamp of the data item, and generate a regular log and a regular record.

[0065] For incremental update operations on data items, further determine whether the timestamp of the transaction that writes the data item is less than the maximum normal write timestamp of the data item cached in memory. If it is less, roll back the transaction; otherwise, update the maximum write timestamp of the data item and generate incremental logs and incremental records.

[0066] The read transaction concurrency control method includes:

[0067] At the start of each transaction, a timestamp is obtained from the timestamp service and used as the version number for subsequent records of each transaction;

[0068] For each data item read in a transaction, a Reader object is generated separately;

[0069] For each data item that a transaction needs to read, retrieve the last ordinary record in the cache that is less than the corresponding transaction timestamp and all subsequent incremental records; if the cache is not hit, load the last ordinary record that is less than the transaction timestamp and all subsequent records from the persistence module into the cache.

[0070] Based on the dependencies between transactions that read data items and transactions that write data items, a dependency graph of a Reader object is maintained based on the last normal record and all subsequent incremental records.

[0071] The Reader object is executed when the last ordinary record and all subsequent incremental records in the dependency graph are committed data.

[0072] The Reader object merges the values ​​of all the Writer objects it depends on from the dependency graph to obtain the value of the data item to be read, and atomically updates the maximum read timestamp.

[0073] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0074] Compared to traditional databases where all update operations must be executed serially, requiring at least 1000 * RTT to complete 1000 incremental update transactions, this invention designs two different data record formats and a carefully designed concurrency control system. This allows the concurrency control protocol to avoid handling incremental write conflicts, enabling multiple incremental update transactions to be processed in parallel. Completing 1000 incremental update transactions requires only 1 RTT, significantly improving the throughput and latency of the database in concurrent update scenarios. Compared to traditional databases, this invention offers a performance improvement of thousands of times in concurrent update scenarios.

[0075] This invention ensures that read transactions can read all write records smaller than the read transaction by maintaining a Writer->Reader dependency graph, and atomically updates the timestamp of the largest transaction of the data item after all the Writers that the Reader depends on have committed, thereby ensuring the serializability of the schedule.

[0076] This invention achieves a serializable isolation level and eliminates cascading rollbacks.

[0077] This invention significantly reduces the number of edges in the dependency graph and improves processing efficiency by introducing placeholder transactions.

[0078] This invention supports parallel updates of data items with constraints and indexes. Attached Figure Description

[0079] Figure 1 This is a schematic diagram of a database system supporting parallel updates disclosed in an embodiment of the present invention;

[0080] Figure 2 This is a flowchart of a write transaction concurrency control method disclosed in one embodiment of the present invention;

[0081] Figure 3 This is a flowchart of a read transaction concurrency control method disclosed in one embodiment of the present invention. Detailed Implementation

[0082] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0083] The present invention will now be described in further detail with reference to the accompanying drawings:

[0084] like Figure 1 As shown, the present invention provides a database system that supports parallel updates, including: a concurrency control protocol, a write-ahead log module, a caching module, a persistence module, a write flow control module, a read flow control module, a dependency graph optimization module, and a condition update module;

[0085] (1) Concurrency control protocol, used for:

[0086] At the start of each transaction, a timestamp is obtained from a timestamp service (which can be a centralized timestamp server, or TrueTime or HLC) and used as the version number for all subsequent records of that transaction.

[0087] For each data item Q read in the database, the timestamp of the largest transaction that reads that data item is cached in memory as the maximum read timestamp (maxReadTimestamp); for range queries in the database, the range is decomposed into non-overlapping intervals, and the timestamp of the largest transaction that reads these intervals is cached in memory, called the gap maximum read timestamp (gap maxReadTimestamp).

[0088] For each data item that has been written in the database, the timestamp of the largest transaction that wrote the data item and the timestamp of the largest normal record transaction that wrote the data item are cached in memory, respectively, as the maximum write timestamp (maxWriteTimestamp) and the maximum normal write timestamp (maxNormalWriteTimestamp).

[0089] (2) The write-ahead log module is used for:

[0090] Logs are recorded in two formats: regular logs and incremental logs. Write-ahead logs need to be saved synchronously to persistent media. Write-ahead logs can use techniques such as group commit to improve the system's throughput.

[0091] (3) Caching module, used for:

[0092] The cache module stores the highest version number of each data item's ordinary record and all subsequent incremental records in memory. Ordinary and incremental records written by transactions are also stored in the cache module, with a separate Writer object generated for each ordinary or incremental record. Ordinary records record specific, definite values, while incremental records only record the incremental operation itself. Incremental operations are limited to operations that can be represented as functions that only accept the current data item as an argument, such as f(q) = q + 1, f(q) = q * q, f(q) = q - 1 if q > 1 elseq, etc. Update operations that cannot be represented as functions that only accept the current data item as an argument need to be converted into ordinary records through a read-modify-write process. The cache module supports fast lookup of all records by data item primary key and version number. The cache module can use data structures such as red-black trees or skiplists.

[0093] (4) Persistence module, used for:

[0094] Records in the cache are asynchronously saved to persistent media (such as HDD, SSD, etc.). The persistence module can use B-tree or LSM-tree, etc.

[0095] (5) Write the flow control module, such as Figure 2 As shown, it is used for:

[0096] Determine the timestamp of the transaction writing the data item. If it is less than the maximum read timestamp (maxReadTimestamp) of the data item cached in memory, roll back the transaction; otherwise, perform the write data item operation, including:

[0097] For normal write operations on data items, such as UPDATE store SET v = 123 WHERE Epk = 1 or INSERT INTO STORE(pk, v) VALUES(1, 123), no cache preloading is required. However, it is necessary to further check whether the timestamp of the transaction writing the data item is less than the maximum write timestamp (maxWriteTimestamp) of the data item cached in memory. If it is less, the transaction is rolled back; otherwise, the maximum write timestamp (maxWriteTimestamp) and the maximum normal write timestamp (maxNormalWriteTimestamp) of the data item are updated; log records (WAL log) and normal records are generated.

[0098] For incremental update operations on data items, such as UPDATE t SET v = v + 1, if the data item exists in the cache, no cache preloading is required; otherwise, the data item is preloaded from the persistent medium. Then, it is necessary to further determine whether the timestamp of the transaction that wrote the data item is less than the maximum normal write timestamp (maxNormalWriteTimestamp) of the data item cached in memory. If it is less, the transaction is rolled back to avoid writing an expired data item; otherwise, the maximum write timestamp (maxWriteTimestamp) of the data item is updated, and the incremental update operation on the data item is completed, generating incremental records and logs. The incremental record actually represents the update action itself, i.e., f(x). In the above example, the log record can be represented as ADD 1.

[0099] in,

[0100] Rolling back a transaction includes deleting all data items written by the transaction and updating the dependency graph of all data items written by the transaction.

[0101] In addition to rolling back the transaction directly, you can also choose not to roll back the transaction, but instead push up the transaction version number and recheck the validity of all previous read results. If any read result is no longer valid, then roll back the transaction.

[0102] When preloading the data item from the persistent media, if the data item does not exist in the persistent media, a NotExists error is thrown; if the data item exists in the persistent media, the last normal version and subsequent incremental versions are loaded into memory.

[0103] The last normal record and all subsequent incremental records of each data item are cached in memory, and a Writer object is generated for each normal record or incremental record. A Normal Writer is generated for normal records and a Delta Writer is generated for incremental records.

[0104] in,

[0105] A Writer object contains a transaction ID, value information, and metadata. The metadata includes dirty data and committed data. Writer objects are also divided into Normal Writer and Delta Writer.

[0106] Furthermore,

[0107] Once all data items in a transaction have been written, the transaction writes a transaction record. Each data item written by the transaction contains a pointer to the transaction record. After the transaction record and all data items written by the transaction have been successfully replicated (using consistency protocols such as Paxos / Raft), the ordinary or incremental records generated by the transaction's data writing operations can be considered committed data. After commit, an asynchronous cleanup process begins, modifying the metadata of the data items written by the transaction and changing the dirty data flag to committed. Before the asynchronous cleanup process ends, if other transactions read dirty data items, they can query the transaction record to obtain the transaction's commit status.

[0108] It is displayed as dirty data before it is shown as committed data;

[0109] When a transaction is committed, all written data items of that transaction are checked and the dependency graph is updated. If all write transactions that depend on a Reader that depends on that transaction have been committed, then that Reader can be fired.

[0110] Specific examples are as follows:

[0111] T0 is a non-look-write transaction, as follows:

[0112] T0:BEGIN;INSERT INTO store(pk,v)VALUES(1,123);COMMIT;

[0113] After T0 executes successfully, two concurrent transactions T1 and T2 enter the system.

[0114] T1:BEGIN;UPDATE t SET v=v+1WHERE pk=1;

[0115] T1:BEGIN; UPDATE t SET v=v+2WHERE pk=1;

[0116] Since the two transactions mentioned above have not yet been committed, they are currently in a pending (waiting) state. In this database system, the following WAL (Wait and See) will be generated.

[0117] (T0, 1, 123) (Normal Record)

[0118] (T1, 1, +1), where the + sign indicates a delta record.

[0119] (T2, 1, +2)

[0120] The records in the database cache will contain the following three versions.

[0121]

[0122] The last normal record (non-delta) plus all subsequent delta records can be cached in memory.

[0123] In all three versions above, after successfully writing record (v:123):

[0124] First, the method to update the maximum write timestamp of this data item is as follows:

[0125] maxWriteTimestamp=max(maxWriteTimestamp, TS(Writer)

[0126] Then maxWriteTimestamp will be updated to 0;

[0127] Second: Update the maximum ordinary write timestamp of this data item:

[0128] maxNormalWriteTimestamp is 0.

[0129] In the above three versions, after a record (v:+2) is successfully written, the method to update only the maximum write timestamp of the data item without updating the maximum normal write timestamp is as follows:

[0130] maxWriteTimestamp=max(maxWriteTimestamp, TS(Writer)

[0131] Then maxWriteTimestamp will be updated to 2;

[0132] maxNormalWriteTimestamp remains unchanged (still 0).

[0133] (6) Read the flow control module, such as Figure 3 As shown, it is used for:

[0134] A Reader object is generated based on the transaction of the read data item, containing information such as the transaction version number;

[0135] For the data items that need to be read in this transaction, retrieve the last ordinary record in memory that is less than the timestamp of this transaction and all subsequent incremental records;

[0136] Based on the dependencies between transactions that read data items and transactions that write data items, and based on the last ordinary record and all subsequent incremental records, a dependency graph of a Reader object is maintained. This process is also called registering a Reader. At the same time, if the transaction is a high-priority transaction, the maximum read timestamp (maxReadTimestamp) is updated when registering the Reader; if the transaction is not a high-priority transaction, it is not updated. The update logic is: maxReadTimestamp = max(maxReadTimestamp, TS(Reader)).

[0137] Once the Writer objects corresponding to the last ordinary record and all subsequent incremental records in the dependency graph are committed, the Reader object is executed. The value of the Reader can be read directly from the dependency graph by merging the values ​​of all committed Writer objects of the dependencies. If the Reader is not a high-priority Reader, the maxReadTimestamp of the data item should be updated atomically. The update logic is: maxReadTimestamp = max(maxReadTimestamp, TS(Reader)).

[0138] in,

[0139] Each ordinary or incremental record constitutes a Writer object; all edges of the dependency graph originate from a Writer object and end at a Reader object.

[0140] The dependency graph is stored bidirectionally, meaning that each Reader records all the Writers it depends on, and each Writer records all the Readers that depend on it.

[0141] Specifically,

[0142] For example, assuming the version number of the read transaction is 4, the generated dependency graph would be:

[0143] W0->R4

[0144] W1->R4

[0145] W2->R4

[0146] Furthermore,

[0147] Once the dependency graph of a Reader object is established, if a new transaction arrives to write that data item, the dependency graph of that Reader object needs to be updated, including:

[0148] If the arriving Writer is a NormalWriter (denoted as W)normal ), and let the timestamp of this Writer be TS(W normal If ), then find the value less than TS(W). normal The maximum Writer; if no such Writer exists, it means no Reader depends on the Writer. normal If the dependency graph does not need to be updated, then the dependency graph does not need to be updated; if such a Writer exists, denoted as W. floor Find the dependency W floor For all Readers, for each ReaderR, place R in W. floor All previous dependencies have been removed (including W). floor (Itself), and add a new dependency W normal →R;

[0149] If the arriving Writer is DeltaWriter (denoted as W) delta ), and let the timestamp of this Writer be TS(W delta If ), then find the value less than TS(W). delta The maximum Writer; if no such Writer exists, it means no Reader depends on the Writer. delta If the dependency graph does not need to be updated, then the dependency graph does not need to be updated; if such a Writer exists, denoted as W. floor Find the dependency W floor For all Readers, for each ReaderR, add a new dependency W. delta →R.

[0150] If the transaction is rolled back, the dependency graph related to all Writers of that transaction needs to be updated. Specifically, for each Writer:

[0151] a. If the Writer is a Delta Writer, then remove the Writer from the dependency graph of all Readers that depend on it.

[0152] b. If the Writer is a Normal Writer, then the dependency graph of all Readers that depend on the Writer needs to be recalculated.

[0153] (7) Dependency graph optimization module, used to optimize the number of edges;

[0154] In high-concurrency scenarios, if many incremental versions and many Readers exist simultaneously, the number of edges in the dependency graph will increase dramatically. Optimizing the number of edges in the dependency graph can improve transaction processing efficiency, including:

[0155] 1) Periodically and quantitatively merge incremental records: When the transactions corresponding to the committed ordinary record and all subsequent incremental records of a data item have been completed, merge all incremental records and generate a new ordinary record; the new ordinary record adopts the version number of the last incremental record.

[0156] 2) Placeholder Transaction Generation and Application: When a read transaction generates a Reader object r, a corresponding placeholder transaction is generated. The placeholder transaction contains a Reader / Writer pair (the Writer is a regular Writer), and the timestamp of the placeholder transaction is set to the version number of the last Writer object less than r + 1. A dependency graph is generated for the placeholder Reader. Subsequent transactions that read the data item and whose timestamp is greater than the version number of the placeholder Reader only need to depend on the placeholder Writer. When the Reader object corresponding to the placeholder transaction finishes execution, the value of the Writer corresponding to the placeholder transaction is replaced with the read value, and the placeholder transaction is committed.

[0157] The present invention also sets the interval for the timestamp service to generate timestamps, ensuring that the version number of the placeholder transaction is different from the version number of any real transaction. For example, if the interval for timestamps is set to 2, the version number of the real transaction will always be even, and the version number of the placeholder transaction is the version number of the real transaction (the version number of the Writer object) + 1, so it will always be odd and will not conflict with the version number of the real transaction.

[0158] Example as follows:

[0159] Dependency graph before optimization:

[0160] ①W0->R8

[0161] W2->R8

[0162] W4->R8

[0163] ②W0->R10

[0164] W2->R10

[0165] W4->R10

[0166] Dependency graph optimized with placeholder transactions (generated placeholder transaction T5, containing R5 and W5 read / write pairs):

[0167] ①W0->R5

[0168] W2->R5

[0169] W4->R5

[0170] ②W5->R8

[0171] W5->R10

[0172] As you can see, the number of edges has decreased.

[0173] Generally, if there are w delta version numbers and r readers, the dependency graph before optimization will have w*r edges, while the dependency graph after optimization will only have w+r edges.

[0174] 3) The system maintains a closed timestamp. All write transactions with timestamps less than this closed timestamp will fail and be retried. Therefore, when generating the dependency graph, all consecutive committed Writers with timestamps less than this closed timestamp can be merged, and the last committed DeltaWriter can be converted to a Normal Writer.

[0175] 4) Define a new isolation level: Traditionally, read committed is used as a transaction isolation level, requiring the guarantee that all committed transactions are read. This invention proposes a new isolation level, read-continuously-committed. Under this isolation level, it is only necessary to read all the last committed ordinary records and all subsequent consecutive committed incremental records. For parallel update conditions, this greatly reduces the workload of reading and improves efficiency. (Note that this isolation level is only applicable to read-only transactions and scenarios where read consistency requirements are not high.) For example:

[0176]

[0177] Although version 3 has been committed, under the read-continuously-committed isolation level, transaction T5 (timestamp 5) will read the records written by T0 and T1, i.e., 124. All reads under this isolation level are non-blocking reads.

[0178] (8) Constraint Update and Condition Update Module

[0179] Constraint updates refer to the updates of data items with constraints, such as `create tablestore(inventory integer not null, constraint v check(inventory>=0))`.

[0180] Conditional updates refer to updates of data items with WHERE conditions, such as `update store set v = v - 1 where v >= 1`.

[0181] When writing incremental records to data items, the value of the data item is not read, so it is impossible to determine whether the update conditions are met, and the update cannot be completed using the parallel update method described above.

[0182] 1) For data items with constraints, when the transaction writing the data item is an incremental update operation, calculate the value range of the data item after the current DeltaWriter incremental update operation, and determine whether it intersects with the constraint range. If it does not, it proves that the incremental update operation is illegal and the transaction needs to be rolled back; otherwise, it further determines whether it is a subset of the constraint range. If it is a subset of the constraint range, it proves that the operation must be legal and the transaction can continue to be executed; otherwise, the operation will be transformed into a read-modify-write operation and parallel updates can no longer be performed.

[0183] The methods for updating the domain and range are as follows:

[0184] Assuming the initial committed value of a data item is v, then the range of values ​​(value range) for the data item is [v, v], denoted as range. key The domain is also [v, v], denoted as domain. key .

[0185] For each DeltaWriterw, wf represents the function corresponding to the w update operation, and .map indicates a call to that function. wfmap(domain) key ) indicates that the function corresponding to the w update operation is applied to the domain. key Perform the calculations to obtain the range, then:

[0186] range key =wfmap(domain) key )

[0187] domain key =domain key ∪range key

[0188] (Because if the Delta Writer commits, then the domain') key =range key If the DeltaWriter rolls back, then the domain' key =domain key constant)

[0189] In practical applications, the union operation can be simplified using a relaxed union, that is, as long as any domain is found. key Make That's all.

[0190] For example:

[0191] The initial value of the inventory of the store with id = 1 is 10, and the inventory constraint is that the value cannot be negative, that is, the constraint range is inventory ∈ [0, ∞);

[0192] There are currently three concurrent transactions. Assume TS(T1) < TS(T2) < TS(T3) and the three transactions arrive in the order of timestamp size.

[0193] T1: update store set inventory = inventory – 5 where id = 1

[0194] T2: update store set inventory = inventory – 11 where id = 1

[0195] T3: update store set inventory = inventory – 6 where id = 1

[0196] After T1 writes but has not committed, the value range of inventory becomes [5, 5]. [5, 5] is a subset of the constraint range [0, ∞), so the operation is legal. The domain range becomes [10, 10] ∪ [5, 5] = [5, 10];

[0197] T2 wants to write. Calculate the value range of inventory and find that the value range becomes [-6, -1], which has no intersection with the constraint range [0, ∞), so the operation is illegal and the transaction should be rolled back;

[0198] T3 wants to write. The value range becomes [-1, 4], which has an intersection with the constraint range [0, ∞) but is not a subset of the constraint range, so it cannot be determined whether the operation is legal and it needs to be degraded to a read - modify - write operation.

[0199] 2) For updates with a where condition, when the transaction writing the data item is an incremental update operation, deduce the domain range of the data item before the current incremental update operation and determine whether there is an intersection with the update condition. If not, it proves that the incremental update operation is illegal and the transaction needs to be rolled back; otherwise, further determine whether it is a subset of the update condition. If so, it proves that the operation must be legal and the transaction can continue to be executed; otherwise, the operation will be transformed into a read - modify - write operation and parallel updates cannot be performed anymore.

[0200] For example:

[0201] The initial value of the inventory of the store with id = 1 is 10, and there is no constraint (constraint) on this table.

[0202] There are currently three concurrent transactions. Assume TS(T1) < TS(T2) < TS(T3) and the three transactions arrive in the order of their timestamps.

[0203] T1: update store set inventory=inventory–5 where id=1 and inventory>=5

[0204] T2: update store set inventory=inventory–11 where id=1 and inventory>=11

[0205] T3: update store set inventory=inventory–"6 where id=1 and inventory>=6

[0206] When T1 executes, the domain is [10, 10], which is a subset of the update condition [5, ∞), so the operation is legal. After the transaction statement is executed, the value range of inventory becomes [5, 5], and the domain becomes [10, 10] ∪ [5, 5] = [5, 10];

[0207] T2 wants to write, but the domain [5, 10] and the update condition [11, ∞) have no intersection, so the operation is illegal and the transaction should be rolled back;

[0208] T3 wants to write. The domain [5, 10] and the condition set [6, ∞) have an intersection but is not a subset of the condition set, so it cannot be determined whether the operation is legal and it needs to be degraded to a read - modify - write operation.

[0209] Generally, the update of the value range and the domain should be carried out in the order of the transaction timestamps. However, if the update operation is a commutative function (e.g., addition operation), out - of - order updates are allowed. In the above example, if the arrival order is T3 -> T2 -> T1, the algorithm still guarantees correctness because the above three transactions are all addition operations, which are commutative, and allowing out - of - order updates can significantly reduce the rollback probability.

[0210] Generally, if an operation satisfies both the commutative law and the associative law (i.e., the operation forms an Abelian group), then the operation is order - independent. For example, pure addition and pure multiplication are order - independent, but generally, the combination of addition and multiplication is not order - independent (but operations in the form of f(x) = (m + 1)x - m are order - independent).

[0211] When applying the system of the present invention, a permitted commutative update, such as addition, can be defined for a specific column of a certain table in the database. Commutative updates allow out-of-order updates of the domain and range. However, for non-commutative updates, out-of-order updates are not allowed. A simple implementation is that for non-commutative updates, if TS(w) < maxWriteTimestamp, then roll back the transaction.

[0212] (9) Index update module

[0213] When there is an index on the column of the data item to be updated, an index entry also needs to be created while updating the data item. Generally speaking, to create an index, the value of the index entry needs to be determined. However, in this concurrent control algorithm, after the incremental update operation is completed, the value of the data item cannot be determined yet. A simple solution is to create the index asynchronously, but the cost is that the consistency of index reads cannot be guaranteed. Therefore, the present invention proposes a method to achieve strong consistency reads for non-unique indexes (for unique indexes, update operations need to be converted into read-modify-write operations and parallel updates are no longer allowed).

[0214] The primary key encoding of the non-unique index entry is: index name - index value - primary key value of the indexed data item - transaction version number. The index entry also includes non-primary key columns, which record the metadata information of the index entry and are called index metadata columns. Among them, for the data item of incremental update, the index value will be set to undefined, and the value of the index metadata column is the range of values of this data item (the calculation method of the range of values is the same as that in conditional updates); when the maximum normal write timestamp exceeds the timestamp of this data item,

[0215] Usage of the index:

[0216] For a query using an index indexed_column ∈ C, let the timestamp of the query transaction be TS qFirst, update the maximum gap read timestamp of the query condition C∪{undefined}. Then, scan index items whose index values ​​are not undefined and add them to the result set R if they satisfy the query condition and transaction visibility. Next, scan index items whose index values ​​are undefined. First, determine if visibility is satisfied. If not, skip the index item. Otherwise, continue to determine the relationship between the value range of the index item and the query condition set. If the value range of the index item and the query condition have no intersection, skip the data item. Otherwise, if the value range of the index item is a subset of the query condition set, add the index item to the result set R. Otherwise, register a Reader object for the data item corresponding to the primary key value of the indexed data item. Set the timestamp of the Reader object to the version number of the index item, and wait for all Writers that the Reader depends on to commit. At this time, the value of the index item becomes a definite value that can be compared with the query condition. If it does not satisfy the query condition, skip it. Otherwise, add it to the result set R.

[0217] Index cleanup:

[0218] A large number of undefined indexes in the system can significantly slow down query performance. Therefore, the system needs to clean up undefined indexes regularly. The cleanup method is as follows:

[0219] Scan for undefined indexes. For each undefined index, register a Reader object for the data item corresponding to the primary key value of the indexed data item. The timestamp of this Reader object is set to the version number written to that index item, and the process waits for all Writers that the Reader depends on to commit. At this point, the value of the index item becomes a definite value. Create a new index item with the definite value mentioned above. After creation, delete the corresponding undefined index item. It should be noted that in a distributed database scenario, although the two index items here may reside on different shards, this update process does not need to rely on distributed transactions because failure to delete the corresponding undefined index item has no impact on the correctness of the system. In the event of a deletion failure, a read transaction may read both the newly created index item and the undefined index item simultaneously, but the read transaction can deduplicate the result set. In addition, this cleanup process is performed asynchronously, and therefore will not block parallel updates.

[0220] For example,

[0221] The initial value of the inventory for the store with id 1 is 10. The inventory value is constrained to be non-negative, and an index idxInventory is created on the inventory column. The data type of the inventory is a 32-bit integer.

[0222] There are three concurrent transactions. Assume that TS(T1) < TS(T2) < TS(T3) and the three transactions arrive in the order of timestamp size.

[0223] T1: update store set inventory = inventory - 1 where id = 1

[0224] T2: update store set inventory = inventory - 2 where id = 1

[0225] T3: update store set inventory = inventory - 3 where id = 1

[0226] In implementation, for convenience, the maximum 32-bit integer 2147483647 can be used to represent undefined. Then:

[0227] T1 will generate the index entry idxInventory - 2147483647 - 1 - 1|[9, 9]

[0228] T2 will generate the index entry idxInventory - 2147483647 - 1 - 2|[7, 8]

[0229] T3 will generate the index entry idxInventory - 2147483647 - 1 - 3|[4, 7]

[0230] If transaction T4 (TS(T3) < TS(T4)) executes the index query SELECT * FROM store WHERE inventory <= 3 and inventory >= 1, the data item 1 will not be hit because If transaction T4 executes the index query SELECT * FROM store WHERE inventory >= 0, the data item 1 will be hit because If transaction T4 executes the index query SELECT * FROM store WHERE inventory >= 5, a Reader needs to be registered for the data item 1. The timestamp of this Reader is 3, and it waits for all Writers on which the Reader depends to commit to further determine whether the index condition is met.

[0231] Advantages of the present invention:

[0232] Compared to traditional databases where all update operations can only be executed serially, requiring at least 1000 * RTT to complete 1000 incremental update transactions, this invention designs two different data record formats. This eliminates the need for the concurrency control protocol to handle write conflicts in incremental updates, allowing multiple incremental update transactions to be processed simultaneously in parallel. Completing 1000 incremental update transactions requires only 1 RTT, significantly improving the throughput and latency of the database in concurrent update scenarios. Compared to traditional databases, this invention offers a performance improvement of thousands of times in concurrent update scenarios.

[0233] This invention maintains a Writer->Reader dependency graph, ensuring that read transactions can read all write records smaller than the read transaction, and atomically updates the maximum read timestamp of the data item and reads the value of the data item after all the Writers that the Reader depends on have finished, thereby ensuring that the scheduling is serializable.

[0234] This invention significantly reduces the number of edges in the dependency graph and improves processing efficiency by introducing placeholder transactions.

[0235] This invention achieves non-blocking reads by defining a new isolation level, read-continuously-committed.

[0236] This invention achieves parallel constraint update / condition update by introducing a domain / range iteration algorithm.

[0237] This invention supports parallel updates of data items with non-unique indexes.

[0238] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A database system supporting parallel updates, characterized in that, include: Concurrency control protocol, write-ahead log module, caching module, persistence module, write flow control module, and read flow control module; The concurrency control protocol is used for: At the start of each transaction, a timestamp is obtained from the timestamp service and used as the version number for all subsequent records of that transaction. For each data item Q that has been read in the database, the timestamp of the largest transaction that has read that data item is cached in memory as the maximum read timestamp; For range queries in the database, the range is broken down into non-overlapping intervals, and the timestamp of the largest transaction that has read these intervals is cached in memory as the maximum read timestamp between intervals. For each data item that has been written in the database, the timestamp of the largest transaction that has written to that data item and the timestamp of the largest transaction that has written to that data item are cached in memory respectively, and used as the maximum write timestamp and the maximum ordinary write timestamp. The write-ahead log module is used for: Record logs in two formats: regular logs and incremental logs. Write-ahead logs need to be saved synchronously to persistent media. The cache module is used for: The system caches the highest version number of each data item's ordinary record and all subsequent incremental records in memory. Ordinary and incremental records written by transactions are also stored in the cache module, with a separate Writer object generated for each ordinary or incremental record. Ordinary records record specific, definite values, while incremental records only record the incremental operation itself. Incremental operations are limited to functions that can be represented as functions that only accept the current data item as an argument. Update operations that cannot be represented as functions that only accept the current data item as an argument need to be converted into read-modify-write operations and generate ordinary records. Read-modify-write operations do not support parallel updates. The cache module supports fast lookup of all cached records by data item primary key and version number. The persistence module is used for: Records in the cache are asynchronously saved to the persistent medium. The persistent module also supports fast lookup of database records by data item primary key and version number. The write process control module is used for: Determine the timestamp of the transaction writing the data item. If it is less than the maximum read timestamp of that data item cached in memory, roll back the transaction; otherwise, perform write operations separately for regular write operations and incremental update operations. For a regular write operation on a data item, further determine whether the timestamp of the transaction that writes the data item is less than the maximum write timestamp of the data item cached in memory. If it is less, roll back the transaction; otherwise, update the maximum write timestamp and the maximum regular write timestamp of the data item, and generate a regular log and a regular record. For incremental update operations on data items, further determine whether the timestamp of the transaction that writes the data item is less than the maximum normal write timestamp of the data item cached in memory. If it is less, roll back the transaction; otherwise, update the maximum write timestamp of the data item and generate incremental logs and incremental records. The read flow control module is used for: A Reader object is generated based on the transaction of the read data item; For the data items that need to be read in this transaction, retrieve the last ordinary record with a timestamp less than the transaction's timestamp and all subsequent incremental records from the cache in memory; if the cache is not hit, load the last ordinary record with a timestamp less than the transaction's timestamp and all subsequent records from the persistence module into the cache. Based on the dependencies between transactions that read data items and transactions that write data items, a dependency graph of a Reader object is maintained based on the last normal record and all subsequent incremental records. After the Writer objects corresponding to the ordinary records and all subsequent incremental records in the dependency graph have finished executing, the Reader object is executed. The Reader object merges the values ​​of all the Writer objects it depends on from the dependency graph to obtain the value of the data item to be read, and atomically updates the maximum read timestamp.

2. The database system supporting parallel updates according to claim 1, characterized in that: The process of maintaining a dependency graph based on the dependencies between transactions that read data items and transactions that write data items, and based on the last ordinary record and all subsequent incremental records, includes: Each of the above ordinary records or incremental records constitutes a Writer object. The Writer corresponding to an ordinary record is called an ordinary Writer, and the Writer corresponding to an incremental record is called an incremental Writer. The value of the Writer is the corresponding ordinary or incremental record. All edges in the dependency graph originate from a Writer object and end at a Reader object.

3. The database system supporting parallel updates according to claim 1, characterized in that: Maintain a dependency graph for a Reader object based on the last regular record and all subsequent incremental records; including: Once the dependency graph of a Reader object is established, if a transaction arrives with a new write data item corresponding to the data item that the Reader object needs to read, the dependency graph of the Reader object needs to be updated. When the transaction that wrote to the database corresponding to the last ordinary record or any incremental record is rolled back, the dependency graph of all data items written by that transaction also needs to be updated.

4. The database system supporting parallel updates according to claim 1, characterized in that: It also includes a dependency graph optimization module for periodically and quantitatively fusing incremental records, including: When the transactions corresponding to the regular record of a data item and the subsequent incremental records have been committed, all the above records are merged to generate a new regular record. The new regular record uses the version number of the last incremental record and updates the maximum regular write timestamp. At the same time, the merged record is saved to the persistence module and deleted from the cache.

5. The database system supporting parallel updates according to claim 1, characterized in that: It also includes a dependency graph optimization module, used for: When a transaction that reads a data item generates a Reader object r, a corresponding placeholder transaction is generated. The placeholder transaction contains a Reader / Writer pair, where the Writer is a regular Writer, and the timestamp of the placeholder transaction is set to the version number of the last Writer object less than r + 1. A dependency graph is generated for the Reader of the placeholder transaction. Subsequent transactions that read the same data item and whose timestamp is greater than the Reader version number in the placeholder transaction only need to depend on the Writer of the placeholder transaction. When the Reader object corresponding to the placeholder transaction finishes execution, the value of the Writer corresponding to the placeholder transaction is replaced with the read value, and the placeholder transaction is committed.

6. The database system supporting parallel updates according to claim 1, characterized in that: It also includes constraint update and condition update modules; The constraint update and condition update module is used for: When updating data items with constraints, if the transaction writing the data item is an incremental update operation, the value range of the data item after the current incremental update operation is calculated, and it is determined whether there is an intersection with the constraint range. If there is no intersection, it is proven that the incremental update operation is illegal and the transaction needs to be rolled back; otherwise, it is further determined whether it is a subset of the constraint range. If it is a subset of the constraint range, it is proven that the operation must be legal, and the transaction continues to be executed to complete the incremental update of the data item with constraints; otherwise, the operation will be transformed into a read-modify-write operation and no longer be updated in parallel. When updating data items with conditions, if the transaction writing the data item is an incremental update operation, the domain range of the data item before the current incremental update operation is calculated, and it is determined whether there is an intersection with the update conditions. If there is no intersection, it is proven that the incremental update operation is invalid and the transaction needs to be rolled back; otherwise, it is further determined whether it is a subset of the update conditions. If it is, it is proven that the operation is definitely valid and the transaction continues to be executed; otherwise, the operation will be transformed into a read-modify-write operation and no longer perform parallel updates.

7. The database system supporting parallel updates according to claim 6, characterized in that: The methods for defining and updating the domain and range include: Assuming the initial committed value of a data item is v, then the value range of the data item is [v, v], denoted as range. key The domain is also [v, v], denoted as domain. key ; For each incremental update operation DeltaWriterw, where wf represents the function corresponding to the w update operation, then: range key =w.f.map(domain key ) domain′ key =domain key ∪range key For updates to the value range and domain range, non-commutative update operations are performed in order of transaction timestamp size; for update operations that are independent of order, out-of-order updates can be performed.

8. The database system supporting parallel updates according to claim 1, characterized in that: It also includes an index update module; The index update module is used for: When a unique index exists on the column of the updated data item, the update operation needs to be transformed into a read-modify-write operation and no longer performed in parallel. When a non-unique index exists on the column being updated in the data item being updated, parallel updates are still performed, using the following method: When updating data items, non-unique index items are created. Non-unique index items include: The primary key column is encoded as: Index Name - Index Value - Primary Key Value of the Indexed Data Item - Transaction Version Number, where the index value is the value of the index column and the transaction version number is the version number of the transaction that created the index; non-primary key columns record the metadata information of the index items and are called index metadata columns. For incremental record data items, the index value is set to an ambiguous value, and the value in the index metadata column is the value range of that data item; for index items of ordinary records, the index value is a definite value, and the index metadata column is empty. For a query using an index, indexed_column∈C, let the timestamp of the query transaction be TS. q First, update the query condition C∪{undefined} maximum gap read timestamp; then scan index items whose index value is not undefined, and add the index items that satisfy the query condition and transaction visibility to the result set R; then scan index items whose index value is undefined, first determine whether the visibility is satisfied, if not, skip the index item; otherwise, continue to determine the relationship between the value range of the index item and the query condition set; if the value range of the index item and the query condition have no intersection, skip the data item; otherwise, if the value range of the index item is a subset of the query condition set, add the index item to the result set R; otherwise, register a Reader object for the data item corresponding to the primary key value of the indexed data item, set the timestamp of the Reader object to the version number of the index item, execute the read process for the Reader, after the execution is completed, the index value of the index item is the value read by the Reader, compare it with the query condition, if it is not satisfied, skip; otherwise, add it to the result set R.

9. The database system supporting parallel updates according to claim 1, 6, or 8, characterized in that: The process of transforming an update operation into a read-modify-write operation is as follows: First, register a Reader and a placeholder Writer using the current transaction timestamp as the version number. Execute the read process to obtain the value of the current data item, apply the update operation, and then replace the value in the placeholder Writer with the updated value and execute the write process.

10. The database system supporting parallel updates according to claim 8, characterized in that: The index update module is also used to clean up ambiguous indexes, including: A large number of ambiguous indexes in the system can significantly slow down query performance. It is necessary to clean up ambiguous indexes regularly. The cleanup methods are as follows: Scan for ambiguous indexes. For each ambiguous index, register a Reader object for the data item corresponding to the primary key value of the indexed data item, and wait for all Writer objects that the Reader object depends on to commit. At this point, the value of the index item becomes a definite value. Create a new index item, and use the definite value for the index value. After creation, delete the corresponding ambiguous index item.

11. A method for using a database system supporting parallel updates as described in any one of claims 1 to 10, characterized in that, include: Read transaction concurrency control methods and write transaction concurrency control methods; The write transaction concurrency control method includes: At the start of each transaction, a timestamp is obtained from the timestamp service and used as the version number for subsequent records of each transaction; If the transaction's timestamp is less than the maximum read timestamp, rollback is performed; otherwise, write operations are performed separately for regular write operations and incremental update operations. For a regular write operation on a data item, further determine whether the timestamp of the transaction that writes the data item is less than the maximum write timestamp of the data item cached in memory. If it is less, roll back the transaction; otherwise, update the maximum write timestamp and the maximum regular write timestamp of the data item, and generate a regular log and a regular record. For incremental update operations on data items, further determine whether the timestamp of the transaction that writes the data item is less than the maximum normal write timestamp of the data item cached in memory. If it is less, roll back the transaction; otherwise, update the maximum write timestamp of the data item and generate incremental logs and incremental records. The read transaction concurrency control method includes: At the start of each transaction, a timestamp is obtained from the timestamp service and used as the version number for subsequent records of each transaction; For each data item read in a transaction, a Reader object is generated separately; For each data item that a transaction needs to read, retrieve the last ordinary record in the cache that is less than the corresponding transaction timestamp and all subsequent incremental records; if the cache is not hit, load the last ordinary record that is less than the transaction timestamp and all subsequent records from the persistence module into the cache. Based on the dependencies between transactions that read data items and transactions that write data items, a dependency graph of a Reader object is maintained based on the last normal record and all subsequent incremental records. The Reader object is executed when the last ordinary record and all subsequent incremental records in the dependency graph are committed data. The Reader object merges the values ​​of all the Writer objects it depends on from the dependency graph to obtain the value of the data item to be read, and atomically updates the maximum read timestamp.

Images