A deterministic transaction concurrency control method based on GPU acceleration

Abstract

A deterministic transaction concurrency control method based on GPU acceleration belongs to the field of database transaction processing. The invention migrates database transactions to the GPU for processing, decomposes the transactions into single read operations or write operations, and then executes the transactions in the GPU in the order of first processing the read operations in parallel and then processing the write operations in parallel. After executing the transactions, intra-transaction conflict detection and inter-transaction conflict detection are performed on the transactions, and the transactions are committed after completing the two detections. The method realizes a lock-free full concurrency strategy within and between transactions, and only needs to determine the read-write set of the transaction before the entire transaction is migrated to the GPU for execution, and does not need to construct transaction execution dependency data with high computing resource consumption such as dependency graphs, and only needs to detect conflicts therein after the transaction execution is completed.

Description

Technical Field

[0001] The present invention belongs to the field of database transaction processing, and in particular relates to a deterministic transaction concurrency control method based on GPU acceleration. Background Art

[0002] Transaction processing is a classic problem in databases. A database transaction refers to a series of operations performed as a single logical unit. Let's use an online shopping transaction to illustrate the meaning of a transaction. For example, a complete online shopping transaction process includes at least the following operations on the database: 1) Update the inventory information of the goods selected by the customer, 2) Save the customer's payment information, 3) Generate an order and save it in the database, 4) Update user-related information. Under normal circumstances, the above operations are carried out smoothly, and an online shopping will eventually be completed, and all database information related to the transaction will be successfully updated. However, if a problem occurs in a certain link, the entire transaction will fail. Once it fails, all information in the database must remain unchanged in the state before submission, and database transactions are precisely the technology used to ensure the stability and predictability of transactions in this case.

[0003] BOHM is a single-node deterministic database that runs transactions in two steps. In the first step, the primary key placeholders of each input transaction's write set are inserted into the multi-version storage layer along with the transaction ID. In the second step, the transaction must read a specific version for each key in its read set - the one with the largest TID, up to the transaction's TID - to enforce determinism. When all keys in the transaction's read set are ready, the transaction can execute and update the previously inserted placeholders. To avoid contention when inserting placeholders into the storage layer, each worker thread in BOHM scans all input transactions to find the primary keys to be updated in its own partition.

[0004] PWV is a single-node deterministic database that first decomposes each input transaction into a set of fragments (i.e., subtransactions) such that each fragment reads or writes only one partition. Next, it builds a dependency graph where an edge represents a conflict between fragments within a partition, and uses it to schedule the execution of each fragment to satisfy data dependency and commit dependency requirements.

[0005] Aria is a distributed deterministic OLTP database that executes transactions in two steps. In the first step, each transaction reads from the current snapshot of the database and saves the writes in a local write set. It adopts a single version approach, even though it buffers write transactions until the end of the batch. Once all transactions in the batch have finished executing on the replica, it enters the commit phase. The commit phase on the replica also runs on multiple worker threads, each of which executes independently. During the commit phase, the threads abort transactions that perform operations that conflict with earlier transactions, that is, transactions with smaller TIDs. For example, if a transaction reads a record modified by an earlier transaction, it aborts. Aborted transactions are automatically scheduled for execution at the beginning of the next batch unless the transaction is explicitly aborted. If the transaction does not conflict with previous transactions, the system applies its changes to the database.

[0006] When doing transaction processing on a single CPU, the concurrency effect within and between transactions is not very good.

[0007] The drawback of BOHM is that each worker thread scans all input transactions to find the primary key to be updated in its own partition, consuming a lot of computing resources in the preprocessing stage.

[0008] The drawback of PWV is that it commits each transaction's writes in finer-grained fragments rather than the entire transaction, and although subsequent transactions spend less time waiting to see the writes of earlier transactions, the cost is a more expensive fine-grained dependency graph analysis.

[0009] The drawback of Calvin is that in a multi-core machine, a single-threaded lock manager can hardly saturate system resources because many worker threads are idle waiting for locks. To solve this problem, the database can be divided into multiple partitions per machine, but this introduces additional overhead by introducing more multi-partition transactions, which incur overhead due to redundant execution.

[0010] The problem with Aria is that due to the limitation of the number of CPU threads, the number of transactions in a single batch is limited, and the strategy of executing each operation in a transaction serially is adopted. Although it does not need to know the situation before the transaction is executed in advance, its transaction concurrency on a single machine is not good. Summary of the invention

[0011] In view of the shortcomings of the prior art, the present invention designs a deterministic transaction concurrency control method based on GPU acceleration;

[0012] A method for controlling concurrent transactions based on GPU acceleration includes the following steps:

[0013] Step 1: The CPU receives the transaction, parses the transaction request and accumulates the transaction. After receiving the transaction request, the CPU parses the transaction into a read-write set and assigns each transaction an independent transaction ID, namely, TID. Each read and write operation in the transaction is assigned an independent operation ID, namely, OID. At the same time, a snapshot of the current database state is obtained from the database. The snapshot is the view of the database at a certain point in time, preparing for the subsequent transaction execution. The transaction adopts a series of read and write operations randomly generated by the computer.

[0014] Step 2: Four threads are used as a collaborative group to process a transaction. The transaction is executed by first executing the read operation and then the write operation. The transaction is regarded as a set of read and write operations, and various operations are executed concurrently to achieve better results. At the same time, transaction conflicts are detected for each operation in the transaction to ensure the atomicity of the transaction. The thread is a computing resource in the GPU. When the GPU is started for computing, the number of threads is pre-allocated for its call.

[0015] Step 2.1: Read operation phase: Each thread in the collaboration group obtains the thread ID of the current thread in the collaboration group through the collaboration group function thread_rank, and uses this as the order for processing read operations; before the read operation is performed, the data in the database to be operated is first copied to the local read-write set to avoid data inconsistency caused by directly operating the data in the snapshot; after the read operation is completed, the next operation ID to be operated is set to the current operation ID + 4, and the next operation is performed; until the read operations in all transactions are completed;

[0016] Step 2.2: Write operation phase: Each thread in the collaboration group obtains the thread ID of the current thread in the collaboration group through the collaboration group function thread_rank, and uses this as the order for processing write operations; before the write operation, the data to be operated is saved in the local read-write set to avoid data inconsistency caused by directly operating the data in the snapshot; then the data is written, and the result of the write operation is also saved in the local read-write set; after the read operation is completed, the next operation ID to be operated is set to the current operation ID + 4, and the next operation is performed; until the write operations in all transactions are completed;

[0017] Step 2.3: Intra-transaction conflict detection phase: Each thread in the collaboration group obtains the thread ID of the current thread in the collaboration group through the collaboration group function thread_rank, and uses this as the order for conflict detection; since the data processed by each thread is different and does not involve large-scale simultaneous write operations on a certain data, it has a good concurrency effect; during the detection process, if there is a conflict, modify the INSIDE_STATE in the read-write set to set it to an abnormal state within the transaction, and perform serial processing in the next batch of processing; in the transaction part, the standard for detecting conflicts is that there are no RAW, WAR, WAW conflicts in the operations within the transaction; if the detection is completed and there is no such conflict, go to step 3;

[0018] Step 3: Perform conflict detection between transactions, that is, conflict detection with other transaction operations before the transaction reads and writes the result. All threads in the GPU are grouped into four threads, and then every four threads process the conflict detection of one transaction.

[0019] Step 3.1: Each thread in the collaboration group obtains the thread ID of the current thread in the collaboration group through the collaboration group function thread_rank, and uses this as the order of conflict detection; if the INSIDE_STATE of the transaction is an abnormal state, the external conflict detection is skipped;

[0020] Step 3.2: Each thread of the collaboration group calls RAW, WAR, and WAW detection functions respectively to detect whether there is a corresponding conflict between the read or write operation in the transaction being detected and the read or write operation of other transactions; other transactions only refer to transactions with smaller transaction IDs than the transaction being detected; if there is a conflict, the flag of the corresponding conflict in this transaction is set to "conflict exists" for use in the subsequent commit stage; the transaction does not have WAW, WAR, and RAW; if the transaction does not have WAW, and one of WAR and RAW exists, it can be considered that the transaction does not conflict. This is a logical reordering of transactions, which greatly reduces transaction aborts caused by transaction conflicts, and the resulting repeated consumption of computing resources;

[0021] Step 4: Perform a pre-submission review based on the intra-transaction conflict detection results in step 2 and the inter-transaction conflict detection results in step 3;

[0022] Step 4.1: If there is no internal or external conflict in the transaction after pre-commit detection, the transaction is committed. When committing, the data in the local data center of the transaction is written back to the snapshot. The data in the CPU memory is persistently recorded when the transaction is returned to the CPU.

[0023] Step 4.2: If there is an internal conflict, there are two cases:

[0024] Case 1: If there is no external conflict, after the commit phase, the transactions are serialized within the transaction and executed in parallel between transactions. After the execution, conflict detection and commit operations are performed to avoid consuming too much bandwidth and memory resources.

[0025] Case 2: If there is an external conflict, the CPU will prioritize the serial execution of the transaction when receiving the next batch of transactions;

[0026] Step 4.3: If there is an external conflict, the transaction is returned to the CPU and transmitted to the GPU together with the next batch of transactions. When the next batch of transactions obtains the transaction ID, the original transaction ID remains unchanged, that is, the small transaction ID is saved and is executed relatively preferentially in conflict detection.

[0027] Beneficial technical effects of the present invention:

[0028] The present invention proposes a deterministic transaction concurrency processing algorithm that utilizes the powerful computing power of GPU to accelerate transaction processing speed. In order to increase the concurrency between transactions, increase the concurrency within transactions and ensure the atomicity of transactions, large-scale concurrency of transactions and reduce resource consumption caused by conflicts are achieved by controlling branch divergence and utilizing the characteristics of cooperative groups that can achieve finer-grained thread synchronization.

[0029] The present invention transplants the transaction processing part into the GPU, controls branch divergence to maximize the use of GPU resources, adopts cooperative group group synchronization to make GPU thread scheduling more flexible, and can configure the number of cooperative group threads according to the operation conditions of specific transactions. At the same time, it is to avoid multiple transmissions consuming PCI-E bandwidth resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Schematic diagram of a deterministic transaction concurrency control algorithm according to an embodiment of the present invention;

[0031] Figure 2 A test effect diagram of a deterministic transaction concurrency control method based on GPU acceleration in an embodiment of the present invention;

[0032] Figure 3 The embodiment of the present invention is a logical architecture between CPU and GPU based on a deterministic transaction concurrency control method accelerated by GPU;

[0033] Figure 4 An embodiment of the present invention provides a deterministic transaction concurrency control method based on GPU acceleration, which is a manifestation of the CPU and GPU on the hardware system; wherein TID represents the transaction ID. DETAILED DESCRIPTION

[0034] The present invention will be further described below in conjunction with the accompanying drawings and embodiments;

[0035] Structurally, GPUs usually include general computing units, controllers, and registers. Due to its highly parallel structure, GPUs are more efficient than CPUs in processing graphics data and complex algorithms. GPUs have more ALUs, or logical operation units, than CPUs, allowing them to process multiple data in parallel at the same time.

[0036] Nowadays, GPU has been developing towards general-purpose graphics processor, namely GPGPU, which means that GPGPU will take on many computing tasks that were originally handled by CPU. When faced with single instruction multiple data (SIMD) and the amount of data processing is far greater than the needs of data scheduling and transmission, the general-purpose graphics processor greatly surpasses the traditional central processing unit in performance.

[0037] A deterministic transaction concurrency control method based on GPU acceleration, as shown in the attached Figure 1 As shown, the specific steps include:

[0038] Step 1: The CPU receives the transaction, parses the transaction request and accumulates the transaction. After receiving the transaction request, the CPU parses the transaction into a read-write set and assigns each transaction an independent transaction ID, namely, TID. Each read and write operation in the transaction is assigned an independent operation ID, namely, OID. At the same time, a snapshot of the current database state is obtained from the database. The snapshot is the view of the database at a certain point in time, preparing for the subsequent transaction execution. The transaction adopts a series of read and write operations randomly generated by the computer.

[0039] Step 2: Four threads are used as a collaborative group to process a transaction. The transaction is executed by first executing the read operation and then the write operation. The transaction is regarded as a set of read and write operations, and various operations are executed concurrently to achieve better results. At the same time, transaction conflicts are detected for each operation in the transaction to ensure the atomicity of the transaction. The thread is a computing resource in the GPU. When the GPU is started for computing, the number of threads is pre-allocated for its call.

[0040] Step 2.1: Read operation phase: Each thread in the collaboration group obtains the thread ID of the current thread in the collaboration group through the collaboration group function thread_rank, and uses this as the order for processing read operations; before the read operation is performed, the data in the database to be operated is first copied to the local read-write set to avoid data inconsistency caused by directly operating the data in the snapshot; after the read operation is completed, the next operation ID to be operated is set to the current operation ID + 4, and the next operation is performed; until the read operations in all transactions are completed;

[0041] Step 2.2: Write operation phase: Each thread in the collaboration group obtains the thread ID of the current thread in the collaboration group through the collaboration group function thread_rank, and uses this as the order for processing write operations; before the write operation, the data to be operated is saved in the local read-write set to avoid data inconsistency caused by directly operating the data in the snapshot; then the data is written, and the result of the write operation is also saved in the local read-write set; after the read operation is completed, the next operation ID to be operated is set to the current operation ID + 4, and the next operation is performed; until the write operations in all transactions are completed;

[0042] Step 2.3: Intra-transaction conflict detection phase: Each thread in the collaboration group obtains the thread ID of the current thread in the collaboration group through the collaboration group function thread_rank, and uses this as the order for conflict detection; since the data processed by each thread is different and does not involve large-scale simultaneous write operations on a certain data, it has a good concurrency effect; during the detection process, if there is a conflict, modify the INSIDE_STATE in the read-write set to set it to an abnormal state within the transaction, and perform serial processing in the next batch of processing; in the transaction part, the standard for detecting conflicts is that there are no RAW, WAR, WAW conflicts in the operations within the transaction; if the detection is completed and there is no such conflict, go to step 3;

[0043] Step 3: Perform conflict detection between transactions, that is, conflict detection with other transaction operations before the transaction reads and writes the result. All threads in the GPU are grouped into four threads, and then every four threads process the conflict detection of one transaction.

[0044] Step 3.1: Each thread in the collaboration group obtains the thread ID of the current thread in the collaboration group through the collaboration group function thread_rank, and uses this as the order of conflict detection; if the INSIDE_STATE of the transaction is an abnormal state, the external conflict detection is skipped;

[0045] Step 3.2: Each thread of the collaboration group calls RAW, WAR, and WAW detection functions respectively to detect whether there is a corresponding conflict between the read or write operation in the transaction being detected and the read or write operation of other transactions; other transactions only refer to transactions with smaller transaction IDs than the transaction being detected; if there is a conflict, the flag of the corresponding conflict in this transaction is set to "conflict exists" for use in the subsequent commit stage; the transaction does not have WAW, WAR, and RAW; if the transaction does not have WAW, and one of WAR and RAW exists, it can be considered that the transaction does not conflict. This is a logical reordering of transactions, which greatly reduces transaction aborts caused by transaction conflicts, and the resulting repeated consumption of computing resources;

[0046] Step 4: Perform a pre-submission review based on the intra-transaction conflict detection results in step 2 and the inter-transaction conflict detection results in step 3; Figure 3 It is the logical architecture between CPU and GPU; Figure 4 It is the representation between CPU and GPU on the hardware system; TID stands for transaction ID.

[0047] Step 4.1: If there is no internal or external conflict in the transaction after pre-commit detection, the transaction is committed. When committing, the data in the local data center of the transaction is written back to the snapshot. The data in the CPU memory is persistently recorded when the transaction is returned to the CPU.

[0048] Step 4.2: If there is an internal conflict, there are two cases:

[0049] Case 1: If there is no external conflict, after the commit phase, the transactions are serialized within the transaction and executed in parallel between transactions. After the execution, conflict detection and commit operations are performed to avoid consuming too much bandwidth and memory resources.

[0050] Case 2: If there is an external conflict, the CPU will prioritize the serial execution of the transaction when receiving the next batch of transactions;

[0051] Step 4.3: If there is an external conflict, the transaction is returned to the CPU and transmitted to the GPU together with the next batch of transactions. When the next batch of transactions obtains the transaction ID, the original transaction ID remains unchanged, that is, the small transaction ID is saved and is executed relatively preferentially in conflict detection.

[0052] Figure 2 This is the test result of the present invention, where the horizontal axis is the number of warehouses in the TPC-C benchmark test, and the vertical axis is the "number of transactions per second" commonly used in transaction testing, i.e. Tps. In the test, it can be seen that although the system execution speed decreases with the increase in the number of warehouses, this may be related to the increase in memory access time caused by the expansion of the database scale. When the number of warehouses is less than 20, the overall transaction execution speed can reach more than 200K, which is more than 200 times faster than MySQL.

[0053] The deterministic transaction concurrency control method described in the present invention is used to control two transactions in tpcc that are not range scanned, New-Order and Payment transactions, and the other three transactions involve range search, so they are not currently supported. Due to the memory size limitation of the GPU, the number of warehouses cannot reach the hundreds or thousands supported by many distributed database systems. tpcc tests were conducted in 2, 4, 6, 8, 10, 20, 40, 60, 80, and 100 warehouses. The results show that the number of transactions per batch and the number of threads per batch have a great influence on the transaction execution speed, and the number of warehouses has a great influence on transaction conflicts and re-execution operations. From the experimental results, the use of GPU to execute transactions is very effective, basically achieving the design purpose of improving concurrency between transactions, concurrency within transactions, and ensuring transaction atomicity.

Claims

1. A deterministic transaction concurrency control method based on GPU acceleration, characterized in that: The specific steps include: Step 1: The CPU receives the transaction, parses the transaction request and accumulates the transaction. After receiving the transaction request, the CPU parses the transaction into a read-write set and assigns each transaction an independent transaction ID, namely, TID. It also assigns an independent operation ID, namely, OID, to each read and write operation within a transaction. At the same time, it obtains a snapshot of the current database state from the database. The snapshot is the view of the database at a certain point in time, preparing for subsequent transaction execution. Step 2: Take four threads as a collaborative group to process a transaction. The overall approach is to execute the transaction by first executing the read operation and then the write operation. That is, the transaction is regarded as a set of read and write operations, and each operation is executed concurrently to achieve better results. At the same time, transaction conflict detection is performed on each operation in the transaction to ensure the atomicity of the transaction. If the detection is completed and there is no conflict, go to step 3; Step 3: Perform conflict detection between transactions, that is, conflict detection with other transaction operations before the transaction reads and writes the result. All threads in the GPU are grouped into four threads, and then every four threads process the conflict detection of one transaction. Step 4: Perform a pre-commit review based on the intra-transaction conflict detection results in step 2 and the inter-transaction conflict detection results in step 3.

2. According to the method of claim 1, wherein the method comprises: The transaction described in step 1 is performed by a computer randomly generating a series of read and write operations.

3. According to the method of claim 1, wherein: The thread described in step 2 is a computing resource in the GPU. When the GPU is started for computing, the number of threads is pre-allocated for calling.

4. The method for controlling concurrent transactions based on GPU acceleration according to claim 1, characterized in that: Step 2 is as follows: Step 2.1: Read operation phase: Each thread in the collaboration group obtains the thread ID of the current thread in the collaboration group through the collaboration group function thread_rank, and uses this as the order for processing read operations; before the read operation is performed, the data in the database to be operated is first copied to the local read-write set to avoid data inconsistency caused by directly operating the data in the snapshot; After the read operation is completed, the next operation ID is set to the current operation ID + 4, and the next operation is performed; until the read operations in all transactions are completed; Step 2.2: Write operation phase: Each thread in the collaboration group obtains the thread ID of the current thread in the collaboration group through the collaboration group function thread_rank, and uses this as the order for processing write operations; before the write operation, the data to be operated is saved in the local read-write set to avoid data inconsistency caused by directly operating the data in the snapshot; Then the data is written, and the result of the writing operation is also saved in the local read-write set; After the read operation is completed, the next operation ID is set to the current operation ID + 4, and the next operation is performed; until the write operations in all transactions are completed; Step 2.3: Intra-transaction conflict detection phase: Each thread in the collaboration group obtains the thread ID of the current thread in the collaboration group through the collaboration group function thread_rank, and uses this as the order of conflict detection; since each thread processes different data and does not involve large-scale simultaneous write operations on a certain data, it has a good concurrency effect; During the detection process, if there is a conflict, modify INSIDE_STATE in the read-write set to set it to an abnormal state within the transaction, and perform serial processing in the next batch processing; In the transaction part, the conflict detection standard is that there are no RAW, WAR, or WAW conflicts among the operations in the transaction; If the detection is completed and there is no conflict, go to step 3.

5. The method for controlling concurrent transactions based on GPU acceleration according to claim 1, characterized in that: Step 3 is as follows: Step 3.1: Each thread in the collaboration group obtains the thread ID of the current thread in the collaboration group through the collaboration group function thread_rank, and uses this as the order of conflict detection; if the INSIDE_STATE of the transaction is an abnormal state, the external conflict detection is skipped; Step 3.2: Each thread of the collaboration group calls the RAW, WAR, and WAW detection functions respectively to detect whether there is a corresponding conflict between the read or write operation in the transaction being detected and the read or write operation of other transactions; other transactions only refer to transactions with a smaller transaction ID than the transaction being detected; if there is a conflict, the flag corresponding to the conflict in this transaction is set to "conflict exists" for use in the subsequent commit stage; there is no WAW in the transaction, and there is no WAR and RAW; there is no WAW in the transaction, and one of WAR and RAW exists, then it can be considered that there is no conflict in the transaction. This is a logical reordering of transactions, which greatly reduces transaction aborts caused by transaction conflicts, and the resulting repeated consumption of computing resources.

6. The method for controlling concurrent transactions based on GPU acceleration according to claim 1, characterized in that: Step 4 is as follows: Step 4.1: If there is no internal or external conflict in the transaction after the pre-commit check, commit the transaction. When committing, write the local data in the transaction back to the snapshot. The data in the CPU memory is persistently recorded when the transaction is returned to the CPU; Step 4.2: If there is an internal conflict, it is handled in two cases: Step 4.3: If there is an external conflict, the transaction is returned to the CPU and transmitted to the GPU together with the next batch of transactions. When the next batch of transactions obtains the transaction ID, the original transaction ID remains unchanged, that is, the small transaction ID is saved and is executed relatively preferentially in conflict detection.

7. The method for controlling concurrent transactions based on GPU acceleration according to claim 6, characterized in that: The two situations described in step 4.2 are specifically: Case 1: If there is no external conflict, after the commit phase, the transactions are serialized within the transaction and executed in parallel between transactions. After the execution, conflict detection and commit operations are performed to avoid consuming too much bandwidth and memory resources. Case 2: If there is an external conflict, the CPU will prioritize executing the transaction serially when receiving the next batch of transactions.

Images