An optimization method for BatchGroup serialization calls based on CockroachDB

Abstract

The present invention provides an optimization method for BatchGroup serialization call based on CockroachDB, which belongs to the database field. The present invention abandons the use of BatchGroup serialization execution mechanism in CockroachDB, but introduces Sequence, and the Go layer pre-allocates a sequence for each BatchGroup, which is brought into RocksDB and stored. Sequence is incrementally increased, and the current Sequence is also brought when reading. The current coroutine can only see data less than or equal to the current Sequence, thereby decoupling the direct sequential dependencies of each BatchGroup, realizing parallel writing, and thus achieving performance improvement.

Description

Technical Field

[0001] The present invention relates to the field of databases, and in particular to an optimization method for BatchGroup serialization calls based on CockroachDB. Background Art

[0002] CockroachDB is a distributed database that is horizontally scalable and supports transactional ACID features. It mainly consists of the SQL layer, transaction layer, distribution layer, replica layer, and storage layer. When the replica layer calls the write interface of the storage layer, the data is stored in the batch. Multiple batches are sequentially placed in a FIFO queue. The batch at the head of the queue is the leader and forms its own batch group. Because there is a sequential dependency between batch groups, the group calls the storage layer write interface in a serial manner. The storage layer maintains a globally increasing sequence. When inserting user data, it is bound to the current sequence and stored in memory or on disk. The earlier the data is inserted, the smaller the sequence. When reading data, only data that is less than or equal to the current sequence of the storage layer is visible. We can see that because there is a sequential dependency between batch groups, batch groups must be executed serially, which will become a performance bottleneck of the database. Summary of the invention

[0003] In order to solve the problem of BatchGroup serialization calling of CockroachDB, the present invention provides an optimization method for BatchGroup serialization calling based on CockroachDB, which pre-allocates a globally increasing Sequence based on the Go layer to solve the sequential dependency problem between BatchGroups, thereby realizing BatchGroup concurrent writing to the storage layer and improving the writing performance of CockroachDB.

[0004] The technical solution of the present invention is:

[0005] An optimization method based on the serialized call of BatchGroup of CockroachDB. By moving the Sequence of the storage layer to the Go layer, a globally incremented Sequence number is pre-assigned to each BatchGroup when writing data, and the current Sequence is brought to the storage layer when reading data. This enables concurrent writing of BatchGroup and improves database writing performance.

[0006] Furthermore, when BatchGroup calls CGO to perform write operations, it is no longer done in a serial manner. Instead, the Go layer maintains the global variable Sequence, allocates Sequence segments to each BatchGroup in sequence, releases their sequential coupling relationship, and implements concurrent calls of BatchGroup.

[0007] The implementation includes the following:

[0008] 1) Each writing coroutine writes data into a batch and adds it to the pending queue after commit. The first batch in the queue is used as the leader to determine a batch group. The other batch data in the group are merged into the leader and submitted by the leader.

[0009] 2) Maintain a global increasing Sequence, which represents the order in which data is written.

[0010] 3) Before the Leader makes a CGO call, it calculates the number of write operations in the Batch, and then pre-allocates the sequence number segment [Sequence+1, Sequence+count] for it, and updates Sequence to Sequence+count. The sequence number represents the relative order of data writing.

[0011] 4) When reading, NewIter returns a storage layer iterator. If the sequence field of IterOption is empty, it means that it is not a snapshot query, and the current Sequence is used for assignment, which means that only data less than or equal to Sequence can be seen.

[0012] Going further,

[0013] Batches are constantly added to the pending queue. Because the size of each BatchGroup is limited, when there is a backlog of Batch, multiple BatchGroups will appear, separated by NULL. Multiple BatchGroups concurrently request to write to the storage layer, so a semaphore Semophere is maintained to control the number of concurrent BatchGroups. Call the acquire() method of Semophere. If the concurrency threshold is not reached, it returns. Otherwise, it blocks until one of the BatchGroups is executed and releases() the resource.

[0014] The Go layer maintains a globally increasing Sequence, which represents the order in which data is written. When the database is started, the Sequence is loaded into memory from the manifest file, and when the database is running, the Sequence is still written to the storage layer.

[0015] When calling the storage layer write interface to write data, you need to pre-allocate a continuous Sequence segment for the BatchGroup.

[0016] 1) Before calling CGO, the leader of BatchGroup calculates the number of write operations in the current Batch, and then obtains the [Sequence+1, Sequence+count] segment to write data in this group. Because multiple BatchGroups obtain Sequence segments concurrently, a while loop and CAS operation are required. After successful pre-allocation, the leader will exit. Otherwise, the leader will continue to reallocate until successful. At the same time, it should be ensured that the BatchGroup at the front of the pending queue should be pre-allocated to a smaller Sequence segment to ensure the sequential semantics of the data.

[0017] 2) BatchGroup concurrently calls CGO. Before calling, BatchGroup needs to be pushed into the queue. After the call returns, if the previous BatchGroup has been dequeued, it will be directly dequeued and returned; otherwise, it needs to wait for the previous BatchGroup to be completed before returning. This ensures that the semantics remain unchanged during concurrent reading and writing.

[0018] The read interface of the storage layer is to generate an iterator of the storage layer through NewIter. Previously, Sequence was maintained in the storage layer. After Sequence is moved to the Go layer, it is necessary to assign a value to the sequence field of IterOption and pass it to the storage layer every time it is read. The read logic of the storage layer remains unchanged, and Sequence is still used as the basis of its MVCC.

[0019] When writing, the storage layer needs to receive the sequence from the upper layer and write it into the memory or disk together with the data. At the same time, because the Go layer has adopted the Batch Group strategy, the C layer does not form a BatchGroup again.

[0020] The beneficial effects of the present invention are

[0021] The database write performance bottleneck caused by the serialized call of CGO by BatchGroup in CockroachDB has been optimized. The serialized processing mode is no longer used, but the concurrent submission of BatchGroup is implemented based on the pre-allocated Sequence segment mechanism. At the same time, the concurrency of the concurrent execution of the coroutine is controlled by setting the capacity of the semaphore Semophere. This transformation reduces the waiting time of Batch in the pending queue in a large number of write scenarios, which can improve the write performance of the storage engine. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a diagram of CockroachDB's BatchGroup serial execution;

[0023] Figure 2 It is a schematic diagram of the concurrent execution of BatchGroup in the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0025] Parallel execution is the current trend in service operation, and parallel processing greatly improves performance compared to serial processing. CockroachDB's BatchGroup serialization processing is because the sequence number is maintained in the storage layer. The data written first has a smaller sequence number, and only data that is less than or equal to the current sequence is visible when reading. Therefore, if we maintain the sequence of the storage layer in the Go layer, pre-allocate a sequence for each BatchGroup each time it is written, and pass the sequence to the storage layer when reading, then BatchGroup can implement concurrent calls to the storage layer write interface, while also ensuring the order of data, thereby achieving performance improvement.

[0026] The present invention provides an optimization method for serialized calls of BatchGroup based on CockroachDB. This method moves the Sequence of the storage layer to the Go layer, pre-assigns a globally incremented Sequence number to each BatchGroup when writing data, and brings the current Sequence to the storage layer when reading data. This enables concurrent writing of BatchGroup and improves database writing performance. The specific implementation process is as follows:

[0027] 1. In the database system, there will be continuous write requests. Currently, there are two types of calls to the storage layer write interface by the replica layer: one is to directly call the Put interface of the engine, which is mostly called by the background coroutine to store metadata; the other is to encapsulate one or more operations into a Batch, and multiple Batches form a BatchGroup, which is uniformly written by the Leader, which is mostly used to store user data.

[0028] 2. All write requests that are not directly called by Batch are made to go through Batch logic. This reduces CGO calls, which can improve performance. At the same time, it also converges the write interface and facilitates the pre-allocation of Sequence.

[0029] 3. Multiple write operations of the user are stored in a Batch, and added to a global pending queue after Commit. The first Batch in the queue is the Leader, and it forms a BatchGroup with zero or more Batches behind it. Because a Group has a byte size limit of 1M, multiple BatchGroups can exist in a pending queue at the same time, and each BatchGroup is separated by a nil value.

[0030] 4. Maintain a semaphore Semophere to control the concurrency of BatchGroup. After becoming the leader of a BatchGroup, Batch first calls Semophere's acquire() method. If successful, it means that the number of concurrent executions of the current BatchGroup has not reached the limit threshold; if it fails, it is blocked, indicating that the concurrency is limited. After waiting for the BatchGroup being executed to end, call Semophere's release() method to release resources and wake up the blocked coroutine to continue execution.

[0031] 5. Maintain a global incrementing Sequence variable, which represents the sequence number corresponding to the data last written to the storage layer. When the BatchGroup commits, the Leader will merge the data of other batches in the group into its own batch. Before calling CGO to actually store data, the Leader will calculate the number of write operations count in the current Batch. The BatchGroup uses CAS operations to determine that its pre-allocated Sequence space is [Sequence+1, Sequence+count], and updates the Sequence value to Sequence+count.

[0032] 6. For each read operation, an iterator is generated in the storage layer to read data. The iterator uses Sequence to filter out all data greater than the current Sequence value, thereby ensuring that data inserted after the operation will not be read. Previously, Sequence was maintained by the storage layer. Now, Sequence values ​​are brought to the storage layer by the Go layer through the IterOption parameter, and the reading logic remains unchanged.

[0033] 7. When multiple leaders execute concurrently, they first confirm their own BatchGroup through an exclusive lock. Before CGO is called, multiple coroutines concurrently apply for Sequence segments. Concurrent execution needs to ensure three points: 1) Ensure the uniqueness of the Sequence segment. 2) Ensure that the order of the Sequence segment is the same as the order of the BatchGroup, so as to ensure that concurrent execution does not change the semantics of the serial execution statement. 3) Ensure that the response order is the same as the order of the BatchGroup after the concurrent execution ends. The uniqueness of the Sequence segment is guaranteed through the while loop and CAS command. After the pre-allocation is successful, the global Sequence is updated in time. If it fails, it is recalculated until it succeeds. To ensure that the Sequence segment and the response order are consistent with the BatchGroup, it is necessary to maintain a FIFO queue queue, which stores the Leader of the BatchGroup. After the current Leader is completed, the queue is popped out and the next Leader will execute. Through these three points, the semantics of concurrent execution and serial execution are guaranteed to be the same.

[0034] 8. In the reading process, the storage layer iterator parameter IterOption supports the upper layer to pass the Sequence. When NewIter generates a new iterator, it asserts that the parameter Sequence is not empty. In the writing process, because the Go layer has already formed a BatchGroup, the storage layer is no longer formed. After writing is completed, the latest Sequence is recorded in the storage layer.

[0035] Comparison of TPMC performance before and after transformation

[0036] TPMC CPU Mem(G) Control group 95938 89.13％ 75.4 Experimental Group 99801 90.03％ 75.9

[0037] Table 1

[0038] Note: RocksDB is used as the storage engine. The control group is the code before transformation, and the experimental group is the transformation based on the present invention.

[0039] The above description is only a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, and is not used to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. An optimization method for BatchGroup serialization calls based on CockroachDB, characterized in that: By moving the Sequence of the storage layer to the Go layer, a globally incremented Sequence number is pre-assigned to each BatchGroup when writing data, and the current Sequence is brought to the storage layer when reading data, thereby achieving concurrent writing of BatchGroup; When BatchGroup calls CGO to perform a write operation, the Go layer allocates Sequence segments to each BatchGroup in sequence by maintaining the global variable Sequence, releasing their sequential coupling relationship and implementing concurrent calls of BatchGroup; The working steps are as follows: 1) Each writing coroutine writes data into a batch and adds it to the pending queue after commit. The first batch in the queue is used as the leader to determine a batch group. The other batches in the group are merged into the leader and submitted by the leader. 2) Maintain a global increasing Sequence, which represents the order in which data is written; 3) Before the Leader makes a CGO call, it calculates the number of write operations in the Batch (count), and then pre-allocates the sequence number segment [Sequence + 1, Sequence + count] for it. At the same time, it updates the Sequence to Sequence + count. The sequence number represents the relative order of data writing. 4) When reading, NewIter returns a storage layer iterator. If the sequence field of IterOption is empty, it means it is not a snapshot query, and the current Sequence is used for assignment, which means that only data less than or equal to Sequence can be seen; Maintain a semaphore Semophere to control the number of concurrent BatchGroups. Call Semophere's acquire() method. If the concurrency threshold is not reached, it returns. Otherwise, it blocks until one of the BatchGroups is executed and releases the resource.

2. The method according to claim 1, characterized in that The Go layer maintains a globally increasing Sequence, which represents the order in which data is written. When the database is started, the Sequence is loaded into the memory from the manifest file. When the database is running, the Sequence is still written to the storage layer.

3. The method according to claim 1, characterized in that When calling the storage layer write interface to write data, you need to pre-allocate a continuous Sequence segment for the BatchGroup: 1) Before calling CGO, the leader of a batch group calculates the number of write operations in the current batch, and then obtains the [Sequence + 1, Sequence + count] segment for writing data in this group. Because more than one batch group obtains sequence segments concurrently, a while loop and CAS operation are required. The pre-allocation is exited after successful pre-allocation, otherwise reallocation is performed until success. At the same time, it should be ensured that the sequence segments allocated to the batch groups in the pending queue are guaranteed to be sequentially increasing to ensure the sequential semantics of the data. 2) BatchGroup calls CGO concurrently. Before calling, BatchGroup needs to be pushed into the queue. After the call returns, if the previous BatchGroup has been dequeued, it will be directly dequeued and returned; otherwise, it needs to wait for the previous BatchGroup to be completed before returning; to ensure that the semantics remain unchanged during concurrent reading and writing.

4. The method according to claim 1, characterized in that The read interface of the storage layer is to generate a storage layer iterator through NewIter. After the Sequence is promoted to the Go layer, it is necessary to assign a value to the sequence field of IterOption and pass it to the storage layer each time it is read. The read logic of the storage layer remains unchanged and still uses Sequence as the basis of its MVCC.

5. The method according to claim 1, characterized in that When writing, the storage layer needs to receive the sequence from the upper layer and write it into the memory or disk together with the data. At the same time, because the Go layer has adopted the Batch Group strategy, the C layer no longer forms a BatchGroup.

Images