Database transaction load sequencing method based on conflict relation

Abstract

The invention discloses a database transaction load sequencing method based on a conflict relationship, which is characterized in that non-conflict operations are grouped into operation batches in parallel by analyzing data access conflicts among the operations based on an original execution log of a database transaction load; the original execution sequence of conflict operations is deduced according to operation types, version dependence and lock competition conditions, then operation batches are sorted according to the sequence, an operation batch sequence is constructed, and therefore it is ensured that the execution result of the sequence is equivalent to the original execution result of a transaction load. Compared with the prior art, the method has the advantages that a kernel of a database system does not need to be intruded, the analysis speed is linearly expanded along with the scale of the original execution log, and good expandability and application prospects are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of database testing and debugging, and in particular to a database transaction load sequencing method based on conflict relations and proposed for inferring the execution order of transaction loads. Technical Background

[0002] In database systems, because the order in which operations are scheduled by concurrency control mechanisms is invisible to the outside world, clients cannot precisely control the order in which operations are executed when replaying concurrent transactions to replicate the results of transaction workloads, resulting in uncertain replay results. Therefore, to ensure that the replayed transaction workload is equivalent to the original execution results—that is, that the data read or modified by each operation is consistent with the original execution—it is necessary to derive an operation sequence that produces equivalent execution results.

[0003] Existing transaction operation sequencing technologies are mainly divided into two categories: intrusive execution trace capture methods and dependency graph construction and sorting methods. The first type of method requires going deep into the system and obtaining the precise operation scheduling sequence by modifying the database management system (DBMS) or operating system. For example, R3 [1] The Record-Replay-Retroaction framework records shared memory state changes and transaction behaviors, and then replays the program state during the replay phase to ensure repeatable execution. However, this method requires collaborative modifications to the database kernel and client, and requires recording a large amount of logs, resulting in high system overhead and intrusiveness. Similarly, LEAP [2] Lightweight Deterministic Replay (LDR) captures instruction-level information about thread scheduling and shared memory access in a multi-core environment, preserving the original execution order during replay. While this approach offers high accuracy, it is expensive to implement and difficult to deploy in production environments.

[0004] The second type of method attempts to deduce a replayable execution order by constructing an operation dependency graph without intrusion. This type of method usually extracts the read and write behavior of transactions or operations from the original execution log, and establishes dependency edges based on conflict rules such as "read after write", "write after read", or "write after write" to form a directed graph. Any path in the dependency graph represents a set of operations with execution constraints. By topologically sorting the graph, an equivalent serial scheduling order can be obtained. For example, DGCC [3] (Dependency Graph based Concurrency Control) divides transactions into multiple operation "fragments" and determines their scheduling order by building a dependency graph. However, as the log scale increases, the cost of building and maintaining the graph increases sharply, and its time and space complexity grows superlinearly. In a distributed environment, DistDGCC [4]Attempts are made to alleviate bottlenecks by sharding the dependency graph and executing it in parallel, but this still requires a lot of cross-node communication and synchronization, and scalability is still limited.

[0005] In summary, existing transaction sequencing methods rely on internal database mechanisms, resulting in high implementation costs and a sharp performance degradation as log size increases, making them difficult to handle large-scale transaction loads. Therefore, a non-invasive and highly scalable operation sequencing method is urgently needed to support the replay of large-scale transaction loads. Summary of the Invention

[0006] The present invention addresses the shortcomings of the prior art by providing a conflict-relationship-based database transaction load sequencing method. This method employs a non-invasive operation trajectory analysis method to achieve highly scalable transaction load sequencing. Based on the original execution log of the database transaction load, the method analyzes data access conflicts between operations, groups non-conflicting operations into parallel operation batches, and infers the original execution order of the conflicting operations based on operation type, version dependency, and lock contention conditions. The operation batches are then sorted according to this order to construct an operation batch sequence, thereby ensuring that the execution results of the sequence are equivalent to the original execution results of the transaction load. This method does not require intrusion into the database system kernel, and the analysis speed scales linearly with the size of the original execution log, demonstrating excellent scalability and application prospects.

[0007] The specific technical solution for achieving the objectives of the present invention is: a method for resolving ambiguity between baseline length and angle constraints, characterized by employing a non-intrusive operation trajectory analysis method to achieve highly scalable transaction load sequencing. The method specifically includes the following core steps:

[0008] Step 1: Operation trajectory preprocessing

[0009] Extract the operation trace set of each thread from the client's multi-threaded transaction log, and sort all operations according to the timestamp sent by the client.

[0010] Step 2: Build an operation batch sequence

[0011] Traverse the head of each thread's operation trace and filter the operations with overlapping time by timestamp to build a candidate set.

[0012] Step 3: Conflict Detection and Batch Division

[0013] Conflict detection rules are executed against the candidate set to infer causal order. For read operations, the target data item's version is checked to see if it exists in the version chain. If a visible version exists and its timestamp is less than the send timestamp of the operation, the item is allowed to be added to the current batch; otherwise, processing is delayed. For write operations, the item can only be added to the current batch if no other operations in the current batch access the item and the item is not locked by the previous batch.

[0014] Step 4: Iterative Build

[0015] If there are still operations that have not been assigned to the operation batch, return to step 3 to start building a new operation batch. Otherwise, when all operations are included in the operation batch sequence, the transaction load sequencing method ends.

[0016] In step 1, when extracting operation traces from the client's multi-threaded transaction log, the following metadata is parsed for each operation record in the log: operation type (read / write), target data item identifier, client send and receive timestamps, thread identifier, and operation content. Based on the client send timestamp, the operation traces of each thread are arranged in monotonically increasing order to form an ordered operation sequence.

[0017] In step 2, based on the preprocessed global operation sequence, before each batch is built, the receive timestamp of each thread's head operation is obtained and the minimum receive timestamp is compared. The head operations of all threads are then traversed. If any operation has a send timestamp less than the minimum receive timestamp, it is added to the candidate operation set.

[0018] In the step three, for all operations in the candidate operation set, the causal order between the operations is determined based on the conflicting accesses to the same data item, and the read and write operations that meet the requirements are added from the candidate set to the current batch set based on the version chain maintained for each data item. Specifically, each data item maintains a version chain, and the chain nodes contain the version number, creation timestamp, and submission status. For read operations, the version chain is traversed to find the most recently submitted version. If there is a visible version and the timestamp of the version is less than the sending timestamp of the operation, the read operation is allowed to join the current batch, otherwise, the operation is marked as delayed processing. For write operations, a global lock table is established to record the data items locked by the current batch and the submitted batches. The write operation must simultaneously meet the conditions that no other operations in the current batch candidate set access the same data item and that the data item has no unreleased exclusive lock in the lock table. After completing the conflict detection, the qualified operation is removed from the candidate set and added to the current batch, and the version chain and lock table status are updated.

[0019] In step 4, after each operation batch is built, the head element of each thread's unassigned operation is rescanned and the construction begins again from step 2. The iteration terminates when the pointers of all thread operation traces reach the end, and finally outputs a load operation execution order sequence based on the conflict relationship.

[0020] Compared with the prior art, the present invention has the following technical effects and current technological progress:

[0021] 1) The non-intrusive design is compatible with transaction load sequencing in various database systems, reducing sequencing complexity from exponential to linear, and supporting load processing of tens of thousands of operations.

[0022] 2) Determine the execution order between operations based on lock conflicts to ensure consistency between data version access and original execution.

[0023] 3) There is no need to invade the database system kernel, and the analysis speed scales linearly with the size of the original execution log, which has good scalability and application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the process of the present invention;

[0025] Figure 2 Schematic diagram of the execution trajectory of each thread in Example 1;

[0026] Figure 3 Schematic diagram of batch division in Example 1. DETAILED DESCRIPTION

[0027] See Figure 1 The present invention adopts non-invasive operation trajectory analysis, which specifically includes the following steps:

[0028] Step 1: Operation trajectory preprocessing

[0029] Extract the operation trace set of each thread from the client multi-threaded transaction log, denoted as ,in Indicates the Extract the data set accessed by all operations , and initialize the candidate operation set and operation batch sequence

[0030] Step 2: Build an operation batch sequence

[0031] Traverse the head of each thread's operation trace, filter out operations with overlapping time by timestamp, and build a candidate set , specifically including:

[0032] 2-1: Select the earliest ending operation timestamp among all thread traces: , will satisfy The conflicting operations are added to the candidate set ,in Sends a timestamp to the client for the operation.

[0033] Step 3: Conflict Detection and Batch Division

[0034] 3-1: For candidate sets Conflict detection rules are executed to infer causal ordering, for read operations , check its target data item Does the version exist in the version chain? When there is a visible version satisfy Allow to join the current batch, otherwise delay processing; for write operations , you must meet the following conditions to join the current batch:

[0035] 1) The data item is not locked by the previous batch;

[0036] 2) No operation in the current batch accesses the data item; (avoiding read-write conflicts)

[0037] 3) is the earliest returned write operation among the time-overlapping operations, i.e. .

[0038] 3-2: Conflicting operation isolation: For multiple write operations on the same data item, only the earliest completed write operation is retained, and the remaining operations are added to the next round of candidate sets.

[0039] Step 4: Iterative Build

[0040] If there are still operations that have not been assigned to the operation batch, return to step 3 to start building a new operation batch. Otherwise, when all operations are included in the operation batch sequence, the transaction load sequencing method ends.

[0041] The iterative algorithm design is detailed in the iterative process shown in Table 1 below:

[0042] Table 1 Database transaction load sequencing algorithm based on conflict relations

[0043]

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0045] Example 1

[0046] See Figure 2 ,The figure shows the operation trace sequence extracted by each thread in the order of timestamps, where the accessed data items include a and b. ,The first round of batch processing, With the minimum client receiving timestamp. In the database transaction load sequencing algorithm based on conflict relations, all conflicting operations are added to the candidate set, that is, all operations with a timestamp less than the minimum receiving timestamp. For read operations, the read version is exactly the same as The same, so the candidate set contains four operations at the same time: . Next, causal order inference is performed, where A new version of data item a is written and Read in, the latter occurs after the former in causal logic, while the other two operations have no causal relationship.

[0047] See Figure 2 , In addition, in the mutual exclusion lock mechanism implementation of database transactions, the write operation of one transaction must be committed before the write operation of another transaction is allowed to change the same data item. It must occur after all operations in trace 2, and the commit operation does not access the data item. After the second batch is built, the commit operation is considered a separate batch, and finally the write operation of track 3 is included in the fourth batch.

[0048] Through the above iterative construction steps, a non-intrusive database transaction load sequencing method can be implemented based on the database mutual exclusion lock theory mechanism and the conflict relationship between various operations between threads. The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the inventive concept, any changes and advantages that can be conceived by those skilled in the art are included in the present invention and are protected by the appended claims.

Claims

1. A database transaction load sequencing method based on conflict relations, characterized in that: A non-intrusive operation trace analysis method is used to achieve highly scalable transaction load sequencing. The method specifically includes: Step 1: Perform trajectory preprocessing By parsing the original execution log of the database transaction load, extracting the operations and related information, and sorting all operations in ascending order by the client's sending timestamp; Step 2: Build an operation batch sequence Traverse the operations one by one in order and determine whether the operation can be added to the current operation batch based on the conflict judgment conditions of the operations on the same data item. If it can be added, add it to the current operation batch; otherwise, skip the operation; Step 3: Conflict Detection and Batch Division If the subsequent operation does not overlap with the operation in the current batch in time or there is no inter-operation conflict, or there is no subsequent operation, then the construction of the current operation batch is terminated and it is appended to the end of the operation batch sequence. The inter-operation conflict is that multiple operations access the same data and at least one of the operations is a write operation. Step 4: Iterative Build If there are still operations that have not been assigned to an operation batch, return to step 2 to start building a new operation batch; otherwise, end the building process.

2. The database transaction load sequencing method based on conflict relations according to claim 1, characterized in that: The relevant information in step 1 includes: a timestamp sent by the client, a timestamp received by the client, an operation type, and an accessed data object.

3. The database transaction load sequencing method based on conflict relations according to claim 1, characterized in that: The second step determines whether the read operation or write operation can be added to the current operation batch based on the conflict operation judgment condition. The read operation is determined based on the data version it reads. If the data version has been created in the constructed operation batch, the read operation is added to the current operation batch, otherwise it cannot be added to the current operation batch; the write operation is determined based on its data access relationship and lock competition relationship. If there is still an operation in the constructed operation batch holding the lock for accessing the data by the write operation, it cannot be added; if there is an operation in the current operation batch that reads the data accessed by the write operation, it cannot be added; if there is a write operation with overlapping execution time and accessing the same data item, it can be added to the current operation batch only when the client receiving timestamp of the write operation is the smallest; in other cases, the write operation is directly added to the current operation batch.

4. The database transaction load sequencing method based on conflict relations according to claim 1, characterized in that: Step three determines whether subsequent operations can be added to the current operation batch based on the data written by the operations in the current operation batch and the execution time interval of the operations. That is, if the operations in the current operation batch have written all data items in the complete transaction load, then all subsequent operations are not related to the current operation batch.

5. The database transaction load sequencing method based on conflict relations according to claim 1 or claim 3, characterized in that: The step 2 is implemented by programming using an operation batch sequence construction algorithm, and a computing processor is used to implement iterative construction of operation batches to form an operation batch sequence equivalent to the original execution result of the transaction load; the computing processor is an ordinary computer or an embedded processor.

6. The database transaction load sequencing method based on conflict relations according to claim 3, characterized in that: The data version is created in the constructed operation batch using version chain maintenance, that is, if the current operation batch modifies a certain data, the modification result will be added to the version chain as a new data version; when a certain data version in the version chain is not visible to any unfinished transaction in the current operation batch, the data version will be removed from the version chain.

7. The database transaction load sequencing method based on conflict relations according to claim 3 or claim 6, characterized in that: The data version uses a lock set to maintain the locks held by operations in the constructed operation batch. If the current operation batch modifies a certain data, the data will be added to the lock set; if the current operation batch ends a transaction, all data modified by the transaction will be removed from the lock set.

Images