Distributed transaction processing method and device based on hybrid concurrency control

Abstract

The invention provides a distributed transaction processing method and device based on hybrid concurrency control, and the method comprises the steps: calculating a transaction rollback time ratio when an operation mode is a multi-version optimistic concurrency control mode; if the transaction rollback time ratio exceeds a preset rollback time ratio threshold, switching the operation mode to a distributed two-stage lock protocol mode to process the distributed transaction; when the operation mode is a distributed two-stage lock protocol mode, calculating to obtain a transaction waiting lock time proportion; and if the transaction waiting lock time proportion is lower than a preset waiting lock time proportion threshold, switching the operation mode to a multi-version optimistic concurrency control mode to process the distributed transaction. In the scheme, a hybrid concurrency control protocol combining multi-version optimistic concurrency control and a distributed two-stage lock protocol is provided, and a proper concurrency control strategy is selected in different read-write load environments according to the characteristics of the two strategies, so that the aims of improving the transaction processing efficiency and the reliability of the system are fulfilled.

Description

Technical Field

[0001] The present invention relates to the technical field of transaction management, and in particular, to a distributed transaction processing method and apparatus based on hybrid concurrency control. Background Art

[0002] With the rapid development of the Internet and distributed computing, distributed systems have become one of the main architectures of modern applications. Data is usually stored on multiple nodes and accessed and modified in parallel by multiple concurrent users or application programs. In such a complex environment, ensuring data consistency and system reliability has become a major challenge. As one of the key technologies to address this challenge, distributed transaction processing aims to guarantee the atomicity, consistency, isolation, and durability of complex operations across multiple nodes.

[0003] In existing distributed database systems, distributed transaction processing is generally implemented in the way pioneered by System R* in the 1980s, mainly by requiring all participating machines to reach a consensus protocol at the time of commit to ensure atomicity and durability. However, this concurrency control mechanism reduces throughput and increases latency. To ensure isolation, all locks of a transaction must be held during the entire protocol negotiation. The problem with this is that multiple network round-trips are required between all participating machines, so the time required is usually much longer than the time required to execute all local transaction logic. If some frequently accessed records are frequently involved in distributed transactions, the additional time for locking these records will greatly affect the overall transaction throughput.

[0004] In summary, the transaction processing efficiency and reliability of existing distributed database systems are relatively low, which is an urgent problem to be solved currently. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a distributed transaction processing method and apparatus based on hybrid concurrency control to achieve the purpose of improving transaction processing efficiency and system reliability.

[0006] To achieve the above object, embodiments of the present invention provide the following technical solutions:

[0007] A first aspect of an embodiment of the present invention discloses a distributed transaction processing method based on hybrid concurrency control, which is applied to a distributed database system. The method includes:

[0008] Determine the current operating mode of the distributed database system;

[0009] When the current operating mode of the distributed database system is the multi-version optimistic concurrency control mode, calculate the proportion of transaction rollback time;

[0010] If the proportion of the transaction rollback time exceeds a preset rollback time proportion threshold, switch the operating mode of the distributed database system to the distributed two-phase locking protocol mode to process distributed transactions using the distributed two-phase locking protocol;

[0011] When the current operating mode of the distributed database system is the distributed two-phase locking protocol mode, calculate the proportion of the transaction waiting lock time;

[0012] If the proportion of the transaction waiting lock time is lower than a preset waiting lock time proportion threshold, switch the operating mode of the distributed database system to the multi-version optimistic concurrency control mode to process distributed transactions using the multi-version optimistic concurrency control method.

[0013] Preferably, when the current operating mode of the distributed database system is the multi-version optimistic concurrency control mode, calculating the proportion of the transaction rollback time includes:

[0014] When the current operating mode of the distributed database system is the multi-version optimistic concurrency control mode, within a preset first time window, obtain the total first transaction processing time and the transaction rollback time; the transaction rollback time is the total time spent on transaction rollback due to transaction conflicts;

[0015] Based on the total first transaction processing time and the transaction rollback time, calculate the proportion of the transaction rollback time.

[0016] Preferably, when the current operating mode of the distributed database system is the distributed two-phase locking protocol mode, calculating the proportion of the transaction waiting lock time includes:

[0017] When the current operating mode of the distributed database system is the distributed two-phase locking protocol mode, within a preset second time window, obtain the total second transaction processing time and the transaction waiting lock time; the transaction waiting lock time is the total time spent on transaction waiting due to lock resource conflicts;

[0018] Based on the total second transaction processing time and the transaction waiting lock time, calculate the proportion of the transaction waiting lock time.

[0019] Preferably, the method further includes:

[0020] Set the distributed database system so that the proportion of the transaction rollback time of the distributed database system is fixed at a preset fixed value;

[0021] Perform transaction processing tests for each candidate rollback time proportion to obtain the transaction execution time corresponding to each candidate rollback time proportion;

[0022] Set the ratio of the candidate rollback time corresponding to the shortest transaction execution time as the rollback time ratio threshold.

[0023] Preferably, the method further includes:

[0024] Set the distributed database system such that the ratio of the transaction waiting lock time of the distributed database system is successively fixed at multiple preset time fixed values;

[0025] Whenever the ratio of the transaction waiting lock time of the distributed database system is fixed at any of the time fixed values, perform transaction processing tests for each candidate rollback time ratio to obtain multiple transaction execution times corresponding to each candidate rollback time ratio;

[0026] Among all the transaction execution times, find the shortest transaction execution time;

[0027] Set the ratio of the candidate rollback time corresponding to the shortest transaction execution time as the rollback time ratio threshold.

[0028] Preferably, the method further includes:

[0029] Set the distributed database system such that the ratio of the transaction rollback time of the distributed database system is successively fixed at multiple preset ratio fixed values;

[0030] Whenever the ratio of the transaction rollback time of the distributed database system is fixed at any of the ratio fixed values, perform transaction processing tests for each candidate waiting lock time ratio to obtain multiple transaction execution times corresponding to each candidate waiting lock time ratio;

[0031] Among all the transaction execution times, find the shortest transaction execution time;

[0032] Set the ratio of the candidate waiting lock time corresponding to the shortest transaction execution time as the waiting lock time ratio threshold.

[0033] Preferably, the method further includes:

[0034] When in the initial state, run the multi-version optimistic concurrency control mode to process distributed transactions using the multi-version optimistic concurrency control method in the initial state.

[0035] A second aspect of the embodiments of the present invention discloses a distributed transaction processing device based on hybrid concurrency control, which is applied to a distributed database system. The device includes:

[0036] A determination unit, configured to determine the current running mode of the distributed database system;

[0037] A first calculation unit, configured to calculate the proportion of transaction rollback time when the current operating mode of the distributed database system is the multi-version optimistic concurrency control mode;

[0038] A first switching unit, configured to switch the operating mode of the distributed database system to the distributed two-phase locking protocol mode if the proportion of transaction rollback time exceeds a preset rollback time proportion threshold, so as to process distributed transactions by using the distributed two-phase locking protocol;

[0039] A second calculation unit, configured to calculate the proportion of transaction waiting lock time when the current operating mode of the distributed database system is the distributed two-phase locking protocol mode;

[0040] A second switching unit, configured to switch the operating mode of the distributed database system to the multi-version optimistic concurrency control mode if the proportion of transaction waiting lock time is lower than a preset waiting lock time proportion threshold, so as to process distributed transactions by using the multi-version optimistic concurrency control method.

[0041] Preferably, the first calculation unit is specifically configured to:

[0042] When the current operating mode of the distributed database system is the multi-version optimistic concurrency control mode, within a preset first time window, obtain the total first transaction processing time and the transaction rollback time; the transaction rollback time is the total time spent on transaction rollback due to transaction conflicts;

[0043] Based on the total first transaction processing time and the transaction rollback time, calculate the proportion of transaction rollback time.

[0044] Preferably, the second calculation unit is specifically configured to:

[0045] When the current operating mode of the distributed database system is the distributed two-phase locking protocol mode, within a preset second time window, obtain the total second transaction processing time and the transaction waiting lock time; the transaction waiting lock time is the total time spent on transaction waiting due to lock resource conflicts;

[0046] Based on the total second transaction processing time and the transaction waiting lock time, calculate the proportion of transaction waiting lock time.

[0047] A distributed transaction processing method and apparatus based on hybrid concurrency control provided by the embodiments of the present invention are applied to a distributed database system. The method includes: determining the current running mode of the distributed database system; when the current running mode of the distributed database system is the multi-version optimistic concurrency control mode, calculating the proportion of transaction rollback time; if the proportion of transaction rollback time exceeds a preset rollback time proportion threshold, switching the running mode of the distributed database system to the distributed two-phase locking protocol mode to process distributed transactions using the distributed two-phase locking protocol; when the current running mode of the distributed database system is the distributed two-phase locking protocol mode, calculating the proportion of transaction waiting lock time; if the proportion of transaction waiting lock time is lower than a preset waiting lock time proportion threshold, switching the running mode of the distributed database system to the multi-version optimistic concurrency control mode to process distributed transactions using the multi-version optimistic concurrency control method. In this solution, a hybrid concurrency control protocol combining multi-version optimistic concurrency control and distributed two-phase locking protocol is proposed. According to the characteristics of these two strategies, appropriate concurrency control strategies are selected in different read-write load environments, which can optimize the system response speed and throughput, thereby improving transaction processing efficiency, and at the same time ensuring the reliability of the system in various load environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.

[0049] Figure 1 It is a flowchart of a distributed transaction processing method based on hybrid concurrency control disclosed in the embodiments of the present invention;

[0050] Figure 2 It is a schematic diagram of the data item row version organization method disclosed in the embodiments of the present invention;

[0051] Figure 3 It is a state transition diagram of an MVOCC transaction disclosed in the embodiments of the present invention;

[0052] Figure 4 It is a schematic diagram of the communication structure of a D2PL disclosed in the embodiments of the present invention;

[0053] Figure 5 It is a schematic diagram of the transaction execution time of different RTRs under a fixed WLT disclosed in the embodiments of the present invention;

[0054] Figure 6Schematic diagram of transaction execution time for different WLTs under fixed RTR disclosed in the embodiments of the present invention;

[0055] Figure 7 Schematic diagram of transaction execution time in a read-intensive environment disclosed in the embodiments of the present invention;

[0056] Figure 8 Schematic diagram of transaction execution time in a write-intensive environment disclosed in the embodiments of the present invention;

[0057] Figure 9 Schematic diagram of transaction execution time in a read-write balanced environment disclosed in the embodiments of the present invention;

[0058] Figure 10 Structural diagram of a distributed transaction processing device based on hybrid concurrency control disclosed in the embodiments of the present invention. Detailed implementation manners

[0059] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0060] In this application, the term "including", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0061] As can be seen from the background art, the transaction processing efficiency and reliability of existing distributed database systems are relatively low, which is an urgent problem to be solved at present.

[0062] Therefore, the embodiments of the present invention disclose a distributed transaction processing method based on hybrid concurrency control. In this solution, a hybrid concurrency control protocol combining multi-version optimistic concurrency control and distributed two-phase locking protocol is proposed. According to the characteristics of these two strategies, appropriate concurrency control strategies are selected in different read-write load environments, which can optimize the system's response speed and throughput, thereby improving transaction processing efficiency, and at the same time ensuring the reliability of the system in various load environments.

[0063] Such as Figure 1As shown in the figure, it is a flowchart of a distributed transaction processing method based on hybrid concurrency control disclosed in an embodiment of the present invention. This method is applied to a distributed database system and includes the following steps:

[0064] Step S101: Determine the current operating mode of the distributed database system.

[0065] Since the purpose of this solution is to enable the system to flexibly switch between two main concurrency control mechanisms, Optimistic Concurrency Control (OCC) and Pessimistic Concurrency Control (PCC), by combining a hybrid concurrency control protocol of Multi-Version Optimistic Concurrency Control (MVOCC) and Distributed Two-Phase Locking Protocol D2PL, in order to handle different loads and conflict situations, thereby maximizing the performance and stability of the system.

[0066] Among them, OCC assumes that there are few conflicts between transactions, and reduces the need for locks by detecting conflicts during the transaction commit phase, which is suitable for environments with a low probability of conflicts; while PCC prevents conflicts from occurring through a lock mechanism during the transaction execution process, and is applicable to scenarios with frequent updates and a high probability of conflicts.

[0067] Therefore, in step S101, first determine whether the current operating mode of the distributed database system is in the MVOCC mode or the D2PL mode, and then, through the corresponding switching strategy, determine whether the switching condition is met. If it is met, switch from the current operating mode to the other operating mode.

[0068] In one embodiment, when in the initial state, run the multi-version optimistic concurrency control mode to process distributed transactions using the multi-version optimistic concurrency control method in the initial state.

[0069] That is to say, when the distributed database system starts to run, it runs in the multi-version optimistic concurrency control mode.

[0070] Step S102: When the current operating mode of the distributed database system is the multi-version optimistic concurrency control mode, calculate the ratio of the transaction rollback time.

[0071] In step S102, the ratio of the transaction rollback time (Rollback Time Ratio, RTR): within a given time window, the total time T r spent on rolling back distributed transactions due to conflicts t and the total transaction processing time T

[0072]

[0073] Among them, the starting point of the transaction rollback time is the conflict detection moment (usually occurring in the transaction commit phase), and the ending point is the moment when the rollback operation is completed, that is, the distributed database system is restored to the consistent state before the conflict occurs. The total transaction rollback time is the sum of the transaction rollback times.

[0074] In the specific implementation process of step S102, when the current running mode of the distributed database system is the multi-version optimistic concurrency control mode, within a preset first time window, obtain the total first transaction processing time and the transaction rollback time; the transaction rollback time is the total time spent on transaction rollback due to transaction conflicts; based on the total first transaction processing time and the transaction rollback time, calculate the proportion of the transaction rollback time.

[0075] Among them, the first time window represents a time period. Multiple first time windows can be preset to continuously detect until the running mode switches to the distributed two-phase locking protocol mode.

[0076] In step S102, when the current running mode of the distributed database system is the multi-version optimistic concurrency control mode, the distributed database system will use the multi-version optimistic concurrency control method to process distributed transactions. The multi-version optimistic concurrency control MVOCC combines the advantages of MVCC (Multi-Version Concurrency Control) and OCC, aiming to improve the efficiency and concurrency of the distributed database system when processing read-write transactions. The multi-version optimistic concurrency control shows its uniqueness and efficiency in the following aspects:

[0077] 1. Multi-version storage;

[0078] In MVOCC, the distributed database system maintains multiple versions for each data item. The key advantage of this multi-version strategy is that it provides read-write isolation, thus reducing direct conflicts between transactions. Each version generates a new row version for the corresponding data row. The invention adopts an organization method from old to new, and organizes multiple row versions of the record into a linked list through a pointer variable pointer, which is called a version chain. This organization method does not require modifying the row version when creating a new version, and only needs to insert a new version at the end of the chain, which provides convenience for rolling back in case of transaction failure.

[0079] Such as Figure 2 shown, it is a schematic diagram of a data item row version organization method disclosed in an embodiment of the present invention.

[0080] Among them, a simple example of a transaction update record is shown. In this example, the primary key Pkey of the index maps to an original version with a transaction version number txn_id of 0. After the update operation is executed, the row version pointer with txn_id of 1 will point to the new row version, and the start time and end time of the transaction will be recorded. When this version is the latest version, the pointer points to NULL.

[0081] Read operation: A read transaction can access the latest committed data version before the start of the transaction, so it can avoid waiting for the current ongoing write operation. This increases the performance of concurrent read operations, especially in applications with a large number of read-only transactions; Write operation: When creating a new data version, a write transaction does not overwrite the old version but generates a new version instance. In this way, even before the new version of the data is committed, a read transaction can still access the old version of the data.

[0082] 2. Efficient read and write operations;

[0083] Since MVOCC combines multi-version and optimistic detection, it can efficiently support read and write operations:

[0084] Non-blocking read: Read transactions are not affected by the current write activity because they can access the historical data versions corresponding to their isolation levels. Through multi-version concurrency control (MVCC), the database generates a new version for each update of the data and retains the old versions, enabling read transactions to access the appropriate data versions according to their isolation levels.

[0085] For example, in the "read committed" isolation level, a read transaction can only access the latest committed data version of other transactions; in the "repeatable read" isolation level, a read transaction can only access the committed versions in the locked view at the start of the transaction, and the changes made by subsequent write transactions are invisible to it. This mechanism ensures the mutual independence between read and write operations, effectively avoiding problems such as dirty reads, non-repeatable reads, and phantom reads, thus improving the concurrency performance and data consistency of transactions.

[0086] Parallel write operation: Write transactions can be carried out without directly blocking read transactions because each write operation creates a new data version. This strategy significantly improves the performance in read-intensive and write-intensive workloads.

[0087] 3. Conflict handling;

[0088] As Figure 3 shown, this is a transaction state transition diagram of MVOCC disclosed in an embodiment of the present invention, which shows the transition relationship between transaction states and the conflict handling process.

[0089] The strategy for MVOCC to handle transaction conflicts is achieved through rollback and redo.

[0090] Rollback operation: When conflict detection in the commit phase discovers a conflict, that is, when multiple transactions are executed concurrently and the inconsistency caused by read and write operations on the same data object threatens data consistency.

[0091] Common conflicts include: write-write conflict (two transactions simultaneously attempt to modify the same piece of data) and read-write conflict (one transaction reads a certain data while another transaction modifies that data). The transaction will revoke all executed operations and restore the relevant data to the previous consistent state. This process ensures that uncompleted transactions do not affect the consistency and correctness of the system.

[0092] Redo operation: First, restart the transaction and restart it using the original operation logic. This usually means that the transaction will execute from the beginning, possibly using a new timestamp. Second, re-execute the operations. The transaction re-executes all its operations, this time with the new timestamp. During this process, the transaction may read the new data version of the resource it needs to access according to the current new timestamp, that is, when a certain transaction modifies a data object and successfully commits, the database generates a new version for this object and marks it as committed. When other transactions restart, they will read these latest committed versions to ensure that the accessed data is the latest and visible consistent data; or the transaction avoids the previously conflicting situation at different time points because when the transaction restarts, a new timestamp is assigned, and the new timestamp is usually later than the previous one, indicating that the transaction "restarts" logically. Using the new timestamp, the transaction will read the data version consistent with its timestamp from the database instead of the version that has previously conflicted. The transaction re-executes its logic from the beginning and processes the data from a new perspective to avoid repeating the operations that previously caused conflicts. Finally, attempt to commit again. After re-execution, the transaction attempts to commit again. If no conflict occurs in this commit, the transaction will be successfully committed; if there are still conflicts, according to the system policy, there may be more retries or ultimately give up.

[0093] Example: Assume that transactions T1 and T2 conflict: T1 modifies data A, but when committing, it is found that T2 is also modifying A and has already committed. T1 triggers a rollback, releases resources, and reads the latest data version committed by T2. T1 assigns a new timestamp, re-executes the operations, and commits in the new context.

[0094] In summary, in a distributed environment with a relatively low conflict probability, MVOCC provides a concurrency control method that can significantly improve concurrency and system throughput while maintaining data consistency.

[0095] Step S103: If the proportion of transaction rollback time exceeds a preset rollback time proportion threshold, switch the operating mode of the distributed database system to the distributed two-phase locking protocol mode to handle distributed transactions using the distributed two-phase locking protocol.

[0096] In step S103, the rollback time proportion threshold can be formulated or adjusted based on the historical performance data of the distributed database system, system tolerance, and specific business requirements.

[0097] In one embodiment, during the operation in the MVOCC mode, if the distributed database management system detects an increase in write operations or an increase in the transaction conflict rate, the operating mode is switched to the D2PL mode to effectively reduce the occurrence frequency of conflicts and rollbacks.

[0098] Among them, an increase in write operations means that the proportion of write transactions in the total number of transactions reaches a preset proportion value, and an increase in the conflict rate means that the conflict rate reaches a preset conflict rate value.

[0099] It can be understood that the proportion of transaction rollback time can also reflect the conflict rate to a certain extent.

[0100] Step S104: When the current operating mode of the distributed database system is the distributed two-phase locking protocol mode, calculate the proportion of transaction waiting lock time.

[0101] In step S104, the proportion of transaction waiting lock time (Waiting Lock Time Percentage, WLT): within a given time window, the total time T w spent by the distributed transaction waiting due to lock resource conflicts t and the total transaction processing time T

[0102]

[0103] In the specific implementation process of step S104, when the current operating mode of the distributed database system is the distributed two-phase locking protocol mode, within a preset second time window, obtain the second total transaction processing time and the transaction waiting lock time; the transaction waiting lock time is the total time spent by the transaction waiting due to lock resource conflicts; based on the second total transaction processing time and the transaction waiting lock time, calculate the proportion of transaction waiting lock time.

[0104] Among them, the second time window represents a period. Multiple second time windows can be preset to continuously detect until the operating mode is switched to the multi-version optimistic concurrency control mode.

[0105] In step S104, when the current operating mode of the distributed database system is the distributed two-phase locking protocol mode, the distributed database system processes distributed transactions using the distributed two-phase locking protocol.

[0106] It should be noted that two-phase locking (2PL) is a widely used concurrency control protocol mainly used to ensure the serializability of transactions in a database system. 2PL can be divided into two phases according to its operating mechanism: the locking phase and the unlocking phase. In the locking phase, a transaction attempts to acquire all the necessary locks without releasing any locks; in the unlocking phase, the transaction releases all the held locks and no longer requests new locks. In a distributed database management system, usually the task of managing locks is assigned to a single node (an independent computing unit running the database service in the system, usually consisting of a server or a virtual machine instance), which means that there can only be one lock manager for all nodes. This central node, known as the centralized 2PL algorithm, will quickly become the bottleneck of the entire system. The failure and inaccessibility of the central node will cause serious failures in the system, making the system very unreliable. Therefore, the distributed two-phase locking protocol (D2PL) is adopted.

[0107] As Figure 4 shown, it is a schematic diagram of the communication structure of a D2PL disclosed in an embodiment of the present invention.

[0108] The distributed two-phase locking protocol (D2PL) equips each node with a lock manager. When a transaction is executed, the transaction coordinator first sends lock requests to the lock managers of all participating nodes. Then, the lock manager processes the received requests and, after obtaining a read lock (shared lock, allowing multiple transactions to concurrently read data but preventing other transactions from modifying the data) or a write lock (exclusive lock, allowing one transaction to modify the data and preventing other transactions from reading or modifying the data), records the lock status in the lock table, including information such as the transaction ID holding the lock, the lock type, and the timestamp of locking. And sends the data operation instructions to the data management systems of their respective nodes. After the transaction is completed and committed, the data management program sends a "operation completed" signal to its lock manager. After receiving this information, the transaction manager will instruct the lock manager to release the lock resources. This process ensures the effective coordination and resource management of transactions among collaborating sites.

[0109] As shown in Algorithm 1, the lock manager performs corresponding lock release or acquisition operations according to the operation instructions of the transaction coordinator. When the lock resources required by a transaction are occupied by other transactions, the lock manager will put the current request into a queue and wait for other transactions to release the resources. In this mode, transactions adopt preventive strategies by locking resources during execution to avoid conflicts. This strategy effectively improves the parallelism between transactions and significantly enhances the overall transaction processing performance of the system.

[0110]

[0111] Step S105: If the proportion of the transaction waiting for the lock time is lower than the preset threshold of the proportion of the waiting for the lock time, switch the running mode of the distributed database system to the multi-version optimistic concurrency control mode to process distributed transactions using the multi-version optimistic concurrency control method.

[0112] In step S105, the threshold of the proportion of the waiting for the lock time can be formulated or adjusted based on the historical performance data of the distributed database system, the system tolerance, and the specific business requirements.

[0113] In an embodiment, during the operation of the D2PL mode, if the distributed database management system detects a decrease in write operations and a decrease in the transaction conflict rate, the running mode is switched to the MVOCC mode to improve concurrency.

[0114] Among them, the decrease in write operations means that the proportion of write transactions in the total number of transactions is less than the preset proportion value, and the decrease in the conflict rate means that the conflict rate is less than the preset conflict rate value.

[0115] The embodiment of the present invention discloses a distributed transaction processing method based on the hybrid concurrency control HCC protocol, which is represented by an algorithm as follows:

[0116]

[0117] In an embodiment, through the control variable method, the influence of different rollback time proportion thresholds at a fixed WLT and the influence of different waiting for the lock time proportion thresholds at a fixed RTR are systematically tested. These tests aim to optimize the switching strategy of the concurrency control protocol, that is, to adjust or set the rollback time proportion threshold and the waiting for the lock time proportion threshold to improve the response speed and throughput of the system while maintaining data integrity and system reliability.

[0118] In specific implementation, first set multiple WLTs, that is, time fixed values, and record the execution time of transactions with different candidate rollback time proportions for each WLT. Then, set multiple RTRs, that is, proportion fixed values, and record the execution time of transactions with different candidate waiting for the lock time proportions for each RTR, as follows:

[0119] Set up a distributed database system so that the proportion of transaction waiting lock time in the distributed database system is fixed at multiple preset time fixed values in sequence; whenever the proportion of transaction waiting lock time in the distributed database system is fixed at any time fixed value, conduct transaction processing tests for each candidate rollback time proportion, and obtain multiple transaction execution times corresponding to each candidate rollback time proportion; among all the transaction execution times, find the shortest transaction execution time; set the candidate rollback time proportion corresponding to the shortest transaction execution time as the rollback time proportion threshold.

[0120] As Figure 5 shown, it is a schematic diagram of transaction execution time with different RTRs under a fixed WLT disclosed in an embodiment of the present invention.

[0121] Under the settings of WLT = 3.5, WLT = 4.0, and WLT = 4.5, the transaction execution time first decreases and then increases with the increase of the candidate rollback time proportion, forming a U-shaped curve. This indicates that there is an optimal candidate rollback time proportion that can balance lock waiting and rollback processing, thereby minimizing the transaction execution time.

[0122] Among them, the optimal candidate rollback time proportion is the candidate rollback time proportion corresponding to the shortest transaction execution time. For example, the shortest transaction execution time is Figure 5 the lowest point in the curve shown, and the corresponding candidate rollback time proportion is 1.5.

[0123] Set up a distributed database system so that the proportion of transaction rollback time in the distributed database system is fixed at multiple preset proportion fixed values in sequence; whenever the proportion of transaction rollback time in the distributed database system is fixed at any proportion fixed value, conduct transaction processing tests for each candidate waiting lock time proportion, and obtain multiple transaction execution times corresponding to each candidate waiting lock time proportion; among all the transaction execution times, find the shortest transaction execution time; set the candidate waiting lock time proportion corresponding to the shortest transaction execution time as the waiting lock time proportion threshold.

[0124] As Figure 6 shown, it is a schematic diagram of transaction execution time with different WLTs under a fixed RTR disclosed in an embodiment of the present invention.

[0125] When the RTR is set to 1.5, the transaction execution time shows a relatively stable trend as the proportion of the waiting lock time among the candidate waiting times increases. This indicates that at a relatively low proportion of rollback time, the increase in the waiting lock time has a relatively small impact on system performance. For the settings of RTR = 2.0 and RTR = 2.5, the transaction execution time increases significantly after the proportion of the waiting lock time among the candidate waiting times increases to 5, especially more obvious in the case of RTR = 2.5. This shows that at a relatively high proportion of rollback time, the waiting lock time has a greater impact on transaction execution, probably due to the increase in the frequency of transaction conflicts leading to an extension of the lock waiting time.

[0126] Similarly, set the proportion of the waiting lock time among the candidate waiting times corresponding to the shortest transaction execution time as the threshold of the waiting lock time proportion. For example, the shortest transaction execution time is Figure 6 the lowest point in the curve shown, and the corresponding proportion of the waiting lock time among the candidate waiting times is 4.5.

[0127] The above embodiments of the present invention disclose a distributed transaction processing method for hybrid concurrency control HCC. In order to evaluate the performance of MVOCC, D2PL, and hybrid concurrency control (HCC) protocols in different conflict environments, the following experimental process is provided:

[0128] The experiment runs Seata on a distributed Server cluster, and the basic operations are "read transactions" and "write transactions". In this experiment, three different database operation environments are set, namely read operation intensive, write operation intensive, and balanced read and write operations, to evaluate the performance of different concurrency control protocols under various read and write load conditions. The experiment is carried out on a server configured with 20 threads, simulating 200 transaction operations continuously initiated by different clients. In order to adapt to different read and write environments, the transaction types are adjusted accordingly according to the read and write ratio.

[0129] Specifically, the proportion of write transactions in the experiment is set to 20%, 50%, and 80%, corresponding to read-intensive, balanced read and write, and write-intensive environments respectively.

[0130] As Figure 7 shown, it is a schematic diagram of the transaction execution time in a read-intensive environment disclosed in the embodiments of the present invention.

[0131] In a read-intensive environment (the proportion of write transactions is 20%), the transaction execution time of the D2PL protocol is relatively long. This is mainly because the D2PL protocol needs to lock and release resources before and after transaction execution, and this process increases the transaction execution time. On the contrary, the hybrid concurrency control (HCC) protocol performs better. It mainly runs in the MVOCC mode. In this mode, due to fewer transaction conflicts in a read-intensive environment, it can handle read operations more efficiently, thus reducing the waiting and execution time.

[0132] AsFigure 8 As shown, it is a schematic diagram of the transaction execution time in a write-intensive environment disclosed in an embodiment of the present invention.

[0133] In a read-write balanced environment (with a 50% write transaction ratio), the performance of the HCC protocol remains optimal. This protocol can automatically select the most suitable concurrency control strategy according to the current read-write load. This flexibility enables HCC to optimize the response speed and improve system throughput when dealing with complex read-write operations, while ensuring data integrity and system reliability.

[0134] As Figure 9 shown, it is a schematic diagram of the transaction execution time in a read-write balanced set environment disclosed in an embodiment of the present invention.

[0135] In a read-write balanced environment (with a 50% write transaction ratio), the performance of the HCC protocol remains optimal. This protocol can automatically select the most suitable concurrency control strategy according to the current read-write load. This flexibility enables HCC to optimize the response speed and improve system throughput when dealing with complex read-write operations, while ensuring data integrity and system reliability.

[0136] In summary, due to its ability to flexibly switch the concurrency control mode according to different environments, the HCC protocol shows adaptability and superiority in various read-write environments. This experiment not only verifies the efficiency of the HCC protocol under various loads, but also emphasizes the importance of selecting a suitable concurrency control mechanism when designing a database system to adapt to different operation requirements and performance goals.

[0137] Based on the distributed transaction processing method based on hybrid concurrency control disclosed in the above embodiment of the present invention, in a distributed environment with a low conflict probability, MVOCC provides an effective concurrency control method, which not only maintains data consistency, but also significantly improves concurrency and system throughput. On the contrary, D2PL implements preventive measures by locking resources when transaction resource access conflicts are frequent to avoid conflicts, thereby improving the parallelism between transactions and the overall processing performance of the system. These two strategies show significant performance differences in different read-write modes. In this solution, by combining the hybrid concurrency control protocol of MVOCC (Multi-Version Optimistic Concurrency Control) and D2PL (Distributed Two-Phase Locking Protocol), the system can flexibly switch between the two main concurrency control mechanisms of optimistic concurrency control (OCC) and pessimistic concurrency control (PCC) to cope with different loads and conflict situations, optimize the system's response speed and throughput, thereby improving transaction processing efficiency, and ensuring system reliability in various load environments.

[0138] Corresponding to the distributed transaction processing method based on hybrid concurrency control disclosed in the above embodiment of the present invention, as Figure 10As shown in the figure, it is a structural diagram of a distributed transaction processing device based on hybrid concurrency control disclosed in an embodiment of the present invention. This device is applied to a distributed database system and includes: a determination unit 1001, a first calculation unit 1002, a first switching unit 1003, a second calculation unit 1004, and a second switching unit 1005.

[0139] The determination unit 1001 is used to determine the current operating mode of the distributed database system;

[0140] The first calculation unit 1002 is used to calculate the proportion of transaction rollback time when the current operating mode of the distributed database system is the multi-version optimistic concurrency control mode.

[0141] In one embodiment, the first calculation unit 1002 is specifically used for:

[0142] When the current operating mode of the distributed database system is the multi-version optimistic concurrency control mode, within a preset first time window, obtain the total first transaction processing time and the transaction rollback time; the transaction rollback time is the total time spent on transaction rollback due to transaction conflicts;

[0143] Based on the total first transaction processing time and the transaction rollback time, calculate the proportion of transaction rollback time.

[0144] The first switching unit 1003 is used to switch the operating mode of the distributed database system to the distributed two-phase locking protocol mode if the proportion of transaction rollback time exceeds a preset rollback time proportion threshold, so as to process distributed transactions using the distributed two-phase locking protocol.

[0145] The second calculation unit 1004 is used to calculate the proportion of transaction waiting lock time when the current operating mode of the distributed database system is the distributed two-phase locking protocol mode.

[0146] In one embodiment, the second calculation unit 1004 is specifically used for:

[0147] When the current operating mode of the distributed database system is the distributed two-phase locking protocol mode, within a preset second time window, obtain the total second transaction processing time and the transaction waiting lock time; the transaction waiting lock time is the total time spent on transaction waiting due to lock resource conflicts;

[0148] Based on the total second transaction processing time and the transaction waiting lock time, calculate the proportion of transaction waiting lock time.

[0149] A second switching unit 1005, configured to switch the operation mode of the distributed database system to a multi-version optimistic concurrency control mode if the ratio of the transaction waiting lock time to the total time is lower than a preset waiting lock time ratio threshold, so as to process distributed transactions by using the multi-version optimistic concurrency control method.

[0150] In one embodiment, the apparatus further includes:

[0151] A first setting unit, configured to set the distributed database system so that the ratio of the transaction rollback time of the distributed database system is fixed at a preset fixed value; perform transaction processing tests for each candidate rollback time ratio to obtain the transaction execution time corresponding to each candidate rollback time ratio; and set the candidate rollback time ratio corresponding to the shortest transaction execution time as the rollback time ratio threshold.

[0152] In one embodiment, the apparatus further includes:

[0153] A second setting unit, configured to set the distributed database system so that the ratio of the transaction waiting lock time of the distributed database system is successively fixed at multiple preset time fixed values; whenever the ratio of the transaction waiting lock time of the distributed database system is fixed at any time fixed value, perform transaction processing tests for each candidate rollback time ratio to obtain multiple transaction execution times corresponding to each candidate rollback time ratio; find the shortest transaction execution time among all the transaction execution times; and set the candidate rollback time ratio corresponding to the shortest transaction execution time as the rollback time ratio threshold.

[0154] In one embodiment, the apparatus further includes:

[0155] An initial operation unit, configured to run the multi-version optimistic concurrency control mode when in an initial state, so as to process distributed transactions by using the multi-version optimistic concurrency control method in the initial state.

[0156] Based on the distributed transaction processing device based on hybrid concurrency control disclosed in the above embodiments of the present invention, in a distributed environment with a low conflict probability, MVOCC provides an effective concurrency control method, which not only maintains data consistency, but also significantly improves concurrency and system throughput. On the contrary, D2PL implements preventive measures by locking resources when transaction resource access conflicts are frequent to avoid conflicts, thereby improving the parallelism between transactions and the overall processing performance of the system. These two strategies show significant performance differences under different read-write modes. In this solution, by combining the hybrid concurrency control protocol of MVOCC (Multi-Version Optimistic Concurrency Control) and D2PL (Distributed Two-Phase Locking Protocol), the system can flexibly switch between two main concurrency control mechanisms, optimistic concurrency control (OCC) and pessimistic concurrency control (PCC), to cope with different loads and conflict situations, optimize the system's response speed and throughput, thereby improving transaction processing efficiency, and at the same time ensure the reliability of the system under various load environments.

[0157] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for a system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative work.

[0158] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0159] The foregoing description of the disclosed embodiments enables those skilled in the art to practice or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A distributed transaction processing method based on hybrid concurrency control, characterized in that: Applied to a distributed database system, the method comprises: Determining a current operating mode of the distributed database system; When the current operation mode of the distributed database system is the multi-version optimistic concurrency control mode, the transaction rollback time ratio is calculated; If the transaction rollback time ratio exceeds a preset rollback time ratio threshold, the operation mode of the distributed database system is switched to a distributed two-phase locking protocol mode to process distributed transactions using a distributed two-phase locking protocol; When the current operation mode of the distributed database system is the distributed two-phase lock protocol mode, the proportion of transaction waiting time for lock is calculated; If the transaction wait lock time ratio is lower than the preset wait lock time ratio threshold, the operation mode of the distributed database system is switched to the multi-version optimistic concurrency control mode to process distributed transactions using the multi-version optimistic concurrency control method.

2. The method according to claim 1, characterized in that When the current operation mode of the distributed database system is the multi-version optimistic concurrency control mode, the transaction rollback time ratio is calculated, including: When the current operation mode of the distributed database system is the multi-version optimistic concurrency control mode, obtaining the total processing time of the first transaction and the transaction rollback time within the preset first time window; the transaction rollback time is the total time spent on transaction rollback due to transaction conflicts; Based on the total processing time of the first transaction and the transaction rollback time, a transaction rollback time ratio is calculated.

3. The method according to claim 1, characterized in that When the current operation mode of the distributed database system is the distributed two-phase lock protocol mode, calculating the transaction waiting lock time ratio includes: When the current operation mode of the distributed database system is the distributed two-phase lock protocol mode, obtaining the second transaction processing total time and the transaction waiting lock time within the preset second time window; the transaction waiting lock time is the total time spent on transaction waiting due to lock resource conflicts; Based on the second transaction processing total time and the transaction waiting lock time, the transaction waiting lock time ratio is calculated.

4. The method according to claim 1, characterized in that: The method further comprises: The distributed database system is configured so that the transaction rollback time ratio of the distributed database system is fixed at a preset fixed value; Perform a transaction processing test for each ratio of the time to be rolled back to be selected, and obtain the transaction execution time corresponding to each ratio of the time to be rolled back to be selected; The ratio of the to-be-selected rollback time corresponding to the shortest transaction execution time is set as the rollback time ratio threshold.

5. The method according to claim 1, characterized in that The method further comprises: The distributed database system is configured so that the proportion of the transaction waiting lock time of the distributed database system is fixed to a plurality of preset fixed time values ​​in sequence; Whenever the transaction waiting lock time ratio of the distributed database system is fixed at any of the fixed time values, a transaction processing test is performed for each to-be-selected rollback time ratio to obtain multiple transaction execution times corresponding to each to-be-selected rollback time ratio; Find the shortest transaction execution time among all the transaction execution times; The ratio of the to-be-selected rollback time corresponding to the shortest transaction execution time is set as the rollback time ratio threshold.

6. The method according to claim 1, characterized in that The method further comprises: The distributed database system is configured so that the transaction rollback time ratio of the distributed database system is fixed to a plurality of preset fixed ratio values ​​in sequence; Whenever the transaction rollback time ratio of the distributed database system is fixed at any of the fixed ratio values, a transaction processing test is performed for each waiting lock time ratio to be selected, and a plurality of transaction execution times corresponding to each waiting lock time ratio are obtained; Find the shortest transaction execution time among all the transaction execution times; The proportion of the waiting lock time to be selected corresponding to the shortest transaction execution time is set as the waiting lock time proportion threshold.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: When in the initial state, the multi-version optimistic concurrency control mode is run to process distributed transactions using the multi-version optimistic concurrency control method in the initial state.

8. A distributed transaction processing device based on hybrid concurrency control, characterized in that: Applied to a distributed database system, the device comprises: A determination unit, configured to determine a current operation mode of the distributed database system; A first calculation unit is used to calculate the transaction rollback time ratio when the current operation mode of the distributed database system is a multi-version optimistic concurrency control mode; A first switching unit is used to switch the operation mode of the distributed database system to a distributed two-phase locking protocol mode if the transaction rollback time ratio exceeds a preset rollback time ratio threshold, so as to process distributed transactions using the distributed two-phase locking protocol; A second calculation unit is used to calculate the proportion of transaction waiting time for lock when the current operation mode of the distributed database system is the distributed two-phase lock protocol mode; The second switching unit is used to switch the operation mode of the distributed database system to the multi-version optimistic concurrency control mode if the transaction waiting lock time ratio is lower than a preset waiting lock time ratio threshold, so as to process distributed transactions using the multi-version optimistic concurrency control method.

9. The device according to claim 8, characterized in that The first computing unit is specifically configured to: When the current operation mode of the distributed database system is a multi-version optimistic concurrency control mode, obtaining a first transaction processing total time and a transaction rollback time within a preset first time window; The transaction rollback time is the total time taken for transaction rollback due to transaction conflicts; Based on the total processing time of the first transaction and the transaction rollback time, a transaction rollback time ratio is calculated.

10. The device according to claim 8, characterized in that The second computing unit is specifically configured to: When the current operation mode of the distributed database system is the distributed two-phase lock protocol mode, obtaining the second transaction processing total time and the transaction waiting lock time within a preset second time window; The transaction waiting lock time is the total time spent on transaction waiting due to lock resource conflicts; Based on the second transaction processing total time and the transaction waiting lock time, the transaction waiting lock time ratio is calculated.

Images