A method, device, and storage medium for processing transactions

Abstract

The embodiments of the present application provide a method, device, and storage medium for processing transactions. For distributed transactions, it is innovatively proposed to add exclusive locks to the data records involved in the read set during the read phase; during the write phase, the write operations for the write set can be directly executed based on these exclusive locks, and the exclusive locks on the relevant data records are released immediately after the write operations are completed. The exclusive locks added to the data records during the write phase can maintain the exclusivity of the distributed transaction to these data records, which can avoid the problem of write operation abortion during the write phase. That is, all write operations of the distributed transaction will surely succeed, which ensures the atomicity of distributed transaction processing. In addition, since the exclusive locks are released immediately after the write operations are completed, the lock holding time of the distributed transaction is only the time required for executing its transaction logic inherently, which greatly shortens the lock holding time of the distributed transaction, thereby effectively improving the throughput of the distributed transaction and improving the transaction processing efficiency.

Description

Technical Field

[0001] This application relates to the field of database technologies, and in particular, to a method, device, and storage medium for processing transactions. Background Art

[0002] A database transaction is a set of indivisible SQL statements, and this set of SQL statements is a logical unit of work. Among them, a database transaction involving multiple shards in a distributed database can be called a distributed transaction. A distributed transaction can include multiple sub-transactions. Different sub-transactions may need to access different shards, and each sub-transaction may contain at least one statement logic.

[0003] An example of a distributed transaction is transferring money from one bank account to another. Usually, this involves two steps: one UPDATE statement is responsible for subtracting a certain amount of money from the total amount of one bank account, and another UPDATE statement is responsible for adding the corresponding amount of money to another bank account. These two operations of subtraction and addition must be permanently recorded in the relevant shards, otherwise the money will be lost. If there is a problem with the money transfer, both the subtraction and addition operations must be cancelled simultaneously. That is, the processing of a distributed transaction needs to ensure atomicity.

[0004] Currently, the two-phase commit protocol (2PC) is usually adopted to ensure the atomicity of distributed transactions. The first phase of the two-phase commit protocol is the voting phase. In this phase, all participants (shards) send feedback information on whether the current sub-transaction can be successfully executed to the coordinator (shard). The second phase is the commit phase. In this phase, the coordinator, based on the feedback information sent by all participants, notifies all participants to commit the current sub-transaction or roll back the current sub-transaction in unison. Therefore, the two-phase commit protocol needs to introduce multiple network round-trips in the transaction processing path, and the lock-holding time of the distributed transaction in the two-phase commit protocol is too long, resulting in poor throughput of the distributed transaction. Summary of the Invention

[0005] Multiple aspects of this application provide a method, device, and storage medium for processing transactions to improve the processing efficiency of transactions.

[0006] An embodiment of this application provides a method for processing transactions, which is applicable to a transaction coordinator. The method includes:

[0007] Receiving a processing request for a target transaction;

[0008] If it is determined that the target transaction is a distributed transaction, then exclusive locks are obtained for each data record involved in the read set of the target transaction during the read phase of the target transaction;

[0009] Calculate the write set of the target transaction;

[0010] Release the exclusive lock on the data records in the read set that do not require write operations;

[0011] In the write phase of the target transaction, after completing the write operation for the write set based on the exclusive lock, release the exclusive lock on the relevant data records.

[0012] The embodiments of the present application also provide a method for processing transactions, which is applicable to the transaction participating end. The method includes:

[0013] In the case where the target transaction is a distributed transaction, receive the remote read operation initiated by the transaction coordinator of the target transaction in the read phase of the target transaction;

[0014] Add an exclusive lock to the data records requested by the remote read operation in the read phase;

[0015] In the write phase of the target transaction, if receiving the remote write operation initiated by the transaction coordinator, respond to the remote write operation and release the exclusive lock on the corresponding data records.

[0016] The embodiments of the present application also provide a transaction coordinator, including a memory, a processor, and a communication component;

[0017] The memory is used to store one or more computer instructions;

[0018] The processor is coupled to the memory and the communication component, and is used to execute the one or more computer instructions for:

[0019] Receive a processing request for a target transaction through the communication component;

[0020] If it is determined that the target transaction is a distributed transaction, obtain exclusive locks for each data record involved in its read set in the read phase of the target transaction;

[0021] Calculate the write set of the target transaction;

[0022] Release the exclusive lock on the data records in the read set that do not require write operations;

[0023] In the write phase of the target transaction, after completing the write operation for the write set based on the exclusive lock, release the exclusive lock on the relevant data records.

[0024] The embodiments of the present application also provide a transaction participating end, including a memory, a processor, and a communication component;

[0025] The memory is used to store one or more computer instructions;

[0026] The processor is coupled to the memory and the communication component and is configured to execute the one or more computer instructions for:

[0027] In the case where the target transaction is a distributed transaction, receiving, via the communication component, a remote read operation initiated by the transaction coordinator of the target transaction during the read phase of the target transaction;

[0028] Adding an exclusive lock to the data record requested by the remote read operation during the read phase;

[0029] In the write phase of the target transaction, if a remote write operation initiated by the transaction coordinator is received, responding to the remote write operation and releasing the exclusive lock on the corresponding data record.

[0030] An embodiment of the present application further provides a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to execute the foregoing method for processing transactions.

[0031] In the embodiment of the present application, for a distributed transaction, it is innovatively proposed that an exclusive lock can be added to the data records involved in its read set respectively during the read phase; in the write phase, the write operation for the write set can be directly executed based on these exclusive locks, and the exclusive lock on the relevant data record is released immediately after the write operation is completed. The exclusive lock added to the data record during the write phase can maintain the exclusivity of the distributed transaction to these data records, which can avoid the problem of write operation abortion during the write phase. That is, all write operations of the distributed transaction will surely succeed, which ensures the atomicity of the distributed transaction processing. In addition, since the exclusive lock is released immediately after the write operation is completed, the lock holding time of the distributed transaction is only the time required for executing its transaction logic inherently, which greatly shortens the lock holding time of the distributed transaction, thereby effectively improving the throughput of the distributed transaction and improving the transaction processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0033] Figure 1 is a schematic structural diagram of a transaction processing system provided by an exemplary embodiment of the present application;

[0034] Figure 2 is a comparison schematic diagram between the traditional two-phase commit protocol and the technical solution of this embodiment provided by an exemplary embodiment of the present application;

[0035] Figure 3A schematic diagram of an exemplary application solution provided for an exemplary embodiment of the present application;

[0036] Figure 4 An application schematic diagram of a persistence check solution provided for an exemplary embodiment of the present application;

[0037] Figure 5 A flowchart of a method for processing a transaction provided for another exemplary embodiment of the present application;

[0038] Figure 6 A flowchart of another method for processing a transaction provided for another exemplary embodiment of the present application;

[0039] Figure 7 A schematic diagram of the structure of a transaction coordinator provided for yet another exemplary embodiment of the present application;

[0040] Figure 8 A schematic diagram of the structure of a transaction participant provided for yet another exemplary embodiment of the present application. Detailed implementation manners

[0041] To make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0042] Currently, the commonly used two-phase commit protocol needs to introduce multiple network round-trips in the transaction processing path, and the lock-holding time of distributed transactions in the two-phase commit protocol is too long, resulting in poor throughput of distributed transactions. Therefore, in some embodiments of the present application: for distributed transactions, exclusive locks can be added to the data records involved in their read sets respectively during the read phase; during the write phase, write operations on the write set can be directly executed based on these exclusive locks, and the exclusive locks on the relevant data records can be released immediately after the write operations are completed. The exclusive locks added to the data records during the write phase can maintain the exclusivity of the distributed transactions to these data records, which can avoid the problem of write operation abortion during the write phase. That is, all write operations of the distributed transaction will surely succeed, which ensures the atomicity of distributed transaction processing. In addition, since the exclusive locks are released immediately after the write operations are completed, the lock-holding time of the distributed transaction is only the time required for executing its transaction logic inherently, which greatly shortens the lock-holding time of the distributed transaction, thereby effectively improving the throughput of the distributed transaction and improving the transaction processing efficiency.

[0043] Before describing the technical solution, first briefly explain the technical concepts that may be involved:

[0044] Distributed system: The data in the system is stored in different local storage areas (which can be called shards), managed by different management systems, run on different machines, supported by different operating systems, and connected together by different communication networks.

[0045] Transaction: It can be composed of a finite sequence of operations, and the elements in the operation sequence are transaction logics one by one. Transactions can include local transactions and distributed transactions. In different application scenarios, the types of transactions can be different. For example, in the database scenario, it corresponds to database transactions.

[0046] Local transaction: It is limited to the access control of a single storage resource. For example, for a distributed database, a database transaction that only accesses a certain shard can be understood as a local transaction on that shard.

[0047] Distributed transaction: It means that the participants in the transaction, the servers supporting the transaction, the resource servers, and the transaction manager are located on different nodes of different distributed systems respectively. A distributed transaction can contain multiple statement logics, and the data records accessed by different statement logics may be distributed on incompletely the same shards.

[0048] Atomicity of distributed transaction: It can be understood that it is necessary to ensure that multiple statement logics in a distributed transaction are either all committed or all rolled back, and the distributed transaction needs to ensure data consistency on different shards.

[0049] Exclusive lock: Also known as X lock. If transaction T adds an exclusive lock to data record A, transaction T can read A and modify A, and other transactions cannot add any lock to A until T releases the exclusive lock on A.

[0050] Shared lock: Also known as S lock. If transaction T adds a shared lock to data record A, then transaction T can read A but cannot modify A. When reading A, the S lock needs to be upgraded to an X lock; other transactions can only add a shared lock to A and cannot add an exclusive lock until T releases the shared lock on A.

[0051] The following will describe in detail the technical solutions provided by the embodiments of the present application with reference to the accompanying drawings.

[0052] Figure 1 It is a schematic structural diagram of a transaction processing system provided for an exemplary embodiment of the present application. Refer to Figure 1, the system may include: a transaction coordinator and at least one transaction participant. Among them, the transaction coordinator and the transaction participant are defined based on work roles. For a distributed system, it contains multiple shards, and each shard can adaptively assume the work role of the coordinator or the participant for different transactions. For different transactions, the involved transaction coordinator and transaction participant may be different, and the number of involved shards may also be different. For example, a certain shard may be the transaction coordinator for transaction T1, but this shard may also be the transaction participant for transaction T2. In practical applications, the shard that has the first data record required to be read by the transaction can be used as the transaction coordinator for this transaction, and the other shards participating in the transaction processing work can be used as the transaction participants for this transaction. Of course, this is only exemplary, and this embodiment is not limited to this one work role definition scheme.

[0053] For a transaction, it can be executed according to the read phase and the write phase. The transaction processing solution provided in this embodiment aims to avoid abortion during the write phase to ensure that the distributed transaction can be necessarily committed after the read phase, thereby ensuring the atomicity of the distributed transaction processing, and abandoning the traditional two-phase commit scheme and proposing a single-phase commit scheme to shorten the lock-holding time of the distributed transaction, thereby improving the throughput.

[0054] For the convenience of description, in this embodiment, the target transaction will be used as an example to describe the implementation scheme of the transaction coordinator and the transaction participant in the process of collaborative processing of the target transaction. However, it should be understood that in addition to the target transaction, the same technical solution can also be used to process other transactions in the transaction processing system provided in this embodiment, supporting concurrent processing of transactions.

[0055] For this reason, refer to Figure 1, a processing request for a target transaction can be sent to the transaction coordinator side, where the processing request is initiated by the initiator of the target transaction. For the transaction coordinator, it can determine whether the target transaction is a distributed transaction. Generally, whether a transaction is a distributed transaction cannot be recognized before execution. Therefore, in an exemplary solution, the transaction coordinator side can run the statement logics of each statement in the target transaction during the read phase of the target transaction; if there is a remote read logic in the statement logics, it is determined that the target transaction is a distributed transaction. For example, the target transaction may contain 3 statement logics. The read logic in the first statement logic is a local read logic, that is, the local data records belonging to the transaction coordinator side need to be read. At this time, it is still impossible to determine whether the target transaction is a distributed transaction; continue to execute the second statement logic. The read logic in the second statement logic is a remote read logic, that is, the data records on a certain transaction participant side of the target transaction need to be read, rather than the local data records of the transaction coordinator side. Then it can be determined that the target transaction is a distributed transaction. Among them, each statement logic in the transaction corresponds to an SQL query statement. In a distributed transaction, the data records corresponding to different statement logics may be distributed on different shards that are not exactly the same.

[0056] If the target transaction is a distributed transaction, exclusive locks can be obtained for each data record involved in its read set during the read phase of the target transaction. That is, for each read operation of the target transaction, an exclusive lock will be generated on the corresponding data record, and the exclusive lock can ensure that the target transaction has the exclusive right to these data records.

[0057] During the process of obtaining exclusive locks for each data record involved in the read set, for the transaction coordinator side, it can determine the local data records of the read set that belong to the transaction coordinator side; add exclusive locks to the local data records. The transaction coordinator side can also trigger each transaction participant side to which each non-local data record in the read set belongs to add an exclusive lock to the corresponding data record. For this reason, the transaction coordinator side can initiate a remote read operation to the transaction participant side, and for the transaction participant side, it can add an exclusive lock to the data record requested by the remote read operation during the read phase to ensure the exclusivity of the target transaction to these data records. In this way, through the collaborative operation between the transaction coordinator side and the transaction participant side of the target transaction, exclusive locks can be obtained for each data record involved in the read set of the target transaction.

[0058] Figure 2 A comparison schematic diagram between the traditional two-phase commit protocol provided in an exemplary embodiment of the present application and the technical solution of this embodiment. Refer to Figure 2It can be clearly found that under the traditional two-phase protocol, the read phase of the distributed transaction is in the voting phase, and the shared lock is obtained for the data records involved in the read set of the distributed transaction in the read phase. The write phase of the distributed transaction is in the commit phase under the two-phase protocol. Figure 2 Under the two-phase protocol, at least two phases of network round trips are required to complete the execution of a distributed transaction. In the technical solution of this embodiment, a single-phase commit method is innovatively proposed. The read and write phases of a distributed transaction are both in this single phase, and in the read phase of a distributed transaction, an exclusive lock is obtained for the data records involved in its read set, rather than a shared lock. Figure 2 In the technical solution of this embodiment, only a single-stage network round trip is required to complete the processing of distributed transactions, which invisibly shortens the network round trip cost required for distributed transactions, thereby effectively improving the processing efficiency of distributed transactions.

[0059] In this embodiment, all statement logics of the target transaction can be run in the read phase of the target transaction, thereby generating a complete read set of the target transaction, and the read set of the target transaction can be cached in a specified data structure on the transaction coordination end. In addition, in the technical solution of this embodiment, the read logic and write logic in the target transaction are allowed to be interleaved, so that after running all statement logics of the target transaction in the read phase, the write set of the target transaction can also be generated, and can be cached in the transaction coordination end for the write phase to call. In this embodiment, we can assume that the read set of the target transaction can cover its write set, that is, the data records involved in the write set are part or all of the data records involved in the read set. In this case, the exclusive lock acquired for the read set can ensure the non-stop writing of the write set. For the situation that violates this assumption, we can call it blind writing, that is, the write set involves data records not involved in the read set. In this case, this embodiment proposes a relief plan. An exemplary relief plan can be: the transaction coordinator can check whether the read set of the target transaction covers the write set. If not, a virtual read can be performed for the data records involved in the write set but not involved in the read set, and an exclusive lock can be acquired for such data records. That is, in the read phase of the target transaction, such data records are not actually read, but only exclusive locks are acquired for such data records. If such data records are non-local data records that do not belong to the transaction coordination end, the operation of acquiring exclusive locks can be executed in batches with other normal remote read operations. In the absence of other normal remote read operations, additional round trips are required for such data records as the cost of violating the assumption.

[0060] In this way, in this embodiment, it can be ensured that each data record involved in the write set of the target transaction has obtained an exclusive lock in the read phase.

[0061] Based on this, for the transaction coordinator, in the write phase of the target transaction, after completing the write operation on the write set based on the exclusive lock, the exclusive lock on the relevant data record can be released. Here, the write operation refers to updating the data record according to the new value calculated for the data record in the write set. Here, the transaction coordinator can batch process the write set of the target transaction. However, it should be understood that the process of releasing the exclusive locks of each data record involved in the write set is independent of each other, without any dependency or synchronization relationship. After the write operation of a single data record is completed, the exclusive lock on it can be immediately released without relying on the lock release situation of other data records.

[0062] In the write phase of the target transaction, for the transaction coordinator, the local data records belonging to the transaction coordinator in the write set can be determined; after completing the write operation based on the exclusive lock on the local data records, release their exclusive locks. The transaction coordinator can also initiate write operations for each non-local data record in the write set to its respective transaction participant, so that the transaction participant releases the exclusive lock on the corresponding data record after responding to the write operation. For this purpose, the transaction coordinator can initiate a remote write operation to the transaction participant. For the transaction participant, if it receives a remote write operation initiated by the transaction coordinator, it can respond to the remote write operation and release the exclusive lock on the corresponding data record. In this way, whether it is a local write operation or a remote write operation, it can be ensured that the exclusive lock on the corresponding data record is immediately released after the write operation is completed.

[0063] It can be seen that the technical solution of this embodiment ensures that once a distributed transaction starts writing, it will not roll back by obtaining an exclusive lock in the read phase, and the exclusive lock will be immediately released after writing. In this way, the lock holding time of the distributed transaction is only the actual execution time of its transaction logic. Continuing to refer to Figure 2 , under the traditional two-phase protocol, the lock holding time of the distributed transaction T1 for the corresponding data record on the shard P1 is: read operation - result - write request - verification - consent - commit, which is very long. Looking at the technical solution of this embodiment again, the lock holding time of the distributed transaction T1 for the corresponding data record on the shard P1 is only: read operation - result - write operation, and the lock holding time is half less than that of the traditional two-phase protocol. And the reduction of the lock holding time of a single transaction can free up processing time for more other transactions. Therefore, the throughput of distributed transaction processing can be effectively improved.

[0064] As mentioned above, we assume that the read set of the target transaction can cover the write set. Therefore, there may be data records in the read set that do not require write operations to be performed. In this embodiment, the exclusive locks on these data records can be released in a timely manner to reduce the lock-holding time of the face transaction on these data records. In this embodiment, after it can be determined which data records in the read set do not require write operations, the exclusive locks on these data records can be released immediately. An exemplary release timing can be: in the write phase of the target transaction, if it is determined that any local data record in the read set does not require a write operation, the exclusive lock on it is released. In this embodiment, after entering the write phase, data records in the read set that do not require write operations can be discovered in a timely manner, and thus the exclusive locks on them can be released immediately. Of course, in this embodiment, the timing for releasing the exclusive locks on data records in the read set that do not require write operations is not limited to this, and it can also be set to other timings.

[0065] In this way, it can be ensured that the exclusive locks obtained for each data record involved in the read set during the read phase can be released in a timely manner, so that the lock-holding time of the distributed transaction on the data records is short enough.

[0066] Accordingly, in this embodiment, for a distributed transaction, exclusive locks can be added to the data records involved in its read set during the read phase; during the write phase, write operations on the write set can be directly performed based on these exclusive locks, and the exclusive locks on the relevant data records are released immediately after the write operations are completed. The exclusive locks added to the data records during the write phase can maintain the exclusivity of the distributed transaction to these data records, which can avoid the problem of write operation abortion during the write phase. That is, all write operations of the distributed transaction will surely succeed, which ensures the atomicity of the distributed transaction processing. In addition, since the exclusive locks are released immediately after the write operations are completed, the lock-holding time of the distributed transaction is only the time required for executing its transaction logic inherently, which greatly shortens the lock-holding time of the distributed transaction, and thus can effectively improve the throughput of the distributed transaction and improve the transaction processing efficiency.

[0067] The single-phase commit scheme proposed in this embodiment is very friendly to distributed transactions and can effectively avoid the write abortion problem in distributed transactions. However, the inventors found during the research process that the exclusive locks obtained in the single-phase commit scheme will prevent other transactions from accessing the data records, especially innocent local transactions, and the proportion of local transactions in all transactions is still quite considerable. The read phase of local transactions will be affected by the exclusive locks introduced in the single-phase commit scheme proposed in this embodiment.

[0068] To this end, in some preferred design solutions, a technical concept of adopting different processing modes for local transactions and distributed transactions is proposed. However, it should be understood that the processing methods for local transactions provided in these preferred solutions are optional. In this embodiment, of course, other processing methods can also be used to process local transactions, and even the same processing method as that for distributed transactions can be adopted. The following will detail the technical concept of adopting different processing modes for local transactions and distributed transactions proposed in the preferred design solutions.

[0069] Following the processing logic for determining whether the target transaction is a distributed transaction in the above text, here, if there is no remote read logic in the statement logics of each statement of the target transaction, it is determined that the target transaction is a local transaction; in the case where the target transaction is a local transaction, the read operation can be executed in a lock-free manner during the read phase of the target transaction to obtain the read set of the target transaction; in the write phase of the target transaction, an exclusive lock is obtained for each data record involved in its write set to execute the write operation.

[0070] In this way, for local transactions, the read operation can be executed in a lock-free manner, that is, in the read phase, no lock needs to be obtained for local transactions, nor does it need to be concerned whether there is a lock on the data record to be read. Instead, it can be defaulted that there is no lock on the data record for unobstructed reading. Therefore, executing the read operation in a lock-free manner can avoid the influence of the exclusive lock introduced in the distributed transaction processing on the read phase of local transactions.

[0071] In this embodiment, the Optimistic Concurrency Control (OCC) technology can be adopted to implement the read phase for local transactions. The implementation solutions adopted can include but are not limited to TICTOC, Multi-Version Concurrency Control (MVCC), etc.

[0072] Since the read phase of local transactions adopts a lock-free manner, it is possible that the relevant data records are updated during the period from the occurrence of the read operation of the local transaction read to the occurrence of the write operation, resulting in data inconsistency problems. To avoid this problem, in this embodiment, a verification phase can be added during the processing of local transactions, and the read set of local transactions can be verified for consistency in the verification phase. In this embodiment, the consistency verification of the read set of local transactions can be implemented based on timestamps. To this end, in this embodiment, it is proposed that timestamps can be introduced during the processing of distributed transactions to support the coordination between local transactions and distributed transactions. In this embodiment, the timestamps introduced for distributed transactions can include but are not limited to the logical timestamp of the transaction, the read timestamp of the data record, or the write timestamp of the data record, etc.

[0073] In addition, there is a special case here. After introducing the solution of performing read operations on local transactions in a lock-free manner, for a distributed transaction, the local read logic before the first remote read logic in the multiple statement logics it contains will be executed in a lock-free manner by default. This results in the lack of exclusive locks on the data records corresponding to this part of the local read logic. Therefore, in this embodiment, exclusive locks can be added retrospectively to this part of the local read logic. For example, there are 3 statement logics in the target transaction, and the first statement logic is a local read logic, which causes the read logic to be executed in a lock-free manner by default when the first statement logic is executed. Therefore, no locking operation is performed for the first statement logic, resulting in the lack of exclusive locks on the data records corresponding to the first statement logic. Then, we determine that the target transaction is a distributed transaction based on the second statement logic (remote read logic). To ensure that exclusive locks are obtained for each data record involved in the read set of the distributed transaction, the transaction coordinator can add an exclusive lock to the first statement logic after the second statement logic is executed. In addition, since the exclusive lock is added retrospectively to the data records corresponding to the first statement logic, it is necessary to perform a consistency verification on the data records corresponding to the first statement logic. That is, it is necessary to verify whether the data record(s) have been updated after the read operation of the first statement logic is executed. If there is no update, the target transaction can be processed normally; if there is an update, the target transaction needs to be aborted or retried to avoid incorrect processing results of the target transaction due to the update of the data records corresponding to the first statement logic.

[0074] In this embodiment, the transaction coordinator of the target transaction can assign a logical timestamp to the target transaction during the write phase of the target transaction. The logical timestamp is assigned based on the execution order between transactions. That is, the logical timestamp is used to represent the execution order between transactions, and its essence can be understood as a kind of number. In this embodiment, during the process of assigning the logical timestamp, the transaction coordinator needs to follow certain assignment rules. The assignment rules can include, but are not limited to, that the logical timestamp is greater than the maximum value of the read timestamp and the write timestamp in each data record in the read set, etc. This can ensure that the logical timestamp assigned to the transaction is monotonically increasing. It should be understood here that when assigning the logical timestamp, there is no need to distinguish between local transactions and distributed transactions, but the two types of transactions are mixed together for assignment.

[0075] Based on the logical timestamp of the target transaction, the transaction coordinator can configure the read timestamp and write timestamp corresponding to its local data record during the write phase. For this purpose, in this embodiment, if the read timestamp of the local data record belonging to the transaction coordinator in the read set is less than the logical timestamp of the target transaction, the transaction coordinator can update the read timestamp of the local data record to the logical timestamp of the target transaction; the transaction coordinator can also update the read timestamp and write timestamp of the local data record belonging to the transaction coordinator in the write set to the logical timestamp of the target transaction. This can ensure the freshness of the read timestamp and write timestamp of the local data record belonging to the transaction coordinator in the read set and write set. In addition, the transaction coordinator can also send the logical timestamp of the target transaction to each transaction participant involved in the write set. Then, for the transaction participants of the target transaction, they can obtain the logical timestamp assigned by the transaction coordinator for the target transaction; during the read phase of the target transaction, if the read timestamp of the data record corresponding to the remote read operation initiated by the transaction coordinator is less than the logical timestamp of the target transaction, update the read timestamp of the data record corresponding to the remote read operation to the logical timestamp of the target transaction; during the write phase of the target transaction, update the read timestamp and write timestamp of the data record corresponding to the remote write operation initiated by the transaction coordinator to the logical timestamp of the target transaction. In this way, the freshness of the read timestamp and write timestamp of the relevant data records on the transaction participants of the target transaction can be ensured.

[0076] Based on this, the read timestamp and write timestamp generated by the distributed transaction for the data record can be used as the basis for consistency verification of the local transaction. In this way, if it is determined that the target transaction is a local transaction, the transaction coordinator can perform consistency verification based on the read timestamp and / or write timestamp of the local data record belonging to the transaction coordinator in its read set. Similarly, for the transaction participants of the target transaction, if the transaction participant is the transaction coordinator of other transactions, then in the case where other transactions are local transactions, consistency verification can be performed based on the read timestamp and / or write timestamp of the local data record in the read set of the other transaction. Thus, it can be timely discovered whether there are consistency problems in the read set of the local transaction. If so, the local transaction can be aborted or retried.

[0077] In addition, during the research process, the inventor found that a deadlock problem may occur in the technical solution of this embodiment. The deadlock problem can be understood as that two distributed transactions both need to wait for the other party to release a certain lock before they can continue to execute. To address the deadlock problem, this embodiment provides an exemplary solution: if a deadlock problem occurs on any data record during the read phase of the target transaction, it is determined whether the start sequence code of the current transaction on this data record is less than the start sequence code of the target transaction. If so, the target transaction is aborted or retried. Similarly, if not, the lock on this data record can be waited for to be released to continue processing the target transaction. This is because the current transaction on this data record will be aborted or retried because its start sequence code is less than that of the target transaction, thereby releasing the lock on this data record and removing the execution obstacle of the target transaction. Among them, the start sequence code is allocated by the transaction coordinator according to the transaction start requests it receives. The earlier the start time of the transaction, the smaller the start sequence code can be allocated. The start sequence code is unique. Here, a logical timestamp is not used to handle the deadlock problem mainly because the logical timestamp is only allocated during the write phase, while the deadlock problem occurs as early as during the read phase. It should be understood that in this embodiment, the deadlock problem may only occur during the read phase because the distributed transaction can be written without interruption based on the exclusive lock during the write phase, and the local transaction can also be written by adding an exclusive lock during the write phase. Therefore, no deadlock problem will occur during the write phase.

[0078] In this embodiment, we can regard the single-phase commit scheme proposed for distributed transactions as the distributed mode, and regard the scheme of performing write operations in a lock-free manner for local transactions as the local mode. Accordingly, different processing modes can be adopted for local transactions and distributed transactions to alleviate the impact of the exclusive lock introduced in the distributed transaction on the read phase of the local transaction, thereby ensuring the processing efficiency of the local transaction. In addition, by introducing a timestamp during the processing of the distributed transaction, it is possible to support the local transaction to perform a consistency verification on its read set, thereby supporting the coordination between the local transaction and the distributed transaction.

[0079] Figure 3 It is a schematic diagram of an exemplary application solution provided for an exemplary embodiment of the present application. The following will be combined with Figure 3 to illustrate the transaction processing logic provided in this embodiment.

[0080] Refer to Figure 3 , transaction T reads x, y, and z and needs to update y and z. Among them, x, y, and z are respectively on shards 1, 2, and 3. Shard 1 is the transaction coordinator of T, and shards 2 and 3 are the transaction participants of T:

[0081] In the read phase of T, shard 1 first executes T in local mode (TicToc), reads x in a lock-free manner, where the wts (write timestamp) of x = 1 and the rts (read timestamp) = 3;

[0082] When T attempts to perform a remote read, it switches to distributed mode and reacquires the exclusive lock on x;

[0083] Perform consistency verification on x. If x has been updated, directly abort and retry transaction T in distributed mode; otherwise, successfully enter distributed mode and remotely read y and z;

[0084] Shard 2 and shard 3 respectively acquire the corresponding exclusive locks and return read results containing the wts and rts of the data records to form the read set of T; refer to Figure 3 , the wts and rts of the returned data record y are [1, 3], and the wts and rts of the data record z are [2, 4];

[0085] According to the transaction logic, T calculates its write set (which is {y = 10, z = 20} in Figure 3 ), and then enters the write phase;

[0086] In the write phase of T, shard 1 assigns a logical timestamp Ts for T, which should be greater than the wts of x and the rts of y and z. The result is Ts = 5; since the rts of x < Ts, the rts of x can be updated to 5;

[0087] After that, shard 1 unlocks x and sends a remote write for y with Ts = 5 to shard 2, and sends a remote write for z with Ts = 5 to shard 3;

[0088] When shard 2 receives the remote write for y, it updates the value of y to 10, initializes the rts = wts = 5 of y, and unlocks y;

[0089] When shard 3 receives the remote write for z, it updates the value of z to 20, initializes the rts = wts = 5 of z, and unlocks z.

[0090] The above exemplarily presents the transaction processing process in the case where transaction T is a distributed transaction. The various timestamps configured in the distributed transaction processing process can support the processing processes of other local transactions. It can be seen that the lock-holding time of the distributed transaction on the above data records x, y, and z has been minimized. Combining with the scheme of performing write operations on local transactions in a lock-free manner proposed in the foregoing embodiments, the impact of the distributed transaction on local transactions can be effectively improved. Therefore, overall, the throughput of transaction processing can be effectively increased.

[0091] As mentioned above, in the single-phase commit scheme provided by this embodiment for distributed transactions, the distributed transaction releases the relevant exclusive lock immediately after executing the write operation, without waiting for its log record to be persisted. To this end, in this embodiment, a persistence check link for distributed transactions is also proposed. An exemplary persistence check scheme can be: based on the aforementioned logical timestamp assigned to the transaction, check whether the target transaction has completed persistence on its transaction coordination end and transaction participating end. Wherein, persistence means that for the target transaction, the log manager on its transaction coordination end and transaction participating end will automatically persist the log record corresponding to the target transaction to the hard disk, and only after the persistence of the log record is completed, the target transaction is considered to have completed global persistence. However, due to the different working efficiency of the log managers on different shards, the progress of the persistence work of the target transaction on different shards is different. Therefore, it is necessary to check whether the target transaction has completed global persistence on all shards involved.

[0092] In this exemplary solution, taking the transaction coordination end of the target transaction as an example, the minimum logical timestamp in the non-persistent transaction on the transaction coordination end can be determined based on the logical timestamp corresponding to each transaction processed on the transaction coordination end, as the shard timestamp on the transaction coordination end; the shard timestamp provided by other shards is obtained; the minimum value of the shard timestamps corresponding to all shards is used as the global timestamp; if the logical timestamp of the target transaction is less than the global timestamp, it is determined that the target transaction has completed the global persistence processing. Here, each shard in the distributed system can determine its own allocated timestamp, obtain the shard timestamps provided by all other allocations, and calculate its own global timestamp. It is worth noting that due to different transactions processed, different processing progress, etc., the global timestamps calculated by different shards in the distributed system may not be the same, which is normal.

[0093] For a single shard, the minimum logical timestamp in the local non-persistent transaction can be checked regularly to generate a shard timestamp. When the shard timestamp on a shard changes, a new shard timestamp can be provided to other shards. In this way, when the shard timestamp itself changes or receives a new shard timestamp provided by other shards, the shard can be triggered to recalculate its own global timestamp, thereby ensuring the freshness of the global timestamp on each shard.

[0094] As mentioned above, the logical timestamp can be used to characterize the execution order between transactions. In this embodiment, the shard timestamp is the threshold of the logical timestamp, so the shard timestamp is monotonic. This monotonicity can ensure that the global timestamp calculated by each shard can accurately reflect the transaction persistence status on each shard of the distributed system, thereby ensuring that it can accurately determine whether the target transaction has completed global persistence.

[0095] However, there may be two problems in the process of persistence check based on logical timestamp: the first is that some non-persistent transactions may not have been assigned a logical timestamp, resulting in the inability to determine the minimum logical timestamp for the shard where they are located; the second is that after the shard timestamp is calculated on the shard, the new transaction on the shard may be assigned a logical timestamp that is smaller than the shard timestamp, which destroys the monotonicity of the shard timestamp.

[0096] For the first problem, still taking the transaction coordination end of the target transaction as an example, an exemplary solution may be: if there is a special transaction on the transaction coordination end of the target transaction that is not assigned a logical timestamp, the write timestamp on the first data record accessed by the special transaction is used as the logical timestamp lower limit of the special transaction, and the logical timestamp lower limit replaces the logical timestamp as the basis for determining the shard timestamp. Of course, this exemplary solution may also be used for other shards in the distributed system to solve the first problem mentioned above.

[0097] For the second problem, in an exemplary solution, the logical timestamp assigned to the new transaction can be forced to be greater than the shard timestamp on the shard where it is located. To this end, we consider two situations, still taking the transaction coordination end of the target transaction as an example: in one case, after determining the shard timestamp on the transaction coordination end, if the transaction coordination end subsequently serves as the coordination end of the new transaction, the logical timestamp assigned to the new transaction is not less than the shard timestamp; in another case, if the transaction coordination end subsequently serves as the participating end of the new transaction, the write timestamp of the data record carried by it in the new transaction write set is updated to be no less than its own shard timestamp, so as to drive the coordination end of the new transaction to assign a logical timestamp no less than the shard timestamp to the new transaction. This echoes the logic of assigning logical timestamps to transactions in the previous text, that is, when assigning logical timestamps to transaction shards, it is necessary to ensure that the assigned logical timestamp is greater than the write timestamp of the data record in the read set. Therefore, here, the logical timestamp assigned to the new transaction is indirectly affected by reasonably maintaining the write timestamp of the data record involved in the new transaction.

[0098] It can be seen that the solutions proposed under the above two problems can ensure the monotonicity of the shard timestamps calculated on each shard. This ensures the rationality of the shard timestamps, and further ensures the rationality of the global timestamps.

[0099] Figure 4 The application schematic diagram of a persistence check scheme provided for an exemplary embodiment of the present application. Refer to Figure 4 , on shards P1, P2, and P3 in the distributed system, a latest shard timestamp table is maintained respectively. The table records the latest shard timestamps provided by shards P1, P2, and P3. The tables on each shard can be continuously updated, and the global timestamp is calculated by obtaining the smallest number in the table. For example, after partition P1 receives the shard timestamp Wp2 = 5 provided by shard P2 and the shard timestamp Wp3 = 6 provided by shard P3, and combines its own shard timestamp Wp1 = 4, P1 calculates the global timestamp Wg = 4. The shard timestamps do not need to be broadcast synchronously (for example, P3 may broadcast Wp3 = 14 later than other shards), and messages are also allowed to be delayed. Nevertheless, each shard does not block the waiting messages from other shards because each of them processes and retains transactions independently and only communicates the "shard timestamp" asynchronously.

[0100] Here, for the target transaction, only its transaction coordinator determines whether the logical timestamp of the target transaction is less than the global timestamp calculated by the transaction coordinator. If so, the transaction coordinator can determine that the target transaction has completed the global persistence process, and it is no longer necessary for the transaction participant of the target transaction to perform repeated judgments. However, the transaction participant of the target transaction also calculates the global timestamp it believes, because the transaction participant of the target transaction may also act as the transaction coordinator of other transactions, and the global timestamp it calculates needs to be used as the basis for the persistence check process for such other transactions.

[0101] It can be seen that in this embodiment, each shard in the distributed system can calculate the global timestamp independently, and only requires each shard to broadcast the timestamp of its latest persisted transaction regularly. In this way, each shard can learn the shard timestamps of other shards and calculate the global timestamp, that is, the minimum value among the shard timestamps of all shards. As the timestamp monotonically increases, it can be ensured that transactions with timestamps lower than the global timestamp have been persisted on all shards. Generally speaking, by abandoning the persistence check logic during the lock-holding period of the distributed transaction and implementing the persistence check through an additional separate process, the lock-holding time of the distributed transaction can be effectively reduced. Moreover, by allowing the shards to submit independently in groups, through this supplemented group submission method, the global coordination link can be avoided, and the cost of the persistence verification process can be reduced, which can effectively improve the scalability of the distributed system. In the case of the expansion of the distributed system, the cost of the persistence verification process will not increase due to the addition of new shards.

[0102] In addition, during the research process, the inventors found that local failure problems may occur in a distributed system, that is, failures such as deadlocks may occur on some shards. In this embodiment, it is proposed that the fault recovery problem in the distributed system can be solved based on the aforementioned global timestamp. An exemplary solution can be: in the case of a shard failure, obtain a recovery timestamp, where the recovery timestamp is the maximum value among the global timestamps determined on all shards; roll back the transactions processed after the recovery timestamp. The reason for choosing the minimum value among the global timestamps determined on all shards as the recovery timestamp is that each shard independently maintains its own perspective of the global timestamp during normal operation, and all global timestamps can be valid candidates; in order to reduce the number of transactions to be rolled back, the largest one is adopted as the agreed recovery timestamp for all partitions. In this exemplary solution, the consensus mechanism in Zookeeper can be used to ensure that all partitions adopt the same recovery timestamp. Then, transactions with a logical timestamp Ts ≥ recovery timestamp can be safely rolled back. For the shard that has failed, after recovery, it rejoins the Zookeeper cluster to obtain the agreed recovery timestamp, and then repersists all transactions with Ts < recovery timestamp and rolls back transactions with Ts ≥ recovery timestamp. To limit the recovery time, each shard periodically saves a checkpoint of its state, so that it only needs to be repersisted after the checkpoint to recover.

[0103] In this embodiment, some typical problems in the transaction processing process may also be involved, including but not limited to the processing problem of read-only transactions, the processing problem of the read link for a data range, the constraint check problem, etc. The solutions to these problems may intersect with the technical solutions of this embodiment. Here, only a brief exemplary description is given without further elaboration. For example, for the processing problem of read-only transactions, an exemplary solution can be: when the transaction coordinator of the target transaction determines that it is a read-only transaction according to its transaction logic, a read-only flag can be configured for the target transaction, and the read link of the target transaction can be executed in a lock-free manner. Technologies such as snapshots can be used to optimize read-only transactions to achieve lock-free reading. Another example, for the processing problem of the read link for a data range, an exemplary solution can be: if the target transaction is a distributed transaction and its read link involves a read operation on a data range, its transaction coordinator can obtain an exclusive range lock (predicate locks) for the corresponding data range during the read link of the target transaction to ensure the exclusivity of the target transaction to this data range.

[0104] It should be understood that the typical problems in the transaction processing process are not limited to this, and there may be more problems. The solutions to these problems can be adapted and cooperate with the technical solutions provided in this embodiment. Here, these typical problems are not enumerated exhaustively.

[0105] Figure 5 A schematic flowchart of a method for processing a transaction provided in another exemplary embodiment of the present application. The method can be executed by a data processing device, which can be implemented as a combination of software and / or hardware, and the data processing device can be integrated in the transaction coordination end of the transaction. Refer to Figure 5 , the method may include:

[0106] Step 500, receiving a processing request for a target transaction;

[0107] Step 501, if it is determined that the target transaction is a distributed transaction, obtain exclusive locks for each data record involved in the read set during the read phase of the target transaction;

[0108] Step 502, calculating the write set of the target transaction;

[0109] Step 503, releasing the exclusive locks on the data records in the read set that do not need to perform write operations;

[0110] Step 504, after completing the write operation on the write set based on the exclusive lock during the write phase of the target transaction, release the exclusive locks on the relevant data records.

[0111] In an optional embodiment, the step of obtaining exclusive locks for each data record involved in the read set during the read phase of the target transaction includes:

[0112] Determine the local data records in the read set that belong to the transaction coordination end;

[0113] Add exclusive locks to the local data records;

[0114] Trigger each transaction participant end to which each non-local data record in the read set belongs to add an exclusive lock to the corresponding data record.

[0115] In an optional embodiment, the step of releasing the exclusive locks on the data records in the read set that do not need to perform write operations includes:

[0116] During the write phase of the target transaction, if it is determined that any local data record in the read set does not need to perform a write operation, release the exclusive lock on it.

[0117] In an optional embodiment, the method may further include:

[0118] During the read phase of the target transaction, run the statement logic in the target transaction;

[0119] If there is remote read logic in each statement logic, determine that the target transaction is a distributed transaction.

[0120] In an optional embodiment, the method further includes:

[0121] If there is no remote read logic in the logic of each statement, determine that the target transaction is a local transaction;

[0122] When the target transaction is a local transaction, perform a read operation in a lock-free manner during the read phase of the target transaction to obtain the read set of the target transaction;

[0123] During the write phase of the target transaction, obtain an exclusive lock for each data record involved in its write set to perform the write operation.

[0124] In an alternative embodiment, the step of calculating the write set of the target transaction includes:

[0125] During the read phase of the target transaction, calculate the write set of the target transaction;

[0126] Cache the write set locally at the transaction coordinator for use in the write phase.

[0127] In an alternative embodiment, the step of, after completing the write operation on the write set based on the exclusive lock during the write phase of the target transaction, releasing the exclusive lock on the relevant data record includes:

[0128] During the write phase of the target transaction, determine the local data records of the transaction coordinator that belong to the write set;

[0129] After completing the write operation based on the exclusive lock on the local data record, release its exclusive lock;

[0130] Initiate a write operation for each non-local data record in the write set to its respective transaction participant, so that the transaction participant releases the exclusive lock on the corresponding data record after responding to the write operation.

[0131] In an alternative embodiment, the method further includes:

[0132] During the write phase of the target transaction, assign a logical timestamp to the target transaction, and the logical timestamp is assigned based on the execution order between transactions;

[0133] If the read timestamp of the local data record of the transaction coordinator in the read set is less than the logical timestamp of the target transaction, update the read timestamp of the local data record to the logical timestamp of the target transaction;

[0134] Update the read timestamp and write timestamp of the local data record of the transaction coordinator in the write set to the logical timestamp of the target transaction.

[0135] In an alternative embodiment, the method further includes:

[0136] Send the logical timestamp of the target transaction to the transaction participating ends involved in the write set, so that the transaction participating ends can update the read timestamp and / or write timestamp of the data records carried in their read sets and / or write sets based on the logical timestamp of the target transaction.

[0137] In an optional embodiment, the method further includes:

[0138] If it is determined that the target transaction is a local transaction, perform consistency verification based on the read timestamp and / or write timestamp of the local data records belonging to the transaction coordinator in its read set.

[0139] In an optional embodiment, the method further includes:

[0140] Based on the logical timestamps respectively corresponding to the transactions processed on the transaction coordinator, determine the minimum logical timestamp among the unpersisted transactions on the transaction coordinator as the shard timestamp on the transaction coordinator;

[0141] Obtain the shard timestamps provided by other shards;

[0142] Take the minimum value among the shard timestamps respectively corresponding to all shards as the global timestamp;

[0143] If the logical timestamp of the target transaction is less than the global timestamp, determine that the target transaction has completed global persistence processing.

[0144] In an optional embodiment, the method further includes:

[0145] If there is a special transaction on the transaction coordinator for which no logical timestamp is assigned, use the write timestamp of the data record first accessed by the special transaction as the lower limit of the logical timestamp of the special transaction, and the lower limit of the logical timestamp is used as the basis for determining the shard timestamp instead of the logical timestamp.

[0146] In an optional embodiment, the method further includes:

[0147] After determining the shard timestamp on the transaction coordinator, if the transaction coordinator subsequently serves as the coordinator of a new transaction, the logical timestamp assigned to the new transaction by it is not less than the shard timestamp;

[0148] If the transaction coordinator subsequently serves as a participating end of a new transaction, update the write timestamp of the data records carried in its write set of the new transaction to be not less than its own shard timestamp, so as to drive the coordinator of the new transaction to assign a logical timestamp not less than the shard timestamp to the new transaction.

[0149] In an optional embodiment, the method further includes:

[0150] In the event of a shard failure, obtain a recovery timestamp, which is the maximum value among the global timestamps determined on all shards;

[0151] Roll back the transactions processed after the recovery timestamp.

[0152] It should be noted that for the technical details in the above embodiments of the transaction processing method, reference can be made to the relevant descriptions of the transaction coordinator in the foregoing system embodiments. To save space, they will not be elaborated here, but this should not cause a loss of the protection scope of this application.

[0153] Figure 6 The following is a flowchart of another transaction processing method provided by another exemplary embodiment of this application. This method can be executed by a data processing device, which can be implemented as a combination of software and / or hardware, and can be integrated into the transaction participant of the transaction. Refer to Figure 6 , this method may include:

[0154] Step 600: When the target transaction is a distributed transaction, receive a remote read operation initiated by the transaction coordinator of the target transaction during the read phase of the target transaction;

[0155] Step 601: Add an exclusive lock to the data record requested by the remote read operation during the read phase;

[0156] Step 602: In the write phase of the target transaction, if a remote write operation initiated by the transaction coordinator is received, respond to the remote write operation and release the exclusive lock on the corresponding data record.

[0157] In an optional embodiment, this method further includes:

[0158] Obtain the logical timestamp assigned by the transaction coordinator for the target transaction. The logical timestamp is assigned based on the execution order between transactions;

[0159] During the read phase, if the read timestamp of the data record corresponding to the remote read operation is less than the logical timestamp of the target transaction, update the read timestamp of the data record corresponding to the remote read operation to the logical timestamp of the target transaction;

[0160] During the write phase, update the read timestamp and write timestamp of the data record corresponding to the remote write operation to the logical timestamp of the target transaction.

[0161] In an optional embodiment, this method further includes:

[0162] If the transaction participant serves as the transaction coordinator of other transactions, in the case where other transactions are local transactions, perform consistency verification based on the read timestamp and / or write timestamp of the local data records in its read set.

[0163] In an alternative embodiment, the method further includes:

[0164] Based on the logical timestamps corresponding to the respective transactions processed on the transaction participating end, determine the minimum logical timestamp among the unpersisted transactions on the transaction participating end as the shard timestamp on the transaction participating end;

[0165] Provide its own shard timestamp to the transaction coordinator of the target transaction, so that the transaction coordinator can use the minimum value among the shard timestamps corresponding to all shards as the global timestamp, and determine that the target transaction has completed global persistence processing when the logical timestamp of the target transaction is less than the global timestamp.

[0166] In an alternative embodiment, the method further includes:

[0167] After determining the shard timestamp, if the transaction participating end subsequently serves as the coordinator of a new transaction, the logical timestamp assigned to it is not less than the shard timestamp;

[0168] If the transaction participating end subsequently serves as a participant of a new transaction, update the timestamps of the data records carried in its write set for the new transaction to be not less than the shard timestamp, so as to drive the coordinator of the new transaction to assign a logical timestamp not less than the assigned timestamp to the new transaction.

[0169] It should be noted that for the technical details in the above embodiments of the transaction processing method, reference can be made to the relevant descriptions of the transaction participating end in the foregoing system embodiments. To save space, they are not elaborated here, but this should not cause loss of the protection scope of this application.

[0170] In addition, in some of the processes described in the above embodiments and the accompanying drawings, a plurality of operations appear in a specific order. However, it should be clearly understood that these operations can be executed not in the order in which they appear in this document or in parallel. The operation numbers such as 501, 502, etc. are only used to distinguish different operations, and the numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations can be executed sequentially or in parallel.

[0171] Figure 7 FIG. [ID] is a schematic structural diagram of a transaction coordinator provided for another exemplary embodiment of the present application. As Figure 7 shown, the transaction coordinator may include: a memory 70, a processor 71, and a communication component 72.

[0172] The processor 71 is coupled to the memory 70 and the communication component 72 and is configured to execute a computer program in the memory 70 for:

[0173] Receive a processing request for a target transaction through the communication component 72;

[0174] If it is determined that the target transaction is a distributed transaction, obtain exclusive locks for each data record involved in the read set during the read phase of the target transaction;

[0175] Calculate the write set of the target transaction;

[0176] Release the exclusive locks on the data records in the read set that do not need to perform write operations;

[0177] In the write phase of the target transaction, after completing the write operation on the write set based on the exclusive lock, release the exclusive locks on the relevant data records.

[0178] In an optional embodiment, during the process of the processor 71 obtaining exclusive locks for each data record involved in the read set during the read phase of the target transaction, it can be used for:

[0179] Determine the local data records in the read set that belong to the transaction coordinator;

[0180] Add exclusive locks to the local data records;

[0181] Trigger each transaction participant end to which each non-local data record in the read set belongs to add an exclusive lock to the corresponding data record.

[0182] In an optional embodiment, during the process of the processor 71 releasing the exclusive locks on the data records in the read set that do not need to perform write operations, it can be used for:

[0183] In the write phase of the target transaction, if it is determined that any local data record in the read set does not need to perform a write operation, release the exclusive lock on it.

[0184] In an optional embodiment, the processor 71 can also be used for:

[0185] During the read phase of the target transaction, run the statement logics in the target transaction;

[0186] If there is remote read logic in the statement logics, determine that the target transaction is a distributed transaction.

[0187] In an optional embodiment, the processor 71 can also be used for:

[0188] If there is no remote read logic in the statement logics, determine that the target transaction is a local transaction;

[0189] In the case where the target transaction is a local transaction, perform a read operation in a lock-free manner during the read phase of the target transaction to obtain the read set of the target transaction;

[0190] In the write phase of the target transaction, exclusive locks are acquired for each data record involved in its write set to perform the write operation.

[0191] In an alternative embodiment, during the process of the processor 71 calculating the write set of the target transaction, it can be used for:

[0192] In the read phase of the target transaction, calculate the write set of the target transaction;

[0193] Cache the write set locally at the transaction coordinator for the write phase to call.

[0194] In an alternative embodiment, during the process of the processor 71 releasing the exclusive lock on the relevant data record after completing the write operation on the write set based on the exclusive lock in the write phase of the target transaction, it can be used for:

[0195] In the write phase of the target transaction, determine the local data records of the transaction coordinator in the write set;

[0196] After completing the write operation based on the exclusive lock on the local data record, release its exclusive lock;

[0197] Initiate a write operation for each non-local data record in the write set to its respective transaction participant, so that the transaction participant releases the exclusive lock on the corresponding data record after responding to the write operation.

[0198] In an alternative embodiment, the processor 71 can also be used for:

[0199] In the write phase of the target transaction, assign a logical timestamp to the target transaction, and the logical timestamp is assigned based on the execution order between transactions;

[0200] If the read timestamp of the local data record of the transaction coordinator in the read set is less than the logical timestamp of the target transaction, update the read timestamp of the local data record to the logical timestamp of the target transaction;

[0201] Update the read timestamp and write timestamp of the local data record of the transaction coordinator in the write set to the logical timestamp of the target transaction.

[0202] In an alternative embodiment, the processor 71 can also be used for:

[0203] Send the logical timestamp of the target transaction to the transaction participants involved in the write set for the transaction participants to update the read timestamp and / or write timestamp of the data records carried in their read set and / or write set based on the logical timestamp of the target transaction.

[0204] In an alternative embodiment, the processor 71 can also be used for:

[0205] If it is determined that the target transaction is a local transaction, consistency verification is performed based on the read timestamp and / or write timestamp of the local data records belonging to the transaction coordinator in its read set.

[0206] In an optional embodiment, the processor 71 may further be configured to:

[0207] Based on the logical timestamps corresponding to the respective transactions processed on the transaction coordinator, determine the minimum logical timestamp among the unpersisted transactions on the transaction coordinator as the shard timestamp on the transaction coordinator;

[0208] Obtain the shard timestamps provided by other shards;

[0209] Take the minimum value among the shard timestamps corresponding to all shards as the global timestamp;

[0210] If the logical timestamp of the target transaction is less than the global timestamp, it is determined that the target transaction has completed global persistence processing.

[0211] In an optional embodiment, the processor 71 may further be configured to:

[0212] If there is a special transaction on the transaction coordinator for which no logical timestamp is assigned, take the write timestamp of the data record first accessed by the special transaction as the lower limit of the logical timestamp of the special transaction, and the lower limit of the logical timestamp is used instead of the logical timestamp as the basis for determining the shard timestamp.

[0213] In an optional embodiment, the processor 71 may further be configured to:

[0214] After determining the shard timestamp on the transaction coordinator, if the transaction coordinator subsequently serves as the coordinator of a new transaction, the logical timestamp assigned to the new transaction is not less than the shard timestamp;

[0215] If the transaction coordinator subsequently serves as a participating end of a new transaction, update the write timestamp of the data record carried in the write set of the new transaction to be not less than its own shard timestamp, so as to drive the coordinator of the new transaction to assign a logical timestamp not less than the shard timestamp to the new transaction.

[0216] In an optional embodiment, the processor 71 may further be configured to:

[0217] In the case of a shard failure, obtain a recovery timestamp, where the recovery timestamp is the maximum value among the global timestamps determined on all shards;

[0218] Roll back the transactions processed after the recovery timestamp.

[0219] Furthermore, as Figure 7 shown, the transaction coordinator further includes other components such as a power supply component 73.Figure 7 Only some components are schematically shown, which does not mean that the transaction coordinator only includes Figure 7 the components shown.

[0220] It should be noted that for the technical details in the above embodiments of the transaction coordinator, reference can be made to the relevant descriptions of the transaction coordinator in the foregoing system embodiments. To save space, they will not be elaborated here, but this should not cause loss of the protection scope of this application.

[0221] Figure 8 The figure is a schematic structural diagram of a transaction participant provided for another exemplary embodiment of this application. As Figure 8 shown, the transaction participant may include: a memory 80, a processor 81, and a communication component 82.

[0222] The processor 81, coupled to the memory 80 and the communication component 82, is configured to execute a computer program in the memory 80 for:

[0223] In the case where the target transaction is a distributed transaction, receiving, through the communication component 82, a remote read operation initiated by the transaction coordinator of the target transaction during the read phase of the target transaction;

[0224] Adding an exclusive lock to the data record requested by the remote read operation during the read phase;

[0225] In the write phase of the target transaction, if a remote write operation initiated by the transaction coordinator is received, responding to the remote write operation and releasing the exclusive lock on the corresponding data record.

[0226] In an optional embodiment, the processor 81 may further be configured to:

[0227] Obtain the logical timestamp assigned by the transaction coordinator for the target transaction, where the logical timestamp is assigned based on the execution order between transactions;

[0228] During the read phase, if the read timestamp of the data record corresponding to the remote read operation is less than the logical timestamp of the target transaction, updating the read timestamp of the data record corresponding to the remote read operation to the logical timestamp of the target transaction;

[0229] During the write phase, updating the read timestamp and the write timestamp of the data record corresponding to the remote write operation to the logical timestamp of the target transaction.

[0230] In an optional embodiment, the processor 81 may further be configured to:

[0231] If the transaction participant acts as the transaction coordinator of other transactions, in the case where the other transactions are local transactions, perform consistency verification based on the read timestamp and / or write timestamp of the local data records in its read set.

[0232] In an alternative embodiment, the processor 81 may also be used for:

[0233] Based on the respective logical timestamps corresponding to the transactions processed on the transaction participating end, determine the minimum logical timestamp among the unpersisted transactions on the transaction participating end as the shard timestamp on the transaction participating end;

[0234] Provide its own shard timestamp to the transaction coordinator of the target transaction, so that the transaction coordinator can use the minimum value among the shard timestamps corresponding to all shards as the global timestamp, and determine that the target transaction has completed global persistence processing when the logical timestamp of the target transaction is less than the global timestamp.

[0235] In an alternative embodiment, the processor 81 may also be used for:

[0236] After determining the shard timestamp, if the transaction participating end subsequently serves as the coordinator of a new transaction, the logical timestamp assigned to it is not less than the shard timestamp;

[0237] If the transaction participating end subsequently serves as a participant of a new transaction, update the timestamps of the data records carried in its write set for the new transaction to be not less than the shard timestamp, so as to drive the coordinator of the new transaction to assign a logical timestamp not less than the assigned timestamp to the new transaction.

[0238] Furthermore, as Figure 8 shown, the transaction participating end further includes: other components such as a power supply component 83. Figure 8 Only some components are schematically shown, and it does not mean that the transaction participating end only includes Figure 8 the components shown.

[0239] It should be noted that for the technical details in the above embodiments of the transaction participating end, reference may be made to the relevant descriptions of the transaction participating end in the foregoing system embodiments. For the sake of brevity, they are not elaborated here, but this should not cause loss of the protection scope of this application.

[0240] Correspondingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, and when the computer program is executed, it can implement the steps executable by the transaction coordinator or the transaction participating end in the above method embodiments.

[0241] The above Figures 7-8The memory therein is used to store computer programs and can be configured to store various other data to support operations on the computing platform. Examples of such data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, pictures, videos, etc. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disks or optical discs.

[0242] The above Figures 7-8 The communication component therein is configured to facilitate communication, in a wired or wireless manner, between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0243] The above Figures 7-8 The power supply component therein provides power for various components of the device where the power supply component is located. The power supply component can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device where the power supply component is located.

[0244] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0245] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0246] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0247] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0248] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0249] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.

[0250] A computer-readable medium includes permanent and non-permanent, removable and non-removable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic tape disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer-readable medium does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0251] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0252] The above description is only for the embodiments of the present application and is not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A method for processing a transaction, applicable to a transaction coordinator, the method comprising: Receiving a processing request for a target transaction; During the read phase of the target transaction, running the statement logics of each statement in the target transaction; If there is a remote read logic among the statement logics, determining that the target transaction is a distributed transaction; If it is determined that the target transaction is a distributed transaction, during the read phase of the target transaction, obtaining exclusive locks for each data record involved in its read set, that is, for each read operation of the target transaction, an exclusive lock will be generated on the corresponding data record; Calculating and caching the write set of the target transaction; Releasing the exclusive locks on the data records in the read set that do not need to perform write operations; During the write phase of the target transaction, after completing the write operation on the write set based on the exclusive lock, releasing the exclusive locks on the relevant data records.

2. The method according to claim 1, wherein the obtaining exclusive locks for each data record involved in its read set during the read phase of the target transaction comprises: Determining the local data records in the read set that belong to the transaction coordinator; Adding exclusive locks to the local data records; Triggering each transaction participant to which each non-local data record in the read set belongs to add an exclusive lock to the corresponding data record.

3. The method according to claim 2, wherein the releasing the exclusive locks on the data records in the read set that do not need to perform write operations comprises: During the write phase of the target transaction, if it is determined that any local data record in the read set does not need to perform a write operation, releasing the exclusive lock thereon.

4. The method according to claim 1, further comprising: If there is no remote read logic among the statement logics, determining that the target transaction is a local transaction; In the case where the target transaction is a local transaction, performing a read operation in a lock-free manner during the read phase of the target transaction to obtain the read set of the target transaction; During the write phase of the target transaction, obtaining exclusive locks for each data record involved in its write set to perform the write operation.

5. The method according to claim 1, wherein the calculating the write set of the target transaction comprises: During the read phase of the target transaction, calculating the write set of the target transaction; Caching the write set locally at the transaction coordinator for use by the write phase.

6. The method according to claim 1, wherein the releasing the exclusive locks on the relevant data records after completing the write operation on the write set based on the exclusive lock during the write phase of the target transaction comprises: During the write phase of the target transaction, determining the local data records in the write set that belong to the transaction coordinator; After completing the write operation based on the exclusive lock on the local data record, releasing its exclusive lock; Initiating a write operation for each non-local data record in the write set to its respective transaction participant, so that the transaction participant releases the exclusive lock on the corresponding data record after responding to the write operation.

7. The method according to claim 1, further comprising: During the write phase of the target transaction, allocating a logical timestamp to the target transaction, the logical timestamp being allocated based on the execution order between transactions; If the read timestamp of the local data record belonging to the transaction coordinator in the read set is less than the logical timestamp of the target transaction, update the read timestamp of the local data record to the logical timestamp of the target transaction; Update the read timestamp and write timestamp of the local data record belonging to the transaction coordinator in the write set to the logical timestamp of the target transaction.

8. The method according to claim 7, further comprising: Send the logical timestamp of the target transaction to the transaction participating ends involved in the write set, so that the transaction participating ends can update the read timestamp and / or write timestamp of the data records carried in their read sets and / or write sets based on the logical timestamp of the target transaction.

9. The method according to claim 7, further comprising: If it is determined that the target transaction is a local transaction, perform consistency verification based on the read timestamp and / or write timestamp owned by the local data record belonging to the transaction coordinator in its read set.

10. The method according to claim 1, further comprising: Based on the logical timestamps corresponding to the respective transactions processed on the transaction coordinator, determine the minimum logical timestamp among the unpersisted transactions on the transaction coordinator as the shard timestamp on the transaction coordinator; Obtain the shard timestamps provided by other shards; Take the minimum value among the shard timestamps corresponding to all shards as the global timestamp; If the logical timestamp of the target transaction is less than the global timestamp, determine that the target transaction has completed global persistence processing.

11. The method according to claim 10, further comprising: If there is a special transaction on the transaction coordinator for which no logical timestamp is assigned, take the write timestamp on the data record first accessed by the special transaction as the lower limit of the logical timestamp of the special transaction, and use the lower limit of the logical timestamp instead of the logical timestamp as the basis for determining the shard timestamp.

12. The method according to claim 10, further comprising: After determining the shard timestamp on the transaction coordinator, if the transaction coordinator subsequently serves as the coordinator for a new transaction, the logical timestamp assigned to the new transaction by it is not less than the shard timestamp; If the transaction coordinator subsequently serves as a participating end in a new transaction, update the write timestamp of the data record carried in its write set in the new transaction to be not less than its own shard timestamp, so as to drive the coordinator of the new transaction to assign a logical timestamp not less than the shard timestamp to the new transaction.

13. The method according to claim 10, further comprising: In the case of a shard failure, obtain a recovery timestamp, where the recovery timestamp is the maximum value among the global timestamps determined on all shards; Roll back the transactions processed after the recovery timestamp.

14. A method for processing transactions, applicable to transaction participating ends, the method comprising: In the case where the target transaction is a distributed transaction, receive a remote read operation initiated by the transaction coordinator of the target transaction during the read phase of the target transaction; The target transaction being a distributed transaction is determined by the transaction coordinator based on the existence of remote read logic in the logical statements of the target transaction; In the reading phase, an exclusive lock is added to the data records requested by the remote read operation. That is, for each read operation of the target transaction, an exclusive lock is generated on the corresponding data record; In the writing phase of the target transaction, if a remote write operation initiated by the transaction coordinator is received, after completing the remote write operation, the exclusive lock on the corresponding data record is released.

15. The method according to claim 14, further comprising: Obtaining the logical timestamp assigned by the transaction coordinator for the target transaction, where the logical timestamp is assigned based on the execution order between transactions; In the reading phase, if the read timestamp of the data record corresponding to the remote read operation is less than the logical timestamp of the target transaction, then update the read timestamp of the data record corresponding to the remote read operation to the logical timestamp of the target transaction; In the writing phase, update the read timestamp and write timestamp of the data record corresponding to the remote write operation to the logical timestamp of the target transaction.

16. The method according to claim 15, further comprising: If the transaction participant acts as the transaction coordinator of other transactions, then in the case where the other transaction is a local transaction, perform consistency verification based on the read timestamp and / or write timestamp of the local data records in its read set.

17. The method according to claim 14, further comprising: Based on the logical timestamps corresponding to the respective transactions processed on the transaction participant, determine the minimum logical timestamp among the unpersisted transactions on the transaction participant as the shard timestamp of the transaction participant; Provide its own shard timestamp to the transaction coordinator of the target transaction, so that the transaction coordinator takes the minimum value among the shard timestamps corresponding to all shards as the global timestamp, and determines that the target transaction has completed global persistence processing when the logical timestamp of the target transaction is less than the global timestamp.

18. The method according to claim 17, further comprising: After determining the shard timestamp, if the transaction participant subsequently acts as the coordinator of a new transaction, the logical timestamp assigned to it is not less than the shard timestamp; If the transaction participant subsequently acts as a participant in a new transaction, then update the write timestamp of the data records carried in its write set in the new transaction to be not less than the shard timestamp, so as to drive the coordinator of the new transaction to assign a logical timestamp not less than the shard timestamp for the new transaction.

19. A transaction coordinator, comprising a memory, a processor, and a communication component; The memory is used to store one or more computer instructions; The processor is coupled to the memory and the communication component and is used to execute the one or more computer instructions for: Receiving a processing request for a target transaction through the communication component; In the reading phase of the target transaction, running the logical statements of the target transaction; If there is remote read logic in the logic of each statement, determine that the target transaction is a distributed transaction; If it is determined that the target transaction is a distributed transaction, obtain exclusive locks for each data record involved in the read set during the read phase of the target transaction. That is, for each read operation of the target transaction, an exclusive lock will be generated on the corresponding data record; Calculate the write set of the target transaction and cache it; Release the exclusive locks on the data records in the read set that do not require write operations; In the write phase of the target transaction, after completing the write operation for the write set based on the exclusive lock, release the exclusive locks on the relevant data records.

20. A transaction participating end, including a memory, a processor, and a communication component; The memory is used to store one or more computer instructions; The processor is coupled to the memory and the communication component and is used to execute the one or more computer instructions for: In the case where the target transaction is a distributed transaction, receive, by means of the communication component, a remote read operation initiated by the transaction coordinator of the target transaction during the read phase of the target transaction; The fact that the target transaction is a distributed transaction is determined by the transaction coordinator according to the existence of remote read logic in the logic of each statement in the target transaction; Add an exclusive lock to the data record requested by the remote read operation during the read phase. That is, for each read operation of the target transaction, an exclusive lock will be generated on the corresponding data record; In the write phase of the target transaction, if a remote write operation initiated by the transaction coordinator is received, release the exclusive lock on the corresponding data record after completing the remote write operation.

21. A computer-readable storage medium storing computer instructions, when the computer instructions are executed by one or more processors, causing the one or more processors to execute the transaction processing method according to any one of claims 1-18.

Images