A transaction processing method and system

Abstract

The present invention discloses a transaction processing method and system. The method comprises: completing the registration of a coordinator service in a distributed lock service; in the process of executing SQL through a valid database session, generating a transaction identity number based on the number of the coordinator obtained in the registration process; starting a new transaction based on the transaction identity number; and submitting the new transaction based on the XA protocol and the created submission milestone table. The method can solve the problems of single point failure, data inconsistency, uncertainty of transaction status, etc. in the existing two-phase submission process under specific conditions.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer technology, and in particular to a transaction processing method and system. Background Art

[0002] Distributed databases need to ensure the ACID properties of transactions, where A, atomicity, and D, durability, are usually implemented by the two-phase commit protocol or its variants. Two-phase commit is an algorithm that ensures the atomicity and durability of transactions in distributed databases.

[0003] The two-phase commit includes the first phase and the second phase. The first phase is the commit request phase, which includes: the coordinator asks all participant nodes whether the commit operation can be performed, and waits for the response of each participant node. The second phase is the commit execution phase, which includes: when the response messages obtained by the coordinator node from all participant nodes are "agree", the coordinator node sends a "formal commit" request to all participant nodes, and the coordinator node completes the transaction after receiving the "complete" message fed back by all participant nodes; if the response message returned to the coordinator node by any participant node is "terminate", or the coordinator node cannot obtain the response message of all participant nodes before the query timeout of the first phase, the coordinator node sends a "rollback operation" request to all participant nodes, and the coordinator node cancels the transaction after receiving the "rollback completed" message fed back by all participant nodes.

[0004] The above two-phase commit has problems such as single point failure, data inconsistency, and uncertainty about a transaction in extreme cases. These problems will affect the atomicity and durability of distributed transactions. Summary of the invention

[0005] The present invention provides a transaction processing method and system to solve the problems of single point failure, data inconsistency, uncertainty of transaction status, etc. in the existing two-phase commit process under specific conditions.

[0006] According to one aspect of the present invention, there is provided a transaction processing method, comprising:

[0007] Complete the registration of the coordinator service in the distributed lock service;

[0008] During the execution of SQL through a valid database session, a transaction identification number is generated based on the coordinator number obtained during the registration process;

[0009] Open a new transaction based on the transaction identification number;

[0010] The new transaction is committed based on the XA protocol and the created commit milestone table.

[0011] According to another aspect of the present invention, there is provided a transaction processing system, comprising:

[0012] At least one coordinator service, a distributed lock service, and at least one sub-database, wherein the distributed lock service is respectively connected to the at least one coordinator service in communication, and each coordinator service is respectively connected to the at least one sub-database in communication;

[0013] The one or more coordinator services are used to complete the registration of the coordinator service in the distributed lock service;

[0014] The one or more coordinator services are further used to generate a transaction identity identification number based on the coordinator number obtained during the registration process;

[0015] The one or more coordinator services are further used to start a new transaction based on the transaction identity number;

[0016] The one or more coordinator services are also used to commit the new transaction based on the XA protocol and the created commit milestone table;

[0017] The one or more sub-databases are used to perform a formal commit operation of the transaction after receiving a formal commit request of the XA protocol sent by the coordinator service;

[0018] The one or more sub-databases are further configured to execute a transaction rollback operation after receiving a rollback request of the XA protocol sent by the coordinator service.

[0019] According to another aspect of the present invention, a coordinator service device is provided, the coordinator service device comprising: at least one processor; and

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the transaction processing method described in any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer readable storage medium is provided.

[0023] The readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the transaction processing method described in any embodiment of the present invention when executed.

[0024] The technical solution of the embodiment of the present invention first completes the registration of the coordinator service in the distributed lock service; then, in the process of executing SQL through a valid database session, a transaction identity identification number is generated based on the number of the coordinator obtained in the registration process; then a new transaction is started based on the transaction identity identification number; finally, the new transaction is committed based on the XA protocol and the created commit milestone table, which solves the problems of single point failure, data inconsistency, uncertainty about the transaction status, etc. in the existing two-phase commit process under specific conditions, and achieves the beneficial effect of effectively committing transactions in a distributed system.

[0025] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 A flowchart of a transaction processing method provided in Embodiment 1 of the present invention;

[0028] Figure 2 A flowchart of a coordinator registering a controller in a transaction processing method provided in the first embodiment of the present invention;

[0029] Figure 3 A schematic diagram of a new transaction submission process in a transaction processing method provided in Embodiment 1 of the present invention;

[0030] Figure 4 A flowchart of a transaction processing method provided in Embodiment 2 of the present invention;

[0031] Figure 5 A schematic diagram of a process flow of fault recovery in a transaction processing method provided in Embodiment 2 of the present invention;

[0032] Figure 6 A schematic diagram of the structure of a transaction processing system provided by Embodiment 3 of the present invention;

[0033] Figure 7 A schematic diagram of the structure of a transaction processing system provided by Embodiment 3 of the present invention;

[0034] Figure 8A schematic diagram of the structure of another transaction processing system provided in Embodiment 3 of the present invention;

[0035] Fig. 9 A schematic diagram of the structure of a coordinator service device of a transaction processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only an embodiment of a part of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should belong to the scope of protection of the present invention. It should be understood that the various steps recorded in the method implementation of the present invention can be performed in different orders and / or in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0037] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0038] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0039] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0040] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes, and are not used to limit the scope of these messages or information.

[0041] Here we need to further explain the single point failure, data inconsistency and uncertainty of a transaction in the two-phase commit process. When the first phase is completed and the second phase is in progress, if the coordinator crashes, all participants who have not received the "formal commit" or "rollback operation" are still in the state of locking transaction resources and cannot continue to complete the transaction operation, which will cause a single point failure; in the second phase, after the coordinator sends a formal commit request to the participant, a local network anomaly occurs or the coordinator fails during the process of sending the formal commit request, which will result in only some participants receiving the formal commit request, and these participants will perform the formal commit operation after receiving the formal commit request, but other machines that have not received the formal commit request cannot perform the transaction commit, so the entire distributed system has data inconsistency; the coordinator crashes after sending the first formal commit request, and the only participant who received this message also crashes, then even if the coordinator generates a new coordinator through the election protocol, the state of this transaction is uncertain, and it is impossible to know whether the transaction has been committed. This situation is the uncertainty of a transaction in the extreme case.

[0042] Embodiment 1

[0043] Figure 1 This is a flow chart of a transaction processing method provided in the first embodiment of the present invention. The method is applicable to the case where transaction processing is performed through a distributed system, and the method can be executed by a distributed system.

[0044] like Figure 1 As shown, a transaction processing method provided in Embodiment 1 of the present invention includes the following steps:

[0045] S110. Complete registration of the coordinator service in the distributed lock service.

[0046] This step is performed before the system performs the second-phase commit. When the coordinator service is started, the registration of the coordinator service is completed in the distributed lock service. In the system, a distributed lock service can communicate with multiple coordinator services.

[0047] The distributed lock service may be a service that can provide a distributed lock function, and the distributed lock service may be Chubby, ZooKeeper, Etcd, etc.

[0048] In this embodiment, the registration of the coordinator service can be completed through the temporary node provided in the distributed lock service. Here, the registration of the coordinator service can also be implemented by any other method, and the implementation method of the coordinator service registration is not specifically limited here.

[0049] Furthermore, the registration of the coordinator service in the distributed lock service includes: after the current coordinator service is started, through the coordinator registration controller in the current coordinator service, obtaining the number of the current coordinator based on the distributed lock service, and creating a record representing the current coordinator in the distributed lock service.

[0050] In one embodiment, each coordinator service may include a coordinator registration controller and a coordinator main module, and the registration of the current coordinator service may be implemented through the coordinator registration controller in the current coordinator service.

[0051] The structures maintained in the distributed lock service may include a mutex, a counter, and a live node list aliveList.

[0052] In this embodiment, after the current coordinator service is started, the registration of the current coordinator service can be completed through the mutex lock, counter and surviving node list in the distributed lock service and the temporary node function provided by the distributed lock service.

[0053] Furthermore, through the coordinator registration controller in the current coordinator service, the number of the current coordinator is obtained based on the distributed lock service, and a record representing the current coordinator is created in the distributed lock service, including: after determining through the coordinator registration controller in the current coordinator service that the coordinator main module in the current coordinator service is working normally, if the distributed lock service has not created a temporary node corresponding to the current coordinator service or the temporary node corresponding to the created current coordinator service is not in a valid state, then after obtaining the mutex lock in the distributed lock service, the counter in the distributed lock service is increased by 1, and the number of the increased counter is used as the number of the current coordinator; through the coordinator registration controller, a temporary node corresponding to the current coordinator service is created in the list of surviving nodes in the distributed lock service, and the temporary node stores the number of the current coordinator and the connection method of the current coordinator; the mutex lock is released through the coordinator registration controller, and after the lease of the coordinator registration controller for a preset number of temporary nodes has expired, continue to check whether the coordinator main module is working normally.

[0054] The preset value may be a pre-set value, and the preset value may be any value less than or equal to 1 / 3. For example, the preset value may be 1 / 3, 1 / 4, or 1 / 5. Within the time range of the preset value of temporary node lease timeouts, the temporary node is valid. Within this time range, the coordinator registration controller may remain in a sleep state to reduce the workload of the coordinator registration controller. The coordinator registration controller being in a sleep state indicates that the coordinator registration controller has stopped working.

[0055] For example, Figure 2 This is a workflow diagram of a coordinator registering a controller in a transaction processing method provided in the first embodiment of the present invention, such as Figure 2 As shown, after the coordinator service is started, the coordinator registration controller can always run the following process: check the working status of the coordinator main module to determine whether the working status is normal. If not, delete the temporary node; if normal, determine whether the temporary node has been created and is in a valid state. If so, renew the temporary node, sleep for 1 / 3 of the temporary nodes, and then return to the step of checking the working status of the coordinator main module; if not, obtain the mutex in the distributed lock service, increase the counter counter=counter+1, and use the increased result as the current coordinator's number coordId, create a temporary node in the surviving node list, release the mutex, sleep for 1 / 3 of the temporary nodes, and then return to the step of checking the working status of the coordinator main module.

[0056] It should be noted that the coordinator registration controller creates temporary nodes in the list of surviving nodes. The client must continuously renew the temporary nodes. If the time from the last renewal to now exceeds the lease timeout, the distributed lock service can automatically delete the temporary node. Therefore, if a coordinator service goes down, the corresponding temporary node will be automatically cleared.

[0057] It should also be noted that since only one service can obtain the mutex at the same time, the coordId numbers obtained by different coordinator services are incremented according to the time when the numbers are obtained and are not repeated. If a coordinator service is restarted after a crash or re-registered after a failure, the number of the coordinator service must also be re-obtained, and it can be regarded as a different coordinator service from the previous coordinator service.

[0058] S120. In the process of executing SQL through a valid database session, a transaction identity number is generated based on the coordinator number obtained during the registration process.

[0059] In this embodiment, before obtaining a valid database session, it is necessary to create a database session and check whether the database session is valid. The database session is created based on a temporary node in the surviving node list during the coordinator service registration process. Among them, creating a database session, checking whether the database session is valid, and executing SQL through a valid database session belong to the session management of the database driver and are executed by the database driver.

[0060] The process of creating a database session includes: selecting a temporary node in the surviving node list as the target node; reading the number of the coordinator in the target node and the connection method of the coordinator; and creating a database session connected to the coordinator according to the connection method.

[0061] Exemplarily, the process of creating a database session includes the following steps: Step 1, the business program can connect to the distributed lock service through the API interface of the database driver, randomly select a temporary node from the list of surviving nodes in the distributed lock service, and record the temporary node as nodeX; Step 2, read the coordinator number and connection method in nodeX; Step 3, create a session connected to the coordinator.

[0062] Furthermore, after creating the database session, it also includes: adding the session object to a relationship graph, the relationship graph maintains the relationship between the coordinator's number and the session, one coordinator corresponds to a list containing multiple database sessions; determining whether the database session of the coordinator is valid.

[0063] The relationship map coordSessionMap may be maintained by the driver. For example, the data structure of coordSessionMap is:

[0064] Map <CoordId,List <dbsession>>coordSessionMap

[0065] Furthermore, determining whether the database session of the coordinator is valid includes: monitoring the target node through the monitoring interface of the distributed lock service; when it is monitored that the target node is deleted, determining that the database session of the coordinator is invalid, traversing the list under the coordinator corresponding to the target node in the relationship graph through the driver, and setting the database sessions in the list to an invalid state.

[0066] For example, after the database session is successfully created, it will be in a valid state. The watch interface of the distributed lock service can monitor nodeX, and it will automatically call back when nodeX is deleted. The deletion of nodeX indicates that the corresponding coordinator service has failed. The driver needs to traverse the list under the corresponding coordinator in coordSessionMap and set the database session object in it to an invalid state.

[0067] In this embodiment, the business program can execute SQL through a valid database session, and the SQL of the business program can be forwarded to the coordinator corresponding to the database session through a driver.

[0068] Executing SQL through a valid database session may include opening a transaction. The business program executes BEGIN to request the coordinator service to open a new transaction. After receiving the transaction opening request, the coordinator service needs to generate a transaction identification number. The transaction identification number is generated based on the coordinator number obtained during the registration process.

[0069] Furthermore, a transaction identity number is generated based on the coordinator number obtained during the registration process, including: after receiving a transaction start request sent by the business program, the concatenation result of the coordinator number obtained during the registration process and the coordinator's local transaction identity number is used as the transaction identity number.

[0070] Exemplarily, the length of the transaction id (gtrid) is 96 bits (12 bytes) and the format is as follows: [32 bits of coordinator number, 64 bits of coordinator local transaction id].

[0071] S130: Start a new transaction based on the transaction identity number.

[0072] The new transaction may be understood as a distributed transaction, and the new transaction may also be understood as a sub-transaction corresponding to a sub-database.

[0073] In this embodiment, it can be determined through which coordinator the transaction is opened according to the transaction identification number.

[0074] Further, starting a new transaction based on the transaction identity number includes: sending a start instruction of the XA protocol to all sub-databases through a coordinator corresponding to the transaction identity number to start a new transaction.

[0075] Among them, the XA protocol is a protocol that uses two-phase commit to ensure that all resources commit or rollback any specific transaction at the same time. Common databases on the market, such as MySQL, support the XA protocol. However, the XA protocol describes the interface between the global transaction manager and the local resource manager in the two-phase commit, and does not restrict the specific implementation.

[0076] For example, after receiving a transaction start request, the coordinator service may perform the following operations: generate a transaction id, send an XA BEGIN <gtrid>Instruction, start a subtransaction, and return a successful response to the business program.

[0077] S140: Commit the new transaction based on the XA protocol and the created commit milestone table.

[0078] Among them, a submission milestone table needs to be maintained during the submission process of a new transaction, as shown in Table 1.

[0079] Table 1 A submission milestone table

[0080] Fields type constraint gtrid BINARY(12) PRIMARY KEY

[0081] Among them, different records of the submission milestone table can be horizontally split according to the hash value of gtrid and saved in multiple sub-databases.

[0082] In this embodiment, when the business program executes a commit statement to request the commit of a new transaction, the coordinator will execute the commit of the new transaction.

[0083] Among them, based on the XA protocol, it can be understood that the requests sent by the coordinator to all sub-databases are requests of the XA protocol, which may include a commit request of the XA protocol, a formal commit operation request of the XA protocol, and a rollback operation request of the XA protocol.

[0084] In this embodiment, the difference between committing the new transaction based on the XA protocol and the created commit milestone table and the two-phase commit is that the requests sent by the coordinator are all requests of the XA protocol; and the transactions agreed to be committed by the participants are inserted into the commit milestone table.

[0085] Specifically, the submission of the new transaction based on the XA protocol and the created submission milestone table includes: after receiving the submission request sent by the business program, sending the submission request of the XA protocol to all sub-databases through the coordinator service; if the response results returned by all sub-databases are that they agree to submit, then recording the new transaction in the created submission milestone table, and sending a formal submission operation request of the XA protocol to all sub-databases, so that all sub-databases complete the submission of the new transaction; if there is a response message returned by a sub-database that disagrees to submit, then sending a rollback operation request of the XA protocol to all sub-databases.

[0086] It should be noted that whether a new transaction should be fully committed or fully rolled back depends on whether the coordinator has written the corresponding record in the commit milestone table. Whether it is completed by the coordinator itself or by the restorer, if the corresponding record is written in the commit milestone table, the new transaction should be fully committed, otherwise it should be fully rolled back. This solves the data consistency problem.

[0087] For example, Figure 3 A schematic diagram of a new transaction submission process in a transaction processing method provided in Embodiment 1 of the present invention is shown in FIG. Figure 3 As shown in the figure, the new transaction commit process executed by the coordinator includes: sending XAPREPARE to all sub-databases <gtrid>That is, the submission request of the XA protocol waits for all sub-databases to return the PREPARE result, and determines whether all sub-databases return PREPARE success, that is, whether the response results returned by all sub-databases agree to submit; if not, send XA ROLLBACK to all sub-databases <gtrid>That is, the rollback operation request of the XA protocol; if so, insert the record of the current transaction in the commit milestone table and send XA COMMIT to all sub-databases. <gtrid>That is, the formal submission operation request of the XA protocol.

[0088] A transaction processing method provided in the first embodiment of the present invention first completes the registration of the coordinator service in the distributed lock service; then, in the process of executing SQL through a valid database session, a transaction identity number is generated based on the number of the coordinator obtained in the registration process; then a new transaction is started based on the transaction identity number; and finally, the new transaction is committed based on the XA protocol and the created commit milestone table. The above method can solve the single point failure problem by completing the registration of the coordinator service, and can solve the data consistency problem by creating a commit milestone table.

[0089] Embodiment 2

[0090] Figure 4 This is a flow chart of a transaction processing method provided by Embodiment 2 of the present invention. Embodiment 2 is optimized on the basis of the above embodiments. In this embodiment, a restorer service is introduced on the basis of Embodiment 1 to perform fault recovery. For details not yet provided in this embodiment, please refer to Embodiment 1.

[0091] like Figure 4 As shown, a transaction processing method provided by Embodiment 2 of the present invention includes the following steps:

[0092] S210. Complete registration of the coordinator service in the distributed lock service.

[0093] S220. In the process of executing SQL through a valid database session, a transaction identity number is generated based on the coordinator number obtained during the registration process.

[0094] S230: Start a new transaction based on the transaction identity number.

[0095] S240: Commit the new transaction based on the XA protocol and the created commit milestone table.

[0096] S250. Complete fault recovery through the restorer service based on the registration of the coordinator service, the XA protocol, and the commit milestone table.

[0097] In this embodiment, when the coordinator service goes down, the sub-transactions that the coordinator has not yet processed can be processed by the restorer service. The restorer can be regularly awakened by a timed awakening mechanism so that the restorer can perform fault recovery.

[0098] Among them, based on the coordinator number, counter number and surviving node list obtained in the registration process of the coordinator service, the command of the XA protocol, the request of the XA protocol and the submission milestone table can complete the fault repair. Further, through the restorer service, based on the registration of the coordinator service, the XA protocol and the submission milestone table, the fault recovery is completed, including: traversing all sub-databases, obtaining unprocessed sub-transactions in all sub-databases, and the unprocessed sub-transactions are obtained from the unprocessed sub-transaction list on each sub-database by executing the recovery command of the XA protocol; parsing the global transaction identity to which the unprocessed sub-transaction belongs, and obtaining the number of the coordinator corresponding to the unprocessed sub-transaction; determining whether the coordinator corresponding to the number of the coordinator corresponding to the unprocessed sub-transaction is down according to the counter number and the surviving node list; if not, not performing operations on the unprocessed sub-transaction; if yes, and determining that the global transaction corresponding to the unprocessed sub-transaction exists in the submission milestone table, executing the formal submission request of the XA protocol; if yes, and determining that the global transaction corresponding to the unprocessed sub-transaction does not exist in the submission milestone table, executing the rollback request of the XA protocol.

[0099] When executing the recovery command of the XA protocol, the sub-database can return the list of unprocessed sub-transactions on it. The unprocessed sub-transactions are sub-transactions that have successfully executed the submission request of the XA protocol, but have not yet executed the formal submission operation of the XA protocol and the transaction rollback operation of the XA protocol. Each record in the unprocessed sub-transaction list contains information such as the global transaction id (gtrid) to which the sub-transaction belongs.

[0100] Among them, the principle of determining whether the coordinator corresponding to the number of the coordinator corresponding to the unprocessed sub-transaction is down based on the number of counters and the surviving node list is: since the number of the coordinator must be increasing, if the number of a coordinator is less than the number of counters, but there is no temporary node corresponding to the coordinator in the surviving node list, it means that the coordinator corresponding to the number of the coordinator has crashed.

[0101] For example, Figure 5 A schematic diagram of a process flow of failure recovery in a transaction processing method provided in Embodiment 2 of the present invention is shown in FIG. Figure 5 As shown, the restorer can execute the following process: read the counter variable, that is, the number of counters, and the activeList survival list, that is, the survival node list, from the distributed lock service, traverse each sub-database, and when traversing each sub-database, execute the XA RECOVER command, that is, the recovery command of the XA protocol, obtain the pending sub-transaction list, that is, the unprocessed sub-transaction list on the sub-database, traverse each pending sub-transaction, that is, the unprocessed sub-transaction list in the pending sub-transaction list, parse the global transaction id (gtrid) to which it belongs, obtain the coordinator number coordId responsible for executing the pending sub-transaction, and determine whether the coordinator corresponding to coordId has crashed according to the counter variable and the activeList survival list. If not, the unprocessed sub-transaction will not be processed temporarily; if it has crashed, continue to determine whether there is a global transaction corresponding to the pending sub-transaction in the commit milestone table. If not, execute XA ROLLBACK to roll back the pending sub-transaction; if so, execute XA COMMIT commits the pending subtransaction and continues to determine whether all pending subtransactions of the current subdatabase have been traversed. If so, traverse the next subdatabase and return to the step of executing the XA RECOVER command; if not, traverse the next pending subtransaction.

[0102] The second embodiment of the present invention provides a transaction processing method, which embodies the process of fault recovery. By using this method, the restorer service can continue to execute the sub-transactions that have not been processed by the coordinator service, effectively solving the uncertainty of a transaction in extreme cases.

[0103] Embodiment 3

[0104] Figure 6 This is a structural diagram of a transaction processing system provided in Embodiment 3 of the present invention. The system can be applicable to situations where transaction processing is performed. The transaction processing system can be implemented by software and hardware. The system can be used to execute the transaction processing method described in any embodiment of the present invention.

[0105] like Figure 6 As shown, a transaction processing system provided by Embodiment 3 of the present invention includes:

[0106] One or more coordinator services 110, distributed lock services 120, and one or more sub-databases 130;

[0107] The distributed lock service 120 is respectively connected to at least one coordinator service 110 in communication, and each coordinator service 110 is respectively connected to at least one sub-database 130 in communication;

[0108] One or more coordinator services 110 are used to complete the registration of the coordinator service in the distributed lock service 120;

[0109] The one or more coordinator services 110 are also used to generate a transaction identity number based on the coordinator number obtained during the registration process;

[0110] The one or more coordinator services 110 are further configured to start a new transaction based on the transaction identification number;

[0111] The one or more coordinator services 110 are further used to commit the new transaction based on the XA protocol and the created commit milestone table;

[0112] One or more sub-databases 130 are used to perform a formal commit operation of a transaction after receiving a formal commit request of the XA protocol sent by the coordinator service 110;

[0113] The one or more sub-databases 130 are further configured to execute a transaction rollback operation after receiving a rollback request of the XA protocol sent by the coordinator service.

[0114] The distributed lock service 120 may communicate with a plurality of coordinator services 110 respectively, and one coordinator service 110 may communicate with at least one sub-database 130 .

[0115] Among them, the sub-database 130 can be a participant in the two-phase commit, and all sub-databases 130 need to support the XA protocol. The sub-database 130 needs to enable the data disaster recovery function, and also has complete disaster recovery for the processing of XA transactions. For example, in MySQL 8.0 and above, when the semi-synchronous replication mode is enabled, the XA PREPARE, XACOMMIT, XA ROLLBACK and other events of the XA transaction must be correctly synchronized to the standby machine before the host returns the client command response, ensuring that the pending XA sub-transactions can continue to be resolved in the event of a master-slave switch.

[0116] In this embodiment, the coordinator service 110 may include a coordinator main module 111 and a coordinator registration controller 112. The coordinator main module 111 is used to implement two-phase commit, and the coordinator registration controller 112 is used to obtain the number of the current coordinator through the distributed lock service 120 and create a record representing the current coordinator in the distributed lock service 120.

[0117] Furthermore, the coordinator registration controller is specifically used to determine that after the coordinator main module is working normally, if the distributed lock service 120 has not created a temporary node corresponding to the current coordinator service or the temporary node corresponding to the created current coordinator service is not in a valid state, then after acquiring the mutex lock in the distributed lock service 120, the counter in the distributed lock service 120 is increased by 1, and the number of the increased counter is used as the number of the current coordinator; a temporary node corresponding to the current coordinator service is created in the list of surviving nodes in the distributed lock service 120, and the temporary node stores the number of the current coordinator and the connection method of the current coordinator; the mutex lock is released, and the preset number of temporary nodes is slept for a period of time, and then the coordinator main module continues to check whether it is working normally.

[0118] Figure 7 A schematic diagram of the structure of a transaction processing system provided by the third embodiment of the present invention is shown in FIG. Figure 7 As shown in the figure, the database driver is connected to the distributed lock service and multiple coordinator services respectively. The database driver will expose the API interface for business programs to call.

[0119] The process of the database driver creating a database session includes: selecting a temporary node in the surviving node list as the target node; reading the number of the coordinator in the target node and the connection method of the coordinator; and creating a database session connected to the coordinator according to the connection method.

[0120] Furthermore, the process of the database driver creating a database session also includes: adding the session object to a relationship graph, the relationship graph maintains the relationship between the coordinator's number and the session, one coordinator corresponds to a list containing multiple database sessions; determining whether the database session of the coordinator is valid.

[0121] Among them, determining whether the database session of the coordinator is valid includes: monitoring the target node through the monitoring interface of the distributed lock service; when it is monitored that the target node is deleted, determining that the database session of the coordinator is invalid, traversing the list under the coordinator corresponding to the target node in the relationship graph through the driver, and setting the database session in the list to an invalid state.

[0122] In this embodiment, the business program can execute SQL through the created database session, and the driver can forward the SQL of the business program to the coordinator service corresponding to the session.

[0123] Furthermore, the coordinator service is specifically used to use the concatenation result of the coordinator number obtained in the registration process and the coordinator's local transaction identification number as the transaction identification number after receiving the transaction opening request sent by the business program;

[0124] Furthermore, the coordinator service is also used to send a start instruction of the XA protocol to all sub-databases through the coordinator corresponding to the transaction identity number to start a new transaction, and the new transaction is a sub-transaction corresponding to the sub-database.

[0125] Furthermore, the new transaction is committed based on the XA protocol and the created commit milestone table, including: after receiving the commit request sent by the business program, sending a commit request of the XA protocol to all sub-databases through the coordinator service; if the response results returned by all sub-databases are that they agree to commit, then the new transaction is recorded in the created commit milestone table, and a formal commit operation request of the XA protocol is sent to all sub-databases, so that all sub-databases complete the submission of the new transaction; if there is a response message returned by a sub-database that disagrees to commit, then a rollback operation request of the XA protocol is sent to all sub-databases.

[0126] A transaction processing system provided by the third embodiment of the present invention can solve the single point failure problem by completing the registration of the coordinator service, and can solve the data consistency problem by creating a commit milestone table.

[0127] Furthermore, Figure 8 This is a schematic diagram of the structure of another transaction processing system provided by Embodiment 3 of the present invention. Figure 8 As shown, the system also includes a restorer service 140, which is respectively connected to the distributed lock service 120 and at least one sub-database 130 for communication; the restorer service 140 is used to complete fault recovery based on the registration of the coordinator service, the XA protocol and the commit milestone table.

[0128] Specifically, the restorer service 140 is specifically used to traverse all sub-databases, obtain unprocessed sub-transactions in all sub-databases 130, and the unprocessed sub-transactions are obtained from the unprocessed sub-transaction list on each sub-database by executing the recovery command of the XA protocol; parse the global transaction identity to which the unprocessed sub-transaction belongs, and obtain the number of the coordinator corresponding to the unprocessed sub-transaction; determine whether the coordinator corresponding to the number of the coordinator corresponding to the unprocessed sub-transaction is down according to the number of counters and the list of surviving nodes; if not, do not perform operations on the unprocessed sub-transaction; if so, and it is determined that the global transaction corresponding to the unprocessed sub-transaction exists in the commit milestone table, execute the formal commit request of the XA protocol; if so, and it is determined that the global transaction corresponding to the unprocessed sub-transaction does not exist in the commit milestone table, execute the rollback request of the XA protocol.

[0129] After the coordinator service goes down, the system continues to execute the sub-transactions that have not been processed by the coordinator service through the restorer service, effectively solving the uncertainty of a transaction in extreme cases.

[0130] Embodiment 4

[0131] Fig. 9 A schematic diagram of the structure of a coordinator service device 10 that can be used to implement an embodiment of the present invention is shown. The coordinator service device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The coordinator service device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0132] like Fig. 9 As shown, the coordinator service device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. Various programs and data required for the operation of the coordinator service device 10 can also be stored in the RAM 13. The processor 11, ROM 12 and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0133] Multiple components in the coordinator service device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0134] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a transaction processing method.

[0135] In some embodiments, the transaction processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps in the transaction processing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the transaction processing method in any other appropriate manner (e.g., by means of firmware).

[0136] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0137] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0138] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0139] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0140] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0141] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0142] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0143] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.< / gtrid> < / gtrid> < / gtrid> < / gtrid> < / dbsession>

Claims

1. A transaction processing method, characterized in that: The method comprises: Complete the registration of the coordinator service in the distributed lock service; During the execution of SQL through a valid database session, a transaction identification number is generated based on the coordinator number obtained during the registration process; Open a new transaction based on the transaction identification number; Commit the new transaction based on the XA protocol and the created commit milestone table; The registration of the coordinator service in the distributed lock service includes: After the current coordinator service is started, the number of the current coordinator is obtained based on the distributed lock service through the coordinator registration controller in the current coordinator service, and a record representing the current coordinator is created in the distributed lock service; Obtaining the number of the current coordinator based on the distributed lock service through the coordinator registration controller in the current coordinator service, and creating a record representing the current coordinator in the distributed lock service, including: After determining through the coordinator registration controller in the current coordinator service that the coordinator main module in the current coordinator service is operating normally, if the distributed lock service has not created a temporary node corresponding to the current coordinator service or the temporary node corresponding to the created current coordinator service is not in a valid state, then after acquiring the mutex lock in the distributed lock service, add 1 to the counter in the distributed lock service, and use the increased number of the counter as the number of the current coordinator; Creating a temporary node corresponding to the current coordinator service in the list of surviving nodes in the distributed lock service through the coordinator registration controller, wherein the temporary node stores the number of the current coordinator and the connection mode of the current coordinator; The coordinator registration controller releases the mutex lock, and after the coordinator registration controller sleeps for a preset number of temporary nodes and the lease times out, continues to check whether the coordinator main module is working normally; The committing of the new transaction based on the XA protocol and the created commit milestone table includes: After receiving the commit request sent by the business program, send the commit request of the XA protocol to all sub-databases through the coordinator service; If all sub-databases return a response that the transaction is committed, the new transaction is recorded in the created commit milestone table, and a formal commit operation request of the XA protocol is sent to all sub-databases, so that all sub-databases complete the commit of the new transaction. If a response message returned by a sub-database indicates that the commit is not agreed, a rollback operation request of the XA protocol is sent to all sub-databases.

2. The method according to claim 1, characterized in that: The valid database session is created through a database driver. The process of creating a database session includes: Select a temporary node in the surviving node list as the target node; Read the number of the coordinator in the target node and the connection mode of the coordinator; A database session connected to the coordinator is created according to the connection mode.

3. The method according to claim 2, characterized in that: Also includes: Adding the session object to a relationship graph, wherein the relationship graph maintains the relationship between the coordinator number and the session, and one coordinator corresponds to a list containing multiple database sessions; Determine whether the coordinator's database session is valid.

4. The method according to claim 3, characterized in that: The determining whether the database session of the coordinator is valid comprises: Monitor the target node through the monitoring interface of the distributed lock service; When it is monitored that the target node is deleted, it is determined that the database session of the coordinator is invalid, and the list under the coordinator corresponding to the target node in the relationship graph is traversed by the driver program, and the database sessions in the list are set to an invalid state.

5. The method according to claim 1, characterized in that Generate a transaction ID based on the coordinator ID obtained during the registration process, including: After receiving the transaction opening request sent by the business program, the concatenation result of the coordinator's number obtained during the registration process and the coordinator's local transaction identification number is used as the transaction identification number.

6. The method according to claim 1, characterized in that The starting of a new transaction based on the transaction identity number includes: A start instruction of the XA protocol is sent to all sub-databases through the coordinator corresponding to the transaction identity number to start a new transaction, where the new transaction is a sub-transaction corresponding to the sub-database.

7. The method according to claim 1, characterized in that Also includes: Fault recovery is completed through the restorer service based on the registration of the coordinator service, the XA protocol and the commit milestone table.

8. The method according to claim 7, characterized in that The restorer service completes the fault recovery based on the registration of the coordinator service, the XA protocol and the commit milestone table, including: Traverse all sub-databases to obtain unprocessed sub-transactions in all sub-databases, wherein the unprocessed sub-transactions are obtained from the unprocessed sub-transaction list on each sub-database by executing a recovery command of the XA protocol; Parsing the global transaction identity to which the unprocessed subtransaction belongs, and obtaining the number of the coordinator corresponding to the unprocessed subtransaction; Determine whether the coordinator corresponding to the number of the coordinator corresponding to the unprocessed subtransaction is down according to the number of counters and the list of surviving nodes; If not, no operation is performed on the unprocessed subtransaction; If yes, and it is determined that there is a global transaction corresponding to the unprocessed subtransaction in the commit milestone table, a formal commit request of the XA protocol is executed; If so, and it is determined that the global transaction corresponding to the unprocessed subtransaction does not exist in the commit milestone table, a rollback request of the XA protocol is executed.

9. A transaction processing system, characterized in that: The system comprises at least one coordinator service, a distributed lock service and at least one sub-database, wherein the distributed lock service is respectively connected to the at least one coordinator service in communication, and each coordinator service is respectively connected to the at least one sub-database in communication; The one or more coordinator services are used to complete the registration of the coordinator service in the distributed lock service; The one or more coordinator services are further used to generate a transaction identity identification number based on the coordinator number obtained during the registration process; The one or more coordinator services are further used to start a new transaction based on the transaction identity number; The one or more coordinator services are also used to commit the new transaction based on the XA protocol and the created commit milestone table; The one or more sub-databases are used to perform a formal commit operation of the transaction after receiving a formal commit request of the XA protocol sent by the coordinator service; The one or more sub-databases are further used to execute a rollback operation of the transaction after receiving a rollback request of the XA protocol sent by the coordinator service; The coordinator service includes a coordinator main module and a coordinator registration controller; The coordinator registration controller is used to obtain the number of the current coordinator through the distributed lock service and create a record representing the current coordinator in the distributed lock service; The coordinator registration controller is specifically used to determine that after the coordinator main module is working normally, if the distributed lock service has not created a temporary node corresponding to the current coordinator service or the temporary node corresponding to the created current coordinator service is not in a valid state, then after acquiring the mutex lock in the distributed lock service, the counter in the distributed lock service is increased by 1, and the number of the increased counter is used as the number of the current coordinator; a temporary node corresponding to the current coordinator service is created in the list of surviving nodes in the distributed lock service, and the temporary node stores the number of the current coordinator and the connection method of the current coordinator; the mutex lock is released, and after the lease of a preset number of temporary nodes has timed out, continue to check whether the coordinator main module is working normally; The committing of the new transaction based on the XA protocol and the created commit milestone table includes: after receiving a commit request sent by the business program, sending a commit request of the XA protocol to all sub-databases through the coordinator service; If the response results returned by all sub-databases are that they agree to commit, the new transaction is recorded in the created commit milestone table, and a formal commit operation request of the XA protocol is sent to all sub-databases so that all sub-databases complete the submission of the new transaction; if the response message returned by any sub-database is that it disagrees to commit, a rollback operation request of the XA protocol is sent to all sub-databases.

10. The system according to claim 9, characterized in that The system further comprises a restorer service, wherein the restorer service is respectively communicatively connected to the distributed lock service and the at least one sub-database; The restorer service is used to complete fault recovery based on the registration of the coordinator service, the XA protocol and the commit milestone table.

Images