Distributed transaction processing methods and frameworks

Abstract

The present invention discloses a distributed transaction processing method and framework, and relates to the field of computer technology. The method comprises: after the local transaction branch is started, the transaction manager records metadata related to the execution of the local transaction branch, and sends the execution result of the local transaction branch to the transaction coordinator; after receiving information that a transaction branch fails to execute, the transaction coordinator sends a compensation instruction to the transaction manager corresponding to the successfully executed transaction branch in the global transaction; after receiving the compensation instruction, the transaction manager compensates the local transaction branch according to the metadata related to the execution of the local transaction branch. Through the above steps, the automatic compensation function of distributed transactions can be realized, and the consistency of online data can be guaranteed as much as possible, so that developers only need to develop the business logic of each application, without considering how to compensate for the abnormality of distributed transactions, thereby improving development efficiency and reducing development costs.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a distributed transaction processing method and framework. Background Art

[0002] With the development of the Internet, single applications can no longer meet the needs of business scenarios. At present, the common practice in the industry is to split applications vertically or horizontally to achieve multiple applications. In the case of multiple applications, the original call within an application becomes a call between multiple applications, so the corresponding single application transaction becomes a distributed transaction.

[0003] Distributed transaction solutions are mainly divided into the following categories: two-phase commit, three-phase commit, TCC (a distributed transaction solution that includes three phases: Try, Confirm, Cancel, hence the name TCC) and message-based distributed transactions. Currently, two-phase commit and three-phase commit are rarely used on the Internet, while TCC and message-based distributed transactions are more commonly used on the Internet.

[0004] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art: whether it is a TCC-based or message-based distributed transaction, developers are required to implement the transaction compensation (or "transaction rollback") mechanism by themselves when the business is abnormal. This is a very troublesome thing for developers, and it is a great test of the R&D personnel's ability to control the business logic. If there is a problem with the transaction compensation (or "transaction rollback") logic, it will not only lead to inconsistent online data, thereby affecting the business, but also make it difficult to troubleshoot the problem. Summary of the invention

[0005] In view of this, the present invention provides a distributed transaction processing method and framework, which can realize the automatic compensation function of distributed transactions and try to ensure the consistency of online data, so that developers only need to develop the business logic of each application without considering how to compensate for abnormalities in distributed transactions, thereby improving development efficiency and reducing development costs.

[0006] To achieve the above objective, according to one aspect of the present invention, a distributed transaction processing method is provided.

[0007] The distributed transaction processing method of the present invention comprises: after the local transaction branch is started, the transaction manager records metadata related to the execution of the local transaction branch, and sends the execution result of the local transaction branch to the transaction coordinator; the execution result is information of successful execution or information of failed execution; after receiving information of failed execution of a transaction branch, the transaction coordinator sends a compensation instruction to the transaction manager corresponding to the successfully executed transaction branch in the global transaction; after receiving the compensation instruction, the transaction manager compensates the local transaction branch according to the metadata related to the execution of the local transaction branch.

[0008] Optionally, the method further includes: after the local transaction branch is successfully executed, the transaction manager transparently transmits the global transaction identifier to the context of the next transaction manager to start the next transaction branch.

[0009] Optionally, the step of the transaction manager recording metadata related to the execution of the local transaction branch includes: intercepting and parsing the database operation statement of the local transaction branch to obtain the metadata related to the execution of the local transaction branch; and locally storing the metadata related to the execution of the local transaction branch.

[0010] Optionally, the database operation statement is an SQL statement; the metadata related to the execution of the local transaction branch includes: metadata of each SQL statement in the local transaction branch; the step of intercepting and parsing the database operation statement of the local transaction branch to obtain the metadata related to the execution of the local transaction branch includes: before the SQL statement of the local transaction branch is executed, parsing the type of the SQL statement; when the type of the SQL statement is an update statement, querying according to the generated mirror query statement to obtain mirror data before the execution of the SQL statement, and using the SQL statement and the mirror data before the execution of the SQL statement as metadata of the SQL statement; when the type of the SQL statement is an insert statement, using the SQL statement as metadata of the SQL statement.

[0011] Optionally, the step of intercepting and parsing the database operation statement of the local transaction branch to obtain the metadata related to the execution of the local transaction branch also includes: when the type of the SQL statement is an update statement, querying according to the generated mirror query statement to obtain the mirror data after the execution of the SQL statement, and using the mirror data after the execution of the SQL statement as the metadata of the SQL statement; the method also includes: before executing the step of compensating the local transaction branch according to the metadata related to the execution of the local transaction branch, firstly performing a consistency check on the current mirror data according to the mirror data after the execution of the SQL statement, and determining that the consistency check passes.

[0012] Optionally, the step of sending the execution result of the local transaction branch to the transaction coordinator, and / or the step of compensating the local transaction branch according to the metadata related to the execution of the local transaction branch, is executed by an asynchronous thread.

[0013] Optionally, the method further includes: when the execution result of each transaction branch received is information of successful execution, the transaction coordinator sends a commit instruction to the transaction manager corresponding to each transaction branch; after receiving the commit instruction, the transaction manager sends the metadata related to the execution of the local transaction branch to the data collector, and deletes the metadata related to the execution of the local transaction branch locally.

[0014] To achieve the above objective, according to another aspect of the present invention, a distributed transaction processing framework is provided.

[0015] The distributed transaction processing framework of the present invention includes: multiple transaction managers and a transaction coordinator; the transaction manager is used to record metadata related to the execution of the local transaction branch after the local transaction branch is started, and send the execution result of the local transaction branch to the transaction coordinator; the execution result is information of successful execution or information of failed execution; the transaction coordinator is used to send a compensation instruction to the transaction manager corresponding to the successfully executed transaction branch in the global transaction after receiving information of failed execution of a transaction branch; the transaction manager is also used to compensate the local transaction branch according to the metadata related to the execution of the local transaction branch after receiving the compensation instruction.

[0016] Optionally, the transaction manager is further configured to, after the local transaction branch is successfully executed, transparently transmit the global transaction identifier to the context of the next transaction manager to start the next transaction branch.

[0017] Optionally, the transaction manager records metadata related to the execution of the local transaction branch, including: the transaction manager intercepts and parses database operation statements of the local transaction branch to obtain the metadata related to the execution of the local transaction branch; and locally stores the metadata related to the execution of the local transaction branch.

[0018] Optionally, the database operation statement is an SQL statement; the metadata related to the execution of the local transaction branch includes: metadata of each SQL statement in the local transaction branch; the transaction manager intercepts and parses the database operation statement of the local transaction branch to obtain the metadata related to the execution of the local transaction branch, including: before the SQL statement of the local transaction branch is executed, the transaction manager parses the type of the SQL statement; when the type of the SQL statement is an update statement, the transaction manager obtains the mirror data before the execution of the SQL statement according to the generated mirror query statement, and uses the SQL statement and the mirror data before the execution of the SQL statement as the metadata of the SQL statement; when the type of the SQL statement is an insert statement, the transaction manager uses the SQL statement as the metadata of the SQL statement.

[0019] Optionally, the transaction manager intercepts and parses the database operation statement of the local transaction branch to obtain the metadata related to the execution of the local transaction branch, and further includes: when the type of the SQL statement is an update statement, the transaction manager obtains the mirror data after the execution of the SQL statement according to the generated mirror query statement, and uses the mirror data after the execution of the SQL statement as the metadata of the SQL statement; the transaction manager is also used to perform a consistency check on the current mirror data according to the mirror data after the execution of the SQL statement before executing the step of compensating the local transaction branch according to the metadata related to the execution of the local transaction branch, and determine that the consistency check passes.

[0020] Optionally, the framework also includes: a data collector; the transaction coordinator is further used to send a commit instruction to the transaction manager corresponding to each transaction branch when the execution result of each transaction branch received is information of successful execution; the transaction manager is further used to send the metadata related to the execution of the local transaction branch to the data collector after receiving the commit instruction, and delete the metadata related to the execution of the local transaction branch from the local.

[0021] To achieve the above objective, according to another aspect of the present invention, an electronic device is provided.

[0022] The electronic device of the present invention comprises: one or more processors; and a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the distributed transaction processing device of the present invention.

[0023] To achieve the above objective, according to yet another aspect of the present invention, a computer readable medium is provided.

[0024] The computer-readable medium of the present invention stores a computer program thereon, and when the program is executed by a processor, the distributed transaction processing method of the present invention is implemented.

[0025] One embodiment of the above invention has the following advantages or beneficial effects: the transaction manager records metadata related to the execution of the local transaction branch and reports the execution result of the local transaction branch. After receiving the information that the execution result of a transaction branch is an execution failure, the transaction coordinator sends a compensation instruction to the transaction manager corresponding to the successfully executed transaction branch in the global transaction. After receiving the compensation instruction, the transaction manager compensates the local transaction branch according to the recorded metadata. These steps can realize the automatic compensation function of distributed transactions, and try to ensure the consistency of online data, so that developers only need to develop the business logic of each application, without considering how to compensate for the abnormality of the distributed transaction, thereby improving development efficiency and reducing development costs. In addition, since the method of the present invention does not affect the execution of the local transaction branches of each application, it has little impact on the performance of each application.

[0026] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0028] Figure 1 is a schematic diagram of main steps of a distributed transaction processing method according to an embodiment of the present invention;

[0029] Figure 2 FIG. 1 is a flow chart of a distributed transaction processing method according to another embodiment of the present invention. Figure 1 ;

[0030] Figure 3 FIG. 1 is a flow chart of a distributed transaction processing method according to another embodiment of the present invention. Figure 2 ;

[0031] Figure 4 is a schematic diagram of main modules of a distributed transaction processing framework according to an embodiment of the present invention;

[0032] Figure 5 is a schematic diagram of main modules of a distributed transaction processing framework according to another embodiment of the present invention;

[0033] Figure 6 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0034] Figure 7It is a schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0036] It should be pointed out that the embodiments of the present invention and the features in the embodiments may be combined with each other if there is no conflict.

[0037] Before introducing the embodiments of the present invention in detail, some technical terms involved in the embodiments of the present invention are first explained.

[0038] Transaction: Transaction is a program execution unit that accesses and possibly updates various data items in the database.

[0039] Transparent transmission: Transparent transmission. Regardless of the transmission service, the transmission network is only responsible for transmitting the service to the destination node and ensuring the quality of transmission, without processing the transmitted service.

[0040] Data source: is a device or raw media that provides some required data. All the information for establishing a database connection is stored in the data source. Just like you can find a file in the file system by specifying the file name, you can find the corresponding database connection by providing the correct data source name.

[0041] Redis database: It is an open source log-type, Key-Value database written in ANSI C language, supporting the network, and can be memory-based or persistent.

[0042] Figure 1 FIG. 1 is a schematic diagram of the main steps of a distributed transaction processing method according to an embodiment of the present invention. Figure 1 As shown, the distributed transaction processing method of the embodiment of the present invention includes:

[0043] Step S101: After the local transaction branch is started, the transaction manager records metadata related to the execution of the local transaction branch and sends the execution result of the local transaction branch to the transaction coordinator.

[0044] A distributed transaction (or "global transaction") includes multiple transaction branches, each of which corresponds to a transaction manager. For example, when the business request is an order request, the distributed transaction may include transaction branch 1 for operating the database of the order application and transaction branch 2 for operating the database of the inventory application. Transaction branch 1 corresponds to transaction manager A, and transaction branch 2 corresponds to transaction manager B. Further, transaction manager A may execute step S101 after transaction branch 1 is started, and transaction manager B may execute step S101 after transaction branch 2 is started.

[0045] In an optional implementation, the transaction manager recording metadata related to the execution of the local transaction branch may include: intercepting and parsing the database operation statement of the local transaction branch to obtain the metadata related to the execution of the local transaction branch; and locally storing the metadata related to the execution of the local transaction branch. The database operation statement may be an SQL statement (structured query statement).

[0046] Exemplarily, when the database operation statement is an SQL statement, the metadata related to the execution of the local transaction branch includes: metadata of each SQL statement in the local transaction branch. The metadata of each SQL statement may include at least one of the following: the original SQL statement, the mirror data before the SQL statement is executed, the mirror data after the SQL statement is executed, and the mirror query statement.

[0047] The execution result is information about successful execution or failed execution. After the local transaction branch is successfully executed (i.e., the transaction branch is locally committed), the transaction manager sends the successful execution information to the transaction coordinator. After the local transaction branch is failed (i.e., the transaction branch is locally rolled back), the transaction manager sends the failed execution information to the transaction coordinator.

[0048] In an optional implementation, the transaction manager may record metadata related to the execution of the local transaction branch through the main thread, and send the execution result of the local transaction branch to the transaction coordinator through the asynchronous thread, thereby not affecting the main thread.

[0049] Step S102: After receiving the information that a transaction branch has failed to execute, the transaction coordinator sends a compensation instruction to the transaction manager corresponding to the successfully executed transaction branch in the global transaction.

[0050] Step S103: After receiving the compensation instruction, the transaction manager compensates the local transaction branch according to the metadata related to the execution of the local transaction branch.

[0051] In a preferred embodiment, the transaction manager may record metadata related to the execution of the local transaction branch through the main thread, and execute step S103 through the asynchronous thread, thereby not affecting the main thread.

[0052] In the embodiment of the present invention, the above steps can realize the automatic compensation function of distributed transactions, and try to ensure the consistency of online data, so that developers only need to develop the business logic of each application, without considering how to compensate for the abnormality of distributed transactions, thereby improving development efficiency and reducing development costs. In addition, since the method of the present invention does not affect the execution of local transaction branches of each application, it has little impact on the performance of each application.

[0053] Combine the following Figure 2 , Figure 3 A distributed transaction processing method according to another embodiment of the present invention is described. Figure 2 , Figure 3 In the illustrated embodiment, it is assumed that a global transaction includes two transaction branches, which are respectively managed by the transaction manager of application A and the transaction manager of application B.

[0054] Figure 2 FIG. 1 is a flow chart of a distributed transaction processing method according to another embodiment of the present invention. Figure 1 . Figure 2 The following figure shows the process flow of a transaction branch that fails to execute. Figure 2 As shown, the distributed transaction processing method of the embodiment of the present invention includes:

[0055] Step S201: After starting a local transaction branch, the transaction manager of application A records metadata related to the execution of the local transaction branch.

[0056] In an optional implementation, the transaction manager recording metadata related to the execution of the local transaction branch may include: intercepting and parsing the database operation statement of the local transaction branch to obtain the metadata related to the execution of the local transaction branch; and locally storing the metadata related to the execution of the local transaction branch. The database operation statement may be an SQL statement (structured query statement).

[0057] Exemplarily, when the database operation statement is an SQL statement, the metadata related to the execution of the local transaction branch includes: metadata of each SQL statement in the local transaction branch. Further, the metadata of each SQL statement may include at least one of the following: the original SQL statement, the mirror data before the SQL statement is executed, the mirror data after the SQL statement is executed, and the mirror query statement.

[0058] In specific implementation, the metadata of each SQL statement in the local transaction branch can be stored in the form of a data stack. For example, the local transaction branch includes three SQL statements, namely SQL statement 1 to SQL statement 3, and the data stack may include metadata of SQL statement 1, metadata of SQL statement 2, and metadata of SQL statement 3. In addition, in specific implementation, since a transaction branch may involve multiple data sources, the data source set used by each transaction branch must also be recorded. Furthermore, when compensating the transaction branch, compensation can be performed for each data source in the data source set.

[0059] In this example, intercepting and parsing the database operation statement of the local transaction branch to obtain the metadata related to the execution of the local transaction branch may include: step 1 to step 3.

[0060] Step 1: Before executing the SQL statement of the local transaction branch, the transaction manager parses the type of the SQL statement.

[0061] The type of the SQL statement may include at least one of the following: an update statement, an insert statement or a delete statement.

[0062] Step 2: When the type of the SQL statement is an update statement, the mirror data before the SQL statement is executed is obtained according to the generated mirror query statement, and the SQL statement and the mirror data before the SQL statement is executed are used as metadata of the SQL statement. For example, the SQL statement is "update table1 set a=1 where id=1", and the corresponding generated mirror query statement is "select*from table1 where id=1". The mirror data before the SQL statement is executed obtained by the mirror query statement is "a=87", then "update table1 set a=1 where id=1" and "a=87" can be used as metadata of the SQL statement.

[0063] Further, step 2 may also include: when the type of the SQL statement is an update statement, querying the generated mirror query statement to obtain the mirror data after the SQL statement is executed, and using the mirror query statement and / or the mirror data after the SQL statement is executed as metadata of the SQL statement. For example, the original SQL statement "update table1 set a=1 whereid=1", the corresponding generated mirror query statement "select*from table1 where id=1", the mirror data "a=87" before the SQL statement is executed, and the mirror data "a=1" after the SQL statement is executed can be used as metadata of the SQL statement.

[0064] Step 3: When the type of the SQL statement is an insert statement, use the SQL statement as metadata of the SQL statement. For example, if the SQL statement is "insert table1(id,a)values(3,99)", then "insert table1(id,a)values(3,99)" can be used as metadata of the SQL statement.

[0065] Further, step 3 may also include: after parsing that the type of the SQL statement is an insert statement, determining whether the insert statement contains a primary key; if the insert statement contains a primary key, the primary key may also be used as metadata of the SQL statement; if the insert statement does not contain a primary key, the primary key may be first queried, and then the queried primary key may also be used as metadata of the SQL statement. For example, the original SQL statement "insert table1(id,a)values(3,99)" and the primary key "id=3" may be used as metadata of the SQL statement.

[0066] Step S202: After the local transaction branch is successfully executed, the transaction manager of application A sends a message of successful execution to the transaction coordinator.

[0067] Among them, “successful execution of the local transaction branch” means “local submission of the transaction branch”. In an optional implementation, the transaction manager of application A may execute step S201 through the main thread and execute step S202 through the asynchronous thread, thereby avoiding the impact on the main thread.

[0068] Step S203: After the local transaction branch is successfully executed, the transaction manager of application A transparently transmits txid to the transaction manager of application B.

[0069] In this step, the transaction manager of application A can transparently transmit txid (global transaction identifier) ​​to the context of the transaction manager of application B to start the transaction branch of application B. In specific implementation, the transparent transmission of txid can be achieved by modifying the RPC (remote procedure call) framework bytecode or by the filter function of the RPC framework. Through step S203, it can be ensured that the downstream application and the upstream application are in the same global transaction, which facilitates the orderly execution of each transaction branch in the distributed transaction.

[0070] Step S204: After the local transaction branch is started, the transaction manager of application B records metadata related to the execution of the local transaction branch.

[0071] Step S205: After the local transaction branch fails to execute, the transaction manager of application B sends execution failure information to the transaction coordinator.

[0072] The “local transaction branch execution failure” means that the transaction branch is locally rolled back (or called “local compensation”). In an optional implementation, the transaction manager of application B may send execution failure information to the transaction coordinator through an asynchronous thread.

[0073] Step S206: After receiving the execution failure information sent by the transaction manager of application B, the transaction coordinator sends a compensation instruction to the transaction manager of application A.

[0074] Step S207: After receiving the compensation instruction, the transaction manager of application A compensates the local transaction branch according to metadata related to the execution of the local transaction branch.

[0075] Specifically, a corresponding compensation statement can be generated according to the metadata of each SQL statement in the local transaction branch, and then the local transaction branch can be compensated according to the compensation statement. For example, the metadata of an SQL statement in the local transaction branch includes: SQL statement "update table1 set a=1where id in(1,2)" and the mirror data "a=87(id=1), a=99(id=2)" before the SQL statement is executed. The compensation statements generated according to the metadata of the SQL statement are "updatetable1 set a=87 where id=1" and "update table1 set a=99 where id=2", and then the original SQL statement can be compensated according to these two compensation statements. For another example, the metadata of an SQL statement in the local transaction branch includes: SQL statement "insert table1(id,a)values(3,99)", and the compensation statement generated according to the metadata of the SQL statement is "delete from table1 where id=3", and then the original SQL statement can be compensated according to this compensation statement.

[0076] Furthermore, when the metadata of each SQL statement also includes the mirror data after the SQL statement is executed, the transaction manager of application A may also perform the following steps before executing step S207: after receiving the compensation instruction, perform a consistency check on the current mirror data according to the mirror data after the SQL statement is executed, and determine that the consistency check passes, that is, determine that the mirror data after the SQL statement is executed is consistent with the current mirror data.

[0077] In addition, when the consistency check fails, the transaction manager of application A can also execute one of the following strategies: Strategy 1: the transaction manager of application A still compensates the local transaction branch according to the metadata related to the execution of the local transaction branch; Strategy 2: the transaction manager of application A sends the information of the consistency check failure to the interface manager, and issues an alarm through the interface manager to notify the relevant administrator of the data anomaly. In an embodiment of the present invention, by performing a consistency check on the data before step S207, the consistency and security of the data can be guaranteed. By providing a variety of executable strategies after the consistency check fails, the flexibility of distributed transaction processing can be improved.

[0078] Step S208: The transaction manager of application A deletes metadata related to the execution of the local transaction branch.

[0079] In an optional implementation, the transaction manager of application A may delete metadata related to the execution of local transaction branches through an asynchronous thread. Further, step S208 may also include: the transaction manager of application A sends metadata related to the execution of local transaction branches to the data collector. The data collector may also store metadata information related to the execution of transaction branches in ES (ElasticSearch, a search server based on Lucene) so that relevant personnel can quickly view the data affected by each distributed transaction. In addition, by collecting metadata information related to the execution of transaction branches, data support can be provided to relevant personnel when manual rollback or manual modification of data is required, without the need for relevant personnel to construct data by themselves, saving time for online problem repair.

[0080] Step S209: The transaction manager of application B deletes metadata related to the execution of the local transaction branch.

[0081] In an optional implementation, the transaction manager of application B may delete metadata related to the execution of the local transaction branch through an asynchronous thread. Further, step S209 may also include: the transaction manager of application B sends metadata related to the execution of the local transaction branch to a data collector. The data collector may store metadata information related to the execution of the transaction branch in ES (ElasticSearch, a search server based on Lucene) so that relevant personnel can quickly view the data affected by each distributed transaction.

[0082] In the embodiment of the present invention, the above steps can realize the automatic compensation function of distributed transactions, thereby ensuring the consistency of online data as much as possible, so that developers only need to develop the business logic of each application, without considering how to compensate for abnormalities in distributed transactions, thereby improving development efficiency and reducing development costs. Furthermore, since the method of the embodiment of the present invention does not affect the execution of local transaction branches of each application and does not need to lock database resources, it has little impact on the performance of each application.

[0083] Figure 3 FIG. 1 is a flow chart of a distributed transaction processing method according to another embodiment of the present invention. Figure 2 . Figure 3 It shows that each transaction branch is successfully executed. Figure 3 As shown, the distributed transaction processing method of the embodiment of the present invention also includes:

[0084] Step S301: After starting a local transaction branch, the transaction manager of application A records metadata related to the execution of the local transaction branch.

[0085] The metadata related to the execution of the local transaction branch may include: metadata of each SQL statement in the local transaction branch. In specific implementation, the metadata of each SQL statement in the local transaction branch may be stored in the form of a data stack. For example, the local transaction branch includes three SQL statements, namely SQL statement 1 to SQL statement 3, and the data stack may include metadata of SQL statement 1, metadata of SQL statement 2, and metadata of SQL statement 3. Furthermore, the metadata of each SQL statement may include at least one of the following: the original SQL statement, the mirror data before the SQL statement is executed, the mirror data after the SQL statement is executed, and the mirror query statement.

[0086] For details on how to implement step S301, please refer to Figure 2 Relevant description about step S201 in the illustrated embodiment.

[0087] Step S302: After the local transaction branch is successfully executed, the transaction manager of application A sends a message of successful execution to the transaction coordinator.

[0088] Among them, "local transaction branch execution is successful" means "local submission is made to the transaction branch."

[0089] Step S303: After the local transaction branch is successfully executed, the transaction manager of application A transparently transmits txid to the transaction manager of application B.

[0090] In this step, the transaction manager of application A may transparently transmit txid (global transaction identifier) ​​to the context of the transaction manager of application B to start the transaction branch of application B.

[0091] Step S304: After the local transaction branch is started, the transaction manager of application B records metadata related to the execution of the local transaction branch.

[0092] Step S305: After the local transaction branch is successfully executed, the transaction manager of application B sends a message of successful execution to the transaction coordinator.

[0093] Among them, "local transaction branch execution is successful" means "local submission is made to the transaction branch."

[0094] Step S306: After receiving the execution success information sent by the transaction manager of application A and the transaction manager of application B, the transaction coordinator sends a commit instruction to the transaction manager of application A.

[0095] In an optional implementation, the transaction coordinator can receive the execution result sent by the transaction manager through an asynchronous thread and send a commit instruction to the transaction manager, thereby not having any impact on the programs of application A and application B. In addition, after receiving the execution result, the transaction coordinator can also save the execution result in the Redis database.

[0096] Step S307: After receiving the execution success information sent by the transaction manager of application A and the transaction manager of application B, the transaction coordinator sends a commit instruction to the transaction manager of application B.

[0097] Step S308: After receiving the commit instruction, the transaction manager of application A deletes metadata related to the execution of the local transaction branch.

[0098] In an optional implementation, the transaction manager of application A may delete metadata related to the execution of the local transaction branch through an asynchronous thread. Further, step S308 may also include: the transaction manager of application A sends metadata related to the execution of the local transaction branch to the data collector. The data collector may store metadata information related to the execution of the transaction branch in ES (ElasticSearch, a search server based on Lucene) so that relevant personnel can quickly view the data affected by each distributed transaction.

[0099] Step S309: After receiving the commit instruction, the transaction manager of application B deletes metadata related to the execution of the local transaction branch.

[0100] In an optional implementation, the transaction manager of application B may delete metadata related to the execution of the local transaction branch through an asynchronous thread. Further, step S309 may also include: the transaction manager of application B sends metadata related to the execution of the local transaction branch to the data collector. The data collector may store metadata information related to the execution of the transaction branch in ES (ElasticSearch, a search server based on Lucene) so that relevant personnel can quickly view the data affected by each distributed transaction.

[0101] In the embodiment of the present invention, the forward process of the distributed transaction can be realized through the above steps. In the forward process, the transaction manager sends the execution result to the transaction coordinator through the asynchronous thread, and deletes the metadata related to the transaction branch execution through the asynchronous thread, which can not affect the main thread and reduce the impact on application performance.

[0102] Figure 4 FIG. 1 is a schematic diagram of the main modules of a distributed transaction processing framework according to an embodiment of the present invention. Figure 4As shown, the distributed transaction processing framework 400 of the embodiment of the present invention includes: multiple transaction managers 401 and a transaction coordinator 402.

[0103] The transaction manager 401 is used to record metadata related to the execution of the local transaction branch after the local transaction branch is started, and send the execution result of the local transaction branch to the transaction coordinator.

[0104] A distributed transaction (or "global transaction") includes multiple transaction branches, each of which corresponds to a transaction manager 401. For example, when the business request is an order request, the distributed transaction may include a transaction branch 1 for operating the database of the order application and a transaction branch 2 for operating the database of the inventory application. The transaction branch 1 corresponds to one transaction manager 401, and the transaction branch 2 corresponds to another transaction manager 401. Furthermore, the transaction manager 401 of the order application may perform operations of recording metadata and sending execution results after the transaction branch 1 is started; the transaction manager 401 of the inventory application may perform operations of recording metadata and sending execution results after the transaction branch 2 is started.

[0105] In an optional implementation, the transaction manager 401 records metadata related to the execution of the local transaction branch, which may include: the transaction manager 401 intercepts and parses the database operation statement of the local transaction branch to obtain the metadata related to the execution of the local transaction branch, and locally stores the metadata related to the execution of the local transaction branch. The database operation statement may be an SQL statement (structured query statement).

[0106] The execution result is information about successful execution or failed execution. After the local transaction branch is successfully executed (i.e., the transaction branch is locally committed), the transaction manager 401 sends the successful execution information to the transaction coordinator 402. After the local transaction branch is failed (i.e., the transaction branch is locally rolled back), the transaction manager 401 sends the failed execution information to the transaction coordinator 402.

[0107] In an optional implementation, the transaction manager 401 may record metadata related to the execution of the local transaction branch through the main thread, and send the execution result of the local transaction branch to the transaction coordinator 402 through an asynchronous thread, thereby not affecting the main thread.

[0108] The transaction coordinator 402 is used to send a compensation instruction to the transaction manager corresponding to the successfully executed transaction branch in the global transaction after receiving the information that a transaction branch has failed to execute.

[0109] The transaction manager 401 is further configured to compensate the local transaction branch according to the metadata related to the execution of the local transaction branch after receiving the compensation instruction.

[0110] In a preferred embodiment, the transaction manager 401 may record metadata related to the execution of the local transaction branch through the main thread, and execute the step of compensating the local transaction branch through an asynchronous thread, thereby not affecting the main thread.

[0111] In the embodiment of the present invention, the distributed transaction processing framework can realize the automatic compensation function of distributed transactions, and try to ensure the consistency of online data, so that developers only need to develop the business logic of each application, without considering how to compensate for the abnormality of distributed transactions, thereby improving development efficiency and reducing development costs. In addition, since the method of the present invention does not affect the execution of local transaction branches of each application, it has little impact on the performance of each application.

[0112] Figure 5 FIG. 1 is a schematic diagram of the main modules of a distributed transaction processing framework according to another embodiment of the present invention. Figure 5 As shown, the distributed transaction processing framework 500 of the embodiment of the present invention includes: multiple transaction managers 501, a transaction coordinator 502, and a data collector 503.

[0113] The transaction manager 501 is used to record metadata related to the execution of the local transaction branch after the local transaction branch is started, and send the execution result of the local transaction branch to the transaction coordinator 502. The execution result is information of successful execution or information of failed execution.

[0114] In an optional implementation, the transaction manager 501 records metadata related to the execution of the local transaction branch, including: intercepting and parsing the database operation statement of the local transaction branch to obtain the metadata related to the execution of the local transaction branch; and locally storing the metadata related to the execution of the local transaction branch. The database operation statement may be an SQL statement.

[0115] Exemplarily, when the database operation statement is an SQL statement, the metadata related to the execution of the local transaction branch includes: metadata of each SQL statement in the local transaction branch. In specific implementation, the metadata of each SQL statement in the local transaction branch can be stored in the form of a data stack. The metadata of each SQL statement can include at least one of the following: the original SQL statement, the mirror data before the SQL statement is executed, the mirror data after the SQL statement is executed, and the mirror query statement.

[0116] In this example, the transaction manager 501 intercepts and parses the database operation statement of the local transaction branch to obtain the metadata related to the execution of the local transaction branch, which may include: before the SQL statement of the local transaction branch is executed, the transaction manager 501 parses the type of the SQL statement; when the type of the SQL statement is an update statement, the transaction manager 501 obtains the mirror data before the execution of the SQL statement according to the generated mirror query statement, and uses the SQL statement and the mirror data before the execution of the SQL statement as the metadata of the SQL statement; when the type of the SQL statement is an insert statement, the transaction manager 501 uses the SQL statement as the metadata of the SQL statement.

[0117] Further, in this example, the transaction manager 501 intercepts and parses the database operation statement of the local transaction branch to obtain the metadata related to the execution of the local transaction branch, which may also include: when the type of the SQL statement is an update statement, the transaction manager 501 obtains the mirror data after the SQL statement is executed according to the generated mirror query statement, and uses the mirror data after the SQL statement is executed as the metadata of the SQL statement.

[0118] The execution result is information about successful execution or failed execution. After the local transaction branch is successfully executed (i.e., the transaction branch is locally committed), the transaction manager 501 can send the successful execution information to the transaction coordinator 502 through an asynchronous thread; after the local transaction branch is failed (i.e., the transaction branch is locally rolled back), the transaction manager 501 can send the failed execution information to the transaction coordinator 502 through an asynchronous thread. Furthermore, after the local transaction branch is successfully executed, the transaction manager 501 can also be used to pass the global transaction identifier to the context of the next transaction manager to start the next transaction branch.

[0119] The transaction coordinator 502 is used to send a compensation instruction to the transaction manager corresponding to the successfully executed transaction branch in the global transaction after receiving the information that a transaction branch failed to execute. Further, the transaction coordinator 502 is also used to send a commit instruction to the transaction manager 501 corresponding to each transaction branch when receiving the information that the execution result of each transaction branch is successful.

[0120] The transaction manager 501 is further configured to compensate the local transaction branch according to the metadata related to the execution of the local transaction branch after receiving the compensation instruction sent by the transaction coordinator 502.

[0121] Specifically, the transaction manager 501 may generate corresponding compensation statements according to metadata of each SQL statement in the local transaction branch, and then compensate the local transaction branch according to the compensation statements.

[0122] Furthermore, when the metadata of each SQL statement also includes the mirror data after the SQL statement is executed, the transaction manager 501 can also be used to perform a consistency check on the current mirror data according to the mirror data after the SQL statement is executed before performing the operation of compensating the local transaction branch according to the compensation statement, and determine whether the consistency check passes. That is, it is determined that the mirror data after the SQL statement is executed is consistent with the current mirror data. In the embodiment of the present invention, the data is firstly checked for consistency, and then the local transaction branch is compensated after the check passes, which can improve the security of the data.

[0123] The transaction manager 501 is further configured to send the metadata related to the execution of the local transaction branch to the data collector and delete the metadata related to the execution of the local transaction branch from the local after receiving the commit instruction sent by the coordinator 502. In an optional implementation, the transaction manager 501 may perform the operations of sending the metadata to the data collector and deleting the metadata from the local through an asynchronous thread.

[0124] The data collector 503 is used to collect metadata information related to the execution of transaction branches, and can also be used to store the metadata information related to the execution of transaction branches in ES (ElasticSearch, a search server based on Lucene), so that relevant personnel can quickly view the data affected by each distributed transaction. In addition, by collecting metadata information related to the execution of transaction branches through the data collector, data support can be provided to relevant personnel when manual data rollback or manual data modification is required, without the need for relevant personnel to construct data by themselves, saving time for online problem repair.

[0125] Furthermore, the distributed transaction processing framework of the embodiment of the present invention may also include: an interface manager, which is used to display the execution results of each transaction branch and metadata related to the transaction branch execution, and can also be used to send alarm information to relevant personnel after the automatic rollback fails.

[0126] In the embodiment of the present invention, the distributed transaction processing framework can realize the automatic compensation function of distributed transactions, thereby ensuring the consistency of online data as much as possible, so that developers only need to develop the business logic of each application, without considering how to compensate for the abnormality of distributed transactions, thereby improving development efficiency and reducing development costs. Furthermore, since the distributed transaction processing framework of the present invention does not affect the execution of local transaction branches of each application and does not need to lock database resources, it has little impact on the performance of each application.

[0127] Figure 6 An exemplary system architecture 600 is shown to which the distributed transaction processing method or distributed transaction processing framework according to an embodiment of the present invention can be applied.

[0128] like Figure 6 As shown, system architecture 600 may include terminal devices 601, 602, 603, network 604 and server 605. Network 604 is used to provide a medium for communication links between terminal devices 601, 602, 603 and server 605. Network 604 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0129] The user can use the terminal devices 601, 602, 603 to interact with the server 605 through the network 604 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 601, 602, 603, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0130] The terminal devices 601 , 602 , and 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0131] The server 605 may be a server that provides various services, such as a background management server that supports shopping websites browsed by users using terminal devices 601, 602, and 603. After receiving an order instruction or a payment instruction, the background management server may process the distributed transaction involved in the instruction according to the distributed transaction processing method or framework of an embodiment of the present invention.

[0132] It should be noted that the distributed transaction processing method provided in the embodiment of the present invention is generally executed by the server 605 , and accordingly, the distributed transaction processing framework is generally set in the server 605 .

[0133] It should be understood that Figure 6 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0134] Figure 7 A schematic diagram of the structure of a computer system 700 suitable for implementing an electronic device of an embodiment of the present invention is shown. Figure 7 The system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0135] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0136] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.

[0137] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-mentioned functions defined in the system of the present invention are executed.

[0138] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0139] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0140] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes a transaction manager and a transaction coordinator. The names of these modules do not constitute a limitation on the modules themselves in some cases, for example, the transaction manager may also be described as a "module for managing local transaction branches".

[0141] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device executes the following process: after the local transaction branch is started, the transaction manager records metadata related to the execution of the local transaction branch, and sends the execution result of the local transaction branch to the transaction coordinator; the execution result is information of successful execution or information of failed execution; after receiving information of failed execution of a transaction branch, the transaction coordinator sends a compensation instruction to the transaction manager corresponding to the successfully executed transaction branch in the global transaction; after receiving the compensation instruction, the transaction manager compensates the local transaction branch according to the metadata related to the execution of the local transaction branch.

[0142] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A distributed transaction processing method, characterized in that: The method comprises: After the local transaction branch is started, the transaction manager intercepts and parses the SQL statement of the local transaction branch to obtain metadata of each SQL statement in the local transaction branch, and sends the execution result of the local transaction branch to the transaction coordinator, including: before the SQL statement of the local transaction branch is executed, parsing the type of the SQL statement; when the type of the SQL statement is an update statement, querying according to the generated mirror query statement to obtain the mirror data before the SQL statement is executed, and querying according to the generated mirror query statement to obtain the mirror data after the SQL statement is executed; using the SQL statement, the mirror data before the SQL statement is executed, and the mirror data after the SQL statement is executed as the metadata of the SQL statement; the execution result is information of successful execution or information of failed execution; After receiving the information that a transaction branch failed to execute, the transaction coordinator sends a compensation instruction to the transaction manager corresponding to the transaction branch that successfully executed in the global transaction; After receiving the compensation instruction, the transaction manager compensates the local transaction branch according to metadata related to the execution of the local transaction branch.

2. The method according to claim 1, characterized in that The method further comprises: After the transaction manager successfully executes the local transaction branch, it passes the global transaction identifier to the context of the next transaction manager to start the next transaction branch.

3. The method according to claim 1, characterized in that Also includes: The metadata related to the local transaction branch execution is stored locally.

4. The method according to claim 3, characterized in that The step of intercepting and parsing the database operation statement of the local transaction branch to obtain the metadata related to the execution of the local transaction branch also includes: when the type of the SQL statement is an insert statement, using the SQL statement as the metadata of the SQL statement.

5. The method according to claim 4, characterized in that The method further includes: before executing the step of compensating the local transaction branch according to the metadata related to the execution of the local transaction branch, first performing a consistency check on the current mirror data according to the mirror data after the execution of the SQL statement, and determining that the consistency check passes.

6. The method according to claim 1, characterized in that The step of sending the execution result of the local transaction branch to the transaction coordinator, and / or the step of compensating the local transaction branch according to the metadata related to the execution of the local transaction branch, are executed by asynchronous threads.

7. The method according to claim 1, characterized in that The method further comprises: When the received execution results of each transaction branch are all information of successful execution, the transaction coordinator sends a commit instruction to the transaction manager corresponding to each transaction branch; After receiving the commit instruction, the transaction manager sends the metadata related to the execution of the local transaction branch to the data collector, and deletes the metadata related to the execution of the local transaction branch locally.

8. A distributed transaction processing framework, characterized in that: The framework includes: multiple transaction managers and transaction coordinators; The transaction manager is used to intercept and parse the SQL statement of the local transaction branch after the local transaction branch is started, so as to obtain the metadata of each SQL statement in the local transaction branch, and send the execution result of the local transaction branch to the transaction coordinator, including: before the SQL statement of the local transaction branch is executed, parsing the type of the SQL statement; when the type of the SQL statement is an update statement, querying according to the generated mirror query statement to obtain the mirror data before the SQL statement is executed, and querying according to the generated mirror query statement to obtain the mirror data after the SQL statement is executed; using the SQL statement, the mirror data before the SQL statement is executed, and the mirror data after the SQL statement is executed as the metadata of the SQL statement; the execution result is information of successful execution or information of failed execution; The transaction coordinator is used to send a compensation instruction to the transaction manager corresponding to the successfully executed transaction branch in the global transaction after receiving the information that a transaction branch has failed to execute; The transaction manager is further configured to compensate the local transaction branch according to metadata related to the execution of the local transaction branch after receiving the compensation instruction.

9. The frame according to claim 8, characterized in that The transaction manager is further configured to transparently transmit the global transaction identifier to the context of the next transaction manager after the local transaction branch is successfully executed, so as to start the next transaction branch.

10. The frame according to claim 8, characterized in that Also includes: The metadata related to the local transaction branch execution is stored locally.

11. The frame according to claim 10, characterized in that The transaction manager intercepts and parses the database operation statement of the local transaction branch to obtain the metadata related to the execution of the local transaction branch, and further includes: when the type of the SQL statement is an insert statement, the transaction manager uses the SQL statement as the metadata of the SQL statement.

12. The frame according to claim 11, characterized in that The transaction manager is further used to perform a consistency check on the current mirror data according to the mirror data after the execution of the SQL statement before executing the step of compensating the local transaction branch according to the metadata related to the execution of the local transaction branch, and determine that the consistency check passes.

13. The frame according to claim 8, characterized in that The framework also includes: a data collector; The transaction coordinator is further used to send a commit instruction to the transaction manager corresponding to each transaction branch when the execution result of each transaction branch received is information of successful execution; the transaction manager is further used to send the metadata related to the execution of the local transaction branch to the data collector after receiving the commit instruction, and delete the metadata related to the execution of the local transaction branch from the local.

14. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

15. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images