A decentralized distributed transaction processing method and apparatus

Abstract

The application relates to the technical field of distributed transaction processing, and provides a decentralized distributed transaction processing method and device. The method comprises the following steps: a client sends a write request to any one storage node in a server; after receiving the write request sent by the client, the storage node pulls the data of the client to a local server and executes a prepare operation; the storage node after executing the prepare operation transmits the prepare operation to a master node by sending a request request to a next storage node; the master node after executing the prepare operation reversely transmits a commit operation to an initial storage node which initiates the request request by sending a response request to a previous storage node; and the initial storage node which initiates the request request processes the distributed transaction according to the received response request. The application can avoid the congestion caused by the fact that the client sends write requests to all copy nodes, reduce the number of requests, and avoid the defect that each copy node pulls data from the client, causing network congestion of the client.

Description

Technical Field

[0001] This invention relates to the field of distributed transaction processing technology, and in particular to a decentralized distributed transaction processing method and apparatus. Background Technology

[0002] Distributed transactions refer to transactions in which participants, supporting servers, resource servers, and transaction managers are located on different nodes of different distributed systems and belong to different applications. Distributed transactions need to ensure that all corresponding operations either succeed or fail.

[0003] like Figure 1 As shown, the distributed system is divided into two parts: a client and a server. The client is responsible for sending service requests and processing them based on the server's responses. The server is a set of storage servers responsible for storing the requests sent by the client. To prevent data loss in case of server failure, the server writes the client's requests to multiple storage servers. Ensuring data consistency across multiple storage servers is crucial for the normal operation of the distributed system and for distributed transaction processing.

[0004] In practical applications, a two-phase commit approach is used for distributed transaction processing. Each transaction is divided into two phases: prepare and commit. The distributed system includes a master, which is responsible for coordinating and managing transactions, and multiple members, who are the participants in the transactions. Figure 2 As shown, the client sends the request to the master node. The master node initiates a transaction, first performing the preparation phase operations, executing prepare operations on its own node and other member nodes. If any node's prepare operation fails, a rollback operation is performed to undo all prepare operations. When all nodes' prepare operations succeed, a response is sent to the client, and then the commit phase begins, executing commit operations on each node to solidify the request to disk.

[0005] The two-phase commit method has the following drawbacks: 1. Prepare and commit operations can only be initiated on the Master server. When there are a large number of client requests, they all need to be sent to the Master server, which will become a network bottleneck, CPU bottleneck, and memory bottleneck; 2. If a transaction involves N storage nodes, then a complete transaction requires (N-1)*2 messages, which is cumbersome and inefficient. For example, if a transaction involves three storage nodes, then the master node needs to send two prepare requests and two commit requests to two member nodes, for a total of 4 requests.

[0006] Therefore, how to provide a method that does not reduce the efficiency of distributed transaction processing has become an urgent technical problem to be solved. Summary of the Invention

[0007] In view of this, in order to overcome the shortcomings of the prior art, the present invention aims to provide a decentralized distributed transaction processing method and apparatus.

[0008] On one hand, the present invention provides a decentralized distributed transaction processing method, comprising:

[0009] Step S1: The client sends a write request to any storage node on the server. After receiving the write request from the client, the storage node pulls the client's data to the local server and performs a prepare operation.

[0010] Step S2: After performing the prepare operation, the storage node sends a request to the next storage node to pass the prepare operation to the master node;

[0011] Step S3: After performing the prepare operation, the master node sends a response request to the previous storage node, thus propagating the commit operation back to the initial storage node that initiated the request.

[0012] Step S4: The initial storage node that initiated the request processes the distributed transaction based on the received response request.

[0013] Furthermore, in step S1 of the decentralized distributed transaction processing method of the present invention, the prepare operation includes: writing the fetched client data and the unique request ID in the write request to the local disk through the prepare operation.

[0014] Furthermore, step S2 of the decentralized distributed transaction processing method of the present invention includes:

[0015] The initial storage node sends a request to the next storage node;

[0016] Upon receiving a request, the storage node pulls the client's data from the previous storage node to the local server. It then executes a prepare operation to write the pulled client data and the unique request ID from the write request to the local disk. After executing the prepare operation, it determines whether it is the master node.

[0017] If the storage node is not the master node, continue to send a request to the next storage node;

[0018] When the storage node is the master node, it replies to the client with a message that the write request has been processed.

[0019] Furthermore, step S3 of the decentralized distributed transaction processing method of the present invention includes:

[0020] The master node writes the unique request ID from the write request to the local disk by performing a commit operation, and then sends a response request to the next storage node.

[0021] After receiving a response request, the storage node identifies whether there is a commit operation request in the response request;

[0022] When a commit operation request is included in the response request, the unique ID of the write request is written to the local disk by executing the commit operation, and then a response request including the commit operation request is sent to the next storage node.

[0023] If the response request does not contain a commit operation request, the response request is sent directly to the next storage node.

[0024] Furthermore, step S4 of the decentralized distributed transaction processing method of the present invention includes:

[0025] When the initial storage node that initiated the request receives a response request containing a commit operation request, it executes the commit operation to write the unique ID of the write request to the local disk and sends a message of successful transaction processing to the client.

[0026] If the initial storage node that initiated the request does not receive a commit operation request in the response request, it sends a transaction processing failure message to the client.

[0027] On the other hand, the present invention provides a decentralized distributed transaction processing device, comprising:

[0028] The client is used to send a write request to any storage node on the server, and the write request includes a unique request ID;

[0029] The server consists of multiple storage nodes connected in a transaction pipeline. The final storage node in the transaction pipeline is the master node, and the remaining storage nodes are member nodes.

[0030] Furthermore, in the decentralized distributed transaction processing device of the present invention, the initial storage node of the transaction pipeline in the server is used to retrieve the client's data to the local server after receiving the write request sent by the client, perform a prepare operation, and write the retrieved client data and the unique request ID in the write request to the local disk through the prepare operation. After performing the prepare operation, the device sends a request to the next storage node and processes the distributed transaction according to the received response request.

[0031] Furthermore, in the decentralized distributed transaction processing device of the present invention, the server is used for:

[0032] The initial storage node sends a request to the next storage node;

[0033] Upon receiving a request, the storage node pulls the client's data from the previous storage node to the local server. By executing a prepare operation, it writes the pulled client data and the unique request ID from the write request to the local disk. After executing the prepare operation, it determines whether it is the master node.

[0034] If the storage node is not the master node, continue to send a request to the next storage node;

[0035] When the storage node is the master node, it replies to the client with a message that the write request has been processed.

[0036] Furthermore, in the decentralized distributed transaction processing device of the present invention, the storage node acting as the master node in the server is used to send a response request to the upper storage node after performing the prepare operation, thereby passing the commit operation back to the initial storage node that initiated the request.

[0037] Furthermore, in the decentralized distributed transaction processing device of the present invention, the server is used for:

[0038] The master node writes the unique request ID from the write request to the local disk by performing a commit operation, and then sends a response request to the next storage node.

[0039] After receiving a response request, the storage node identifies whether there is a commit operation request in the response request;

[0040] When a commit operation request is included in the response request, the unique ID of the write request is written to the local disk by executing the commit operation, and then a response request including the commit operation request is sent to the next storage node.

[0041] If the response request does not contain a commit operation request, the response request is sent directly to the next storage node.

[0042] The decentralized distributed transaction processing method and apparatus of the present invention have the following beneficial effects:

[0043] 1. Decompose the functions of the transaction control node to avoid congestion caused by clients sending all write requests to the replica node.

[0044] 2. Use a pipelined approach to organize transaction processing, combining prepare and commit into a single request to reduce the number of requests.

[0045] 3. Pipelines enable data transfer between replicas, avoiding network congestion on the client side caused by each replica fetching data from the client. Attached Figure Description

[0046] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is an architecture diagram of a distributed system.

[0048] Figure 2 This is a diagram illustrating distributed transaction processing using a two-phase commit method.

[0049] Figure 3 This is a schematic diagram illustrating the interaction principle between the client and server in this invention.

[0050] Figure 4 This is a flowchart illustrating a decentralized distributed transaction processing method according to an exemplary first embodiment of the present invention.

[0051] Figure 5 This is an execution flowchart of a decentralized distributed transaction processing method according to an exemplary first embodiment of the present invention.

[0052] Figure 6 This is an application diagram of a decentralized distributed transaction processing device according to an exemplary fifth embodiment of the present invention. Detailed Implementation

[0053] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0054] It should be noted that, in the absence of conflict, the following embodiments and features can be combined with each other; and, based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0055] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0056] The technical principle of the present invention is as follows:

[0057] The server uses a data redundancy strategy with replica backup by default. There is one master node in the replicas, and the rest are member nodes. Clients can freely send read and write requests to any node, not necessarily the master node. Internally, each replica node organizes received write requests into a transaction pipeline, starting at the current node and ending at the master node. Member nodes request prepare operations from the next hop node in their request. Upon receiving a prepare request, a node processes it, then determines if it is the master node. If not, it sends a request to the next hop node. Finally, the master node receives the transaction request, processes the prepare operation, replies with success to the user, and then performs a commit operation. Upon successful commit, it notifies the previous commit in the response message. Each node processes commit operations sequentially in reverse order of the pipeline. The transaction ends when the node that initiated the transaction has processed the commit message.

[0058] In this invention, the transaction initiating node can be any node, not necessarily the master node. This distributes the load and reduces performance bottlenecks. Each message is divided into two parts: a request and a response, such as... Figure 3 As shown, using requests for prepare processing and commit processing in the response can significantly reduce message interactions.

[0059] Figure 4 This is a flowchart illustrating a decentralized distributed transaction processing method according to an exemplary first embodiment of the present invention. Figure 5 Here is an execution flowchart of a decentralized distributed transaction processing method according to an exemplary first embodiment of the present invention, as follows: Figure 4 and Figure 5 As shown, the method in this embodiment includes:

[0060] Step S1: The client sends a write request to any storage node on the server. After receiving the write request from the client, the storage node pulls the client's data to the local server and performs a prepare operation.

[0061] Step S2: After performing the prepare operation, the storage node sends a request to the next storage node to pass the prepare operation to the master node;

[0062] Step S3: After performing the prepare operation, the master node sends a response request to the previous storage node, thus propagating the commit operation back to the initial storage node that initiated the request.

[0063] Step S4: The initial storage node that initiated the request processes the distributed transaction based on the received response request.

[0064] In step S1 of the method in this embodiment, the prepare operation is performed, which includes writing the fetched client data and the unique request ID in the write request to the local disk through the prepare operation.

[0065] like Figure 4 As shown in this embodiment, when distributed transaction processing fails, a rollback process is initiated.

[0066] This embodiment of the method splits transactions, eliminating the control node for each transaction. The node that initiates prepare and the node that initiates commit do not necessarily have to be the same node.

[0067] The present invention provides a decentralized distributed transaction processing method in an exemplary second embodiment. This embodiment is... Figure 5 In a preferred embodiment of the method shown, step S2 of the method in this embodiment includes:

[0068] The initial storage node sends a request to the next storage node;

[0069] Upon receiving a request, the storage node pulls the client's data from the previous storage node to the local server. It then executes a prepare operation to write the pulled client data and the unique request ID from the write request to the local disk. After executing the prepare operation, it determines whether it is the master node.

[0070] If the storage node is not the master node, continue to send a request to the next storage node;

[0071] When the storage node is the master node, it replies to the client with a message that the write request has been processed.

[0072] The third exemplary embodiment of the present invention provides a decentralized distributed transaction processing method. The method of this embodiment... Figure 5 In a preferred embodiment of the method shown, step S3 of the method in this embodiment includes:

[0073] The master node writes the unique request ID from the write request to the local disk by performing a commit operation, and then sends a response request to the next storage node.

[0074] After receiving a response request, the storage node identifies whether there is a commit operation request in the response request;

[0075] When a commit operation request is included in the response request, the unique ID of the write request is written to the local disk by executing the commit operation, and then a response request including the commit operation request is sent to the next storage node.

[0076] If the response request does not contain a commit operation request, the response request is sent directly to the next storage node.

[0077] The present invention provides a decentralized distributed transaction processing method in an exemplary fourth embodiment. The method of this embodiment... Figure 5 In a preferred embodiment of the method shown, step S4 of the method in this embodiment includes:

[0078] When the initial storage node that initiated the request receives a response request containing a commit operation request, it executes the commit operation to write the unique ID of the write request to the local disk and sends a message of successful transaction processing to the client.

[0079] If the initial storage node that initiated the request does not receive a commit operation request in the response request, it sends a transaction processing failure message to the client and initiates the rollback process.

[0080] Figure 6 This is an application diagram of a decentralized distributed transaction processing device according to an exemplary fifth embodiment of the present invention, such as... Figure 6 As shown, the apparatus of this embodiment includes:

[0081] The client is used to send a write request to any storage node on the server, and the write request includes a unique request ID;

[0082] The server consists of multiple storage nodes connected in a transaction pipeline. The final storage node in the transaction pipeline is the master node, and the remaining storage nodes are member nodes.

[0083] In practical applications, the decentralized distributed transaction processing device of this embodiment uses the initial storage node of the transaction pipeline on the server side to retrieve the client's data to the local server after receiving the write request sent by the client, and perform a prepare operation. Through the prepare operation, the retrieved client data and the unique request ID in the write request are written to the local disk. After performing the prepare operation, a request is sent to the next storage node, and the distributed transaction is processed according to the received response request.

[0084] In the server, the storage node acting as the master node is used to send a response request to the upper storage node after performing the prepare operation, thus passing the commit operation back to the initial storage node that initiated the request.

[0085] This embodiment of the device utilizes a pipeline to combine the prepare and commit processes into one, completing the two-stage task in a single request. Data can flow between replicas, avoiding the network bottleneck caused by fetching data from the client on multiple servers.

[0086] The server in this embodiment is used for:

[0087] The initial storage node sends a request to the next storage node;

[0088] Upon receiving a request, the storage node pulls the client's data from the previous storage node to the local server. By executing a prepare operation, it writes the pulled client data and the unique request ID from the write request to the local disk. After executing the prepare operation, it determines whether it is the master node.

[0089] If the storage node is not the master node, continue to send a request to the next storage node;

[0090] When the storage node is the master node, it replies to the client with a message that the write request has been processed.

[0091] The master node writes the unique request ID from the write request to the local disk by performing a commit operation, and then sends a response request to the next storage node.

[0092] After receiving a response request, the storage node identifies whether there is a commit operation request in the response request;

[0093] When a commit operation request is included in the response request, the unique ID of the write request is written to the local disk by executing the commit operation, and then a response request including the commit operation request is sent to the next storage node.

[0094] If the response request does not contain a commit operation request, the response request is sent directly to the next storage node.

[0095] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A decentralized distributed transaction processing method, characterized in that, The method includes: Step S1: The client sends a write request to any storage node on the server. After receiving the write request from the client, the storage node pulls the client's data to the local server and performs a prepare operation. Step S2: After performing the prepare operation, the storage node sends a request to the next storage node to pass the prepare operation to the master node; Step S3: After performing the prepare operation, the master node sends a response request to the previous storage node, thus propagating the commit operation back to the initial storage node that initiated the request. Step S4: The initial storage node that initiated the request processes the distributed transaction based on the received response request; Step S2 includes: The initial storage node sends a request to the next storage node; Upon receiving a request, the storage node pulls the client's data from the previous storage node to the local server. It then executes a prepare operation to write the pulled client data and the unique request ID from the write request to the local disk. After executing the prepare operation, it determines whether it is the master node. If the storage node is not the master node, continue to send a request to the next storage node; When the storage node is the master node, it replies to the client with a message that the write request has been processed.

2. The decentralized distributed transaction processing method according to claim 1, characterized in that, In step S1, the prepare operation is performed, which includes writing the fetched client data and the unique request ID in the write request to the local disk through the prepare operation.

3. The decentralized distributed transaction processing method according to claim 1, characterized in that, Step S3 includes: The master node writes the unique request ID from the write request to the local disk by performing a commit operation, and then sends a response request to the next storage node. After receiving a response request, the storage node identifies whether there is a commit operation request in the response request; When a commit operation request is included in the response request, the unique ID of the write request is written to the local disk by executing the commit operation, and then a response request including the commit operation request is sent to the next storage node. If the response request does not contain a commit operation request, the response request is sent directly to the next storage node.

4. The decentralized distributed transaction processing method according to claim 1, characterized in that, Step S4 includes: When the initial storage node that initiated the request receives a response request containing a commit operation request, it executes the commit operation to write the unique ID of the write request to the local disk and sends a message of successful transaction processing to the client. If the initial storage node that initiated the request does not receive a commit operation request in the response request, it sends a transaction processing failure message to the client.

5. A decentralized distributed transaction processing device, characterized in that, The device includes: The client is used to send a write request to any storage node on the server, and the write request includes a unique request ID; The server consists of multiple storage nodes, which are connected in a transaction pipeline. The end storage node in the transaction pipeline is the master node, and the remaining storage nodes are member nodes. The server is used for: The initial storage node sends a request to the next storage node; Upon receiving a request, the storage node pulls the client's data from the previous storage node to the local server. By executing a prepare operation, it writes the pulled client data and the unique request ID from the write request to the local disk. After executing the prepare operation, it determines whether it is the master node. If the storage node is not the master node, continue to send a request to the next storage node; When the storage node is the master node, it replies to the client with a message that the write request has been processed.

6. The decentralized distributed transaction processing device according to claim 5, characterized in that, In the server-side transaction pipeline, the initial storage node is used to retrieve the client's data to the local server after receiving a write request from the client, and perform a prepare operation. The prepare operation writes the retrieved client data and the unique request ID in the write request to the local disk. After performing the prepare operation, it sends a request to the next storage node and processes the distributed transaction based on the received response request.

7. The decentralized distributed transaction processing device according to claim 5, characterized in that, In the server, the storage node acting as the master node is used to send a response request to the upper storage node after performing the prepare operation, thus passing the commit operation back to the initial storage node that initiated the request.

8. The decentralized distributed transaction processing device according to claim 7, characterized in that, The server is used for: The master node writes the unique request ID from the write request to the local disk by performing a commit operation, and then sends a response request to the next storage node. After receiving a response request, the storage node identifies whether there is a commit operation request in the response request; When a commit operation request is included in the response request, the unique ID of the write request is written to the local disk by executing the commit operation, and then a response request including the commit operation request is sent to the next storage node. If the response request does not contain a commit operation request, the response request is sent directly to the next storage node.

Images