Distributed transaction processing system and method

Abstract

The application provides a distributed transaction processing system and method, comprising: a client-oriented fusion server and a plurality of incremental servers connected with the fusion server, each incremental server having a plurality of corresponding backup servers, each incremental server being used for storing incremental data, the incremental data stored by the plurality of incremental servers being non-overlapping; the fusion server being used for: when receiving a data update request sent by the client, acquiring a data division rule; determining an incremental server involved in this data update according to data information involved in the data request and the data division rule; if the number of the incremental servers involved is N, splitting a physical update plan corresponding to this data update into N update sub-plans; controlling the N incremental servers involved to execute respective corresponding update sub-plans, and realizing data update. The application can improve the processing efficiency of incremental data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of database technology, in particular to a distributed transaction processing system and method. BACKGROUND

[0002] With the further development of cloud computing, web2.0 and other technologies, the traditional relational database is not capable of dealing with massive data processing. The NoSQL database abandons the strict transaction consistency and paradigm constraints of the traditional relational database, adopts a weak consistency model, supports distributed and horizontal expansion, and meets the needs of massive data management, so it has been widely concerned and applied in the field of big data processing, such as Google's BigTable, Amazon's Dynamo. Compared with traditional relational data, NoSQL database has the characteristics of high cost performance and scalability, so it has become the preferred database for domestic financial enterprises to deal with massive data. However, in addition to processing massive data, traditional financial businesses also need to ensure the strong consistency of transactions during data processing. However, most of the current NoSQL databases do not support strong consistency of transactions, so they cannot meet the needs of financial businesses.

[0003] The distributed massive relational database YaoBase is a relational database system with high reliability, high availability, strong consistency and high scalability of traditional distributed data management systems. YaoBase also supports SQL functions and database transaction functions. YaoBase realizes cross-row and cross-table transactions on a large amount of data, and designs and implements multi-transaction node, multi-redundancy, fault tolerance and load balancing mechanisms to ensure that the system as a whole continuously provides high-quality data read and write services.

[0004] Currently, the transaction processing logic design of YaoBase adopts a single incremental server (incremental server, YaotxnSvr, referred to as TS). In YaoBase, the incremental server is used to store incremental update data, and a primary and backup are used in deployment. Data is synchronized between the primary and backup through operation logs. Only the primary YaotxnSvr is allowed to provide write services in a cluster. When processing write transactions, YaotxnSvr writes incremental data into memory. When the memory usage reaches a certain level, the data in the memory will be dumped to the SSD disk. During daily consolidation, the data in the incremental server will be consolidated into the baseline data server YaodataSvr. YaotxnSvr ensures reliability through write operation logs.

[0005] Under the architecture of the single incremental server, although the existing YaoBase supports strong consistency of transactions, when high-concurrency read-write transactions concentrate, the YaotxnSvr is easy to become a performance bottleneck, especially because most incremental data in the YaotxnSvr is stored in the memory, so that the transaction response time is long, and the database read-write performance cannot be fully exerted, and therefore, it is urgent to make an optimization design for the single incremental server. SUMMARY

[0006] The embodiment of the present application provides a distributed transaction processing system and method, which can reduce the single-point load of the single incremental server and improve the processing efficiency of incremental data.

[0007] In a first aspect, the distributed transaction processing system provided by the embodiment of the present application comprises a client-oriented fusion server and a plurality of incremental servers connected in communication with the fusion server, each incremental server has a plurality of corresponding backup servers, each incremental server is used for storing incremental data, and the incremental data stored by the plurality of incremental servers does not overlap.

[0008] The fusion server is used for: when receiving a data update request sent by the client, obtaining a data division rule; determining the incremental servers involved in the current data update according to the data information involved in the data request and the data division rule; if the number of the incremental servers involved is N, then splitting the physical update plan corresponding to the current data update into N update sub-plans; controlling the N incremental servers involved to execute the respective update sub-plans to realize data update; wherein the data division rule comprises a first mapping relationship between a plurality of data tables and a plurality of groups and a second mapping relationship between the plurality of groups and the plurality of incremental servers, and N is a positive integer greater than 1.

[0009] In a second aspect, the distributed transaction processing method provided by the embodiment of the present application is realized based on the distributed transaction processing system provided in the first aspect, and the method comprises:

[0010] The fusion server is used for: when receiving a data update request sent by the client, obtaining a data division rule; determining the incremental servers involved in the current data update according to the data information involved in the data request and the data division rule; if the number of the incremental servers involved is N, then splitting the physical update plan corresponding to the current data update into N update sub-plans; controlling the N incremental servers involved to execute the respective update sub-plans to realize data update; wherein the data division rule comprises a first mapping relationship between a plurality of data tables and a plurality of groups and a second mapping relationship between the plurality of groups and the plurality of incremental servers, and N is a positive integer greater than 1.

[0011] The distributed transaction processing system and method provided by the embodiment of the present application comprises a fusion server and a plurality of incremental servers, each of which is used to store incremental data, and the incremental data stored by the plurality of incremental servers does not overlap; when the fusion server receives a data update request sent by a client, a data division rule is obtained; according to the data information involved in the data request and the data division rule, the incremental servers involved in the current data update are determined; if the number of the incremental servers involved is multiple, the physical update plan corresponding to the current data update is split into a plurality of update sub-plans; the plurality of incremental servers involved are controlled to execute the respective update sub-plans, so as to realize data update. Since a plurality of incremental servers are arranged in the embodiment of the present application, the data can be orderly divided in the plurality of incremental servers through the data division rule, when high-concurrency read-write transactions concentrate, the single-point load of a single incremental server can be reduced, the processing efficiency of the incremental data is improved, and the overall performance of the distributed database is improved. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 FIG. 1 is a structural block diagram of a distributed transaction processing system in an embodiment of the present application;

[0013] Figure 2 FIG. 2 is a mapping relationship diagram of a data division rule in an embodiment of the present application;

[0014] Figure 3a FIG. 3 is a schematic diagram of a distributed transaction processing system in an embodiment of the present application;

[0015] Figure 3b FIG. 4 is an interaction schematic diagram between a coordinator server and two participant servers in an embodiment of the present application;

[0016] Figure 4 FIG. 5 is a flowchart of physical update plan splitting by a fusion server in an embodiment of the present application;

[0017] Figure 5 FIG. 6 is a transaction processing timing diagram in an incremental server in an embodiment of the present application;

[0018] Figure 6 FIG. 7 is a processing flowchart of a coordinator server after timeout in an embodiment of the present application;

[0019] Figure 7 FIG. 8 is a processing flowchart of a participant server after timeout in an embodiment of the present application. DETAILED DESCRIPTION

[0020] In a first aspect, the present application provides a distributed transaction processing system.

[0021] Referring to Figure 1 The system comprises a client-oriented fusion server and a plurality of incremental servers in communication with the fusion server, each incremental server having a plurality of corresponding backup servers, each incremental server being configured to store incremental data, the incremental data stored by the plurality of incremental servers not overlapping.

[0022] The system is actually part of a YaoBase system architecture. The fusion server is YaosqlSvr, abbreviated as SS. The incremental server is YaotxnSvr, abbreviated as TS. In the new system architecture, an incremental server has a plurality of backup servers, i.e., an incremental server has a plurality of replicas, i.e., one master and multiple backups. The incremental server and the corresponding plurality of replicas form a Paxos group, and the Paxos protocol can be used to ensure the consistency and reliability of data between the incremental server and the corresponding plurality of replicas.

[0023] The fusion server is client-oriented, i.e., it receives requests sent by the client and controls the plurality of incremental servers to perform corresponding processing and then sends corresponding feedback to the client.

[0024] YaoBase is a typical distributed database architecture, with data distributed across different nodes. The data is divided into baseline data and incremental data, which are stored in baseline servers (i.e., YaodataSvr, abbreviated as DS) and incremental servers, respectively. When the fusion server receives a query request sent by the client, it sends a request to the baseline server and the incremental server. The baseline server performs a query operation to obtain static data and returns the static data to the fusion server. The incremental server performs a query operation to obtain incremental data and returns the incremental data to the fusion server. The fusion server combines the static data and the incremental data and returns them to the client.

[0025] The above is a query service provided by YaoBase. Of course, YaoBase can also provide a data write service, i.e., a data update service. Since multiple incremental servers are provided in the embodiment of the present application, and the incremental data is distributed across different incremental servers, the embodiment of the present application mainly solves how to perform related operations based on the provision of multiple incremental servers.

[0026] To solve this problem, referring to Figure 2In this embodiment of the invention, a data partitioning rule is maintained on a server. This data partitioning rule is divided into a first-level partition and a second-level partition. A first mapping relationship can be obtained through the first-level partition, and a second mapping relationship can be obtained through the second-level partition. First, the data is partitioned into multiple groups through the first-level partition. Then, each group is partitioned into different groups through the second-level partition, which maps each group to different incremental servers.

[0027] The first-level partitioning can be done in several ways, such as partitioning by table level or record level to obtain the first mapping relationship; among which:

[0028] In the first mapping relationship obtained by dividing according to the table level, one data table corresponds to one group;

[0029] In the first mapping relationship obtained according to record level, at least one row in a data table corresponds to one group, and a data table corresponds to at least one group.

[0030] Understandably, partitioning by table level means mapping the entire data table to a group, so the entire data table can only correspond to one group. Partitioning by record level, on the other hand, means mapping one or more rows of data in a data table to a group, so a data table may correspond to multiple groups. Especially when using record level partitioning, for a specific data update operation, the data involved may be distributed across different incremental servers.

[0031] In this system, during data writing, the data to be written may be written to different incremental servers according to the data partitioning rules. During data querying, it may be necessary to perform query operations from different incremental servers, and then merge the query results returned by each incremental server into the final query result. The incremental data stored on each incremental server is unique, i.e., there is no overlap. Therefore, compared to a single incremental server system architecture, a multi-incremental server system architecture introduces the problem of data partitioning. This embodiment of the invention solves this problem by employing a two-level partitioning data partitioning rule.

[0032] In the second-level partitioning, the mapping between groups and incremental servers can be based on factors such as load. Since the data partitioning rules are not frequently changed after they are determined, there is a possibility of uneven load on multiple incremental servers. In this case, the data balance can be achieved by appropriately adjusting the second mapping relationship.

[0033] It can be understood that the system architecture adopted by the embodiment of the present application is a distributed system architecture, and reasonable data division can effectively reduce the proportion of distributed transactions. Since the incremental data is divided into multiple incremental servers, reasonable and uniform data division can improve the efficiency of data processing.

[0034] In the above system architecture, the fusion server is configured to: acquire a data division rule when receiving a data update request sent by the client; determine an incremental server involved in this data update according to data information involved in the data request and the data division rule; if the number of the incremental servers involved is N, split a physical update plan corresponding to this data update into N update sub-plans; control the N incremental servers involved to execute respective update sub-plans to implement data update; wherein the data division rule comprises a first mapping relationship between a plurality of data tables and a plurality of groups and a second mapping relationship between the plurality of groups and the plurality of incremental servers, and N is a positive integer greater than 1.

[0035] For example, referring to Figure 3a and Figure 3b The data division rule is stored in a server AS (i.e. AdminServer, which is a server managed by YaoBase to distribute data), and the fusion server SS acquires the data division rule from the server AS when receiving a data write request sent by the client. According to the data division rule, the first mapping relationship is divided according to the record level, and according to the data update request and the data division rule, it is determined that three incremental servers are involved, that is, the to-be-written data needs to be written into the three incremental servers. In order to implement the write operation, it is necessary to determine the physical update plan corresponding to the write operation, and the write operation needs to be executed by the three incremental servers, so the physical update plan is split into three update sub-plans, and then the three incremental servers are controlled to execute the corresponding update sub-plans, so that the three incremental servers implement respective data write operations, and then the to-be-written data is distributed and written into the three incremental servers.

[0036] When receiving a data update request, the fusion server can know which data needs to be written or which data needs to be read, and then according to the data division rule, it can be known which incremental servers need to store the data or which incremental servers need to obtain the data.

[0037] Of course, if the number of the incremental servers involved is one, the physical update plan does not need to be split, and the physical update plan is directly sent to the incremental server, so that the entire incremental server executes the physical update plan.

[0038] Understandably, for update statements involving multiple sets of data, such as "insert into test values(1,2),(3,4)", if the data partitioning rule is based on the table level, the updated data in this statement will be sent to the same incremental server for execution. If it is based on the record level, it may be sent to different incremental servers for execution. Therefore, before executing an update statement, the incremental servers to be used for the update operation are calculated based on the data information involved and the data partitioning rule. For those partitioned at the record level, it is determined whether the relevant data has been partitioned to different incremental servers. If it is determined that the data has been partitioned to different servers, the physical update plan needs to be split into multiple update sub-plans.

[0039] Understandably, if the fusion server receives a data query request, the process is similar, specifically: obtaining the data partitioning rules; determining the incremental servers involved in this data query based on the data information involved in the data query request and the data partitioning rules; if the number of incremental servers involved is greater than 1 (N), then splitting the physical query plan corresponding to this data query into N update query sub-plans; controlling the N incremental servers involved to execute their respective query sub-plans to realize the data query.

[0040] In practical implementation, the steps performed by the fusion server include controlling the N incremental servers involved to execute their respective corresponding update sub-plans, which may specifically include:

[0041] Determine whether each incremental server involved has started a local transaction;

[0042] If so, the corresponding update sub-plan is sent to the incremental server so that the incremental server executes the corresponding update sub-plan;

[0043] Otherwise, the transaction start flag and the corresponding update sub-plan are sent to the incremental server, so that the incremental server sends the local transaction ID to the fusion server after starting the transaction and executes the corresponding update sub-plan.

[0044] That is, the fusion server first determines whether each incremental server involved has started a local transaction. If an incremental server has started a local transaction, the fusion server has recorded the local transaction ID corresponding to the incremental server, and the fusion server directly sends the corresponding update subplan to the incremental server. The incremental server applies the update operation to the corresponding local transaction according to the local transaction ID, thereby executing the update subplan. If no local transaction has been started, the update subplan and the transaction start flag are sent to the incremental server, so that the incremental server starts a local transaction and returns the local transaction ID to the fusion server for recording, and then the incremental server executes the corresponding update subplan.

[0045] For example, referring to Figure 4 When the fusion server receives a data update request, it determines whether the data partitioning rule is according to the table level. If not, it determines whether the physical update plan needs to be split. If it needs to be split, the physical update plan is split into multiple update subplans according to the data partitioning rule. Then, for each incremental server involved, it is determined whether a local transaction has been started. If a local transaction has been started, the update subplan is directly sent to the incremental server. If no transaction has been started, the update subplan and the transaction start flag are sent to the incremental server to start a transaction and execute the update subplan, and the fusion server records the local transaction ID returned by the incremental server. If it is according to the table level, or even if it is according to the record level but only one incremental server is involved, the physical update plan does not need to be split, and the physical update plan is directly sent to the corresponding incremental server to execute the physical update plan.

[0046] The session information between the fusion server and the client is stored in the YaoSQLSessionInfo class, which can save user information, query result information, generated physical execution plan, transaction state, transaction start time, temporary variable, and other information in the session. All session information on the fusion server can be managed by the YaoSQLSessionMgr class.

[0047] The system architecture has two transaction opening modes, one is explicit opening, and the other is implicit opening. The explicit opening mode is to open by using begin statement and commit statement. After explicit opening, the fusion server records the start time and timeout time of the transaction, and then sends the command of opening the transaction to the incremental server. The incremental server initializes the transaction and returns the transaction ID to the fusion server. The fusion server saves the received transaction ID in the YaoSQLSessionInfo class, and then returns success to the client. The subsequent update operation is sent to the corresponding incremental server through the transaction ID, so as to execute the transaction corresponding to the transaction ID. The implicit opening mode is to open the transaction by setting the environment variable autocommit. After implicit opening, the command of opening the transaction is not directly sent to the incremental server, but only the value of the environment variable is marked in the YaoSQLSessionInfo class, and then success is directly returned to the client. When the first data update statement needs to be executed, the physical update plan and the opening transaction mark are sent to the incremental server. The incremental server opens the transaction and executes the plan. The incremental server does not directly commit after opening the transaction and executing the plan, but returns the transaction ID to the fusion server. The fusion server records the transaction ID, start time and timeout time. It can be understood that the above two opening modes are applicable to the case involving one incremental server.

[0048] In practice, the update operation may involve different incremental servers, so it is necessary to record the local transaction ID of multiple incremental servers. In order to reduce the proportion of distributed transactions, one of the multiple incremental servers involved needs to be selected as the coordinator server, and the other incremental servers as participant servers. If the explicit opening mode is used, because the multiple incremental servers involved are unknown, the command of opening the transaction cannot be directly sent to a certain incremental server, but only a transaction opening mark can be set by setting the environment variable. After the specific update statement is parsed by the fusion server, the data involved in the update statement is calculated according to the data division rule to obtain the incremental servers involved, and then the split update sub-plan and the transaction opening mark are sent to the corresponding incremental servers, and the local transaction ID returned by the incremental server is received.

[0049] In an actual scenario, each incremental server can specifically execute the related update sub-plan by using a two-phase commit protocol. To this end, one incremental server needs to be selected as a coordinator server in each incremental server, and each incremental server involved is a participant server. The coordinator server records list information formed by the identifiers of each participant server and the local transaction ID corresponding to each participant server; each participant server records the identifier of the coordinator server. Since the list information formed by each participant server and the local transaction ID corresponding to each participant server are recorded on the coordinator server, the coordinator server can send related information to each participant server, and each participant server also records the information of the coordinator server, so that the participant server can also send information to the coordinator server.

[0050] In a specific implementation, the coordinator server is configured to: send a preprocessing request to each participant server;

[0051] The participant server is configured to: after receiving the preprocessing request, execute a corresponding local transaction according to the update sub-plan corresponding to the participant server, and send confirmation information to the coordinator server after the transaction is executed;

[0052] The coordinator server is further configured to: after receiving the confirmation information returned by all participant servers, send a transaction commit notification to all participant servers;

[0053] The participant server is further configured to: when receiving the transaction commit notification sent by the coordinator server, send local transaction commit information corresponding to the participant server to the fusion server.

[0054] That is, the coordinator server sends a preprocessing request to each participant server, and when the participant server receives the preprocessing request, it will execute a corresponding local transaction according to its own update sub-plan, and after execution, it will not commit but send confirmation information to the coordinator server. When the coordinator server receives the confirmation information sent by all participants, it sends a transaction commit notification to all participant servers. When the participant server receives the transaction commit notification, it will locally commit the transaction, which is the same as the single-machine transaction described below.

[0055] When the current update operation only involves one incremental server, the processing logic of the transaction in the incremental server is as follows: see Figure 5, the TransExecutor on the increment server assigns the task to the TransHandlePool thread pool, which is a transaction queue used to temporarily store the pending transaction. Then the TransHandlePool assigns a thread, starts a transaction, initializes the transaction lock, and updates the memory table used to store the increment data. If the increment data of the row record reaches a certain amount, the increment data is compressed. After the processing is completed, the task is sent to the TransCommitThread thread for processing the transaction commit. The TransCommitThread thread calculates the checksum of the changed part in the transaction to ensure that the modification is correct, records the changes in the form of a log file, and then sends it to the flush_quque queue. The transaction module RWSessionCtx used to manage the read-write session information content is set to the FROZEN state to prevent the timeout thread from killing the transaction task that has already written the log. Because the transaction is done and the log may not have been flushed to the disk temporarily, the FROZEN state needs to be set to prevent the timeout thread from killing the transaction task that has already written the log. Another log synchronization thread synchronizes the log in the flush_queue queue to the standby machine. When idle or the number of logs that have been synchronized reaches a certain amount, the thread handle_flushed_log used to process the log flush flushes the synchronized log to the disk, and then sends the current transaction to the CommitEndHandlePool thread pool for processing. The CommitEndHandlePool makes the changes made by the transaction effective, releases the lock occupied by the transaction, and finally returns the execution result.

[0056] To improve the processing efficiency of distributed transactions and reduce the impact of distributed transactions on single machine transactions, the processing in the two-phase commit process is handled by a thread in the thread pool DistributedTransHandlePool. In the distributed transaction processing, when the coordinator server sends the preprocessing request, i.e., the prepare command, to the participant server, the flow of the participant server processing the prepare command is the same as that of the single machine transaction processing the prepare command, and the difference is that the processing of the prepare command is handled by a thread in the DistributedTransHandlePool thread pool. Figure 5The shown flow chart is slightly different: TransExecutor pushes the task to the DistributedTransHandlePool thread pool for processing, and still pushes into the single-thread TransCommitThread for processing when writing the preparation log, to ensure the sequence of writing the log. After writing the log, the preparation log write completion message is directly returned to the coordinator server, and no longer passes through the thread pool CommitEndHandlePool for processing, because at this time it is not determined to commit, and cannot be changed to take effect. But when the coordinator server receives all the participant server returns the preparation success message confirmation information or timeout, and has not received all the participant server returns the confirmation message, then records the transaction state in the system table as commit (when receiving all the participant server returns the confirmation information) or records the transaction state in the system table as rollback (when timeout and has not received all the participant server returns the confirmation message). Before recording the transaction state in the system table, the memory table version information needs to be checked to ensure that the transaction recorded in the system table and the current executed transaction are for the same version. If it is not the same version, write the rollback message to the system table. The coordinator server sends the transaction commit command or rollback command to the participant server according to the transaction state recorded in the system table. After receiving the transaction commit command or rollback command, the participant server records the final state of the transaction, and then processes the rollback or makes the modification effective through the thread pool CommitEndHandlePool to release the related lock. Since the isolation level is read committed, in order to ensure the consistency of the transaction, the read operation needs to use the select for update statement for explicit locking operation.

[0057] According to the two-phase commit protocol, in the first phase, the coordinator server needs to send a preprocessing request (i.e. preparation instruction) to the participant server. In the second phase, the coordinator server decides to send a transaction commit notification or rollback notification to the participant server according to the transaction state recorded in the system table. In the first phase, each participant server also needs to write its own preparation log. In the second phase, each participant server also needs to write a commit log or a rollback log. It should be noted that in the embodiment of the present application, the local transaction ID of the coordinator needs to be recorded in the preparation log as the ID of this update transaction. Then when writing the commit log or the rollback log, the local transaction ID of the coordinator is also written. After writing the commit log or the rollback log, the commit or abort operation can be executed.

[0058] In a specific implementation, if an incremental server has written a preparation log, a preparation state identifier needs to be added in a local transaction state. If a local transaction has written a preparation log, the transaction cannot be directly killed after a transaction timeout, and a system table needs to be queried or a coordinator server needs to be inquired to obtain a transaction state, and the transaction can be killed only after a transaction confirmation fails.

[0059] In addition, after a transaction is started, memory table version information corresponding to the transaction needs to be added in information returned to a fusion server, and a version number in the memory table version information is used to determine that local transactions of each incremental server are executed based on a same version of memory table, that is, all local transactions are executed on a same version of memory table.

[0060] That is, each incremental server involved in the present update is also used for returning memory table version information corresponding to a local transaction to the fusion server, and correspondingly, the fusion server is also used for determining whether local transactions of each incremental server are executed on a same version of memory table according to the memory table version information sent by each incremental server.

[0061] Based on the foregoing description, if the same version, each incremental server can write a local transaction state in a system table. If not the same version, a rollback message is written in the system table.

[0062] The foregoing description is an execution process of a system architecture in a normal case, but in an actual scenario, some abnormal situations can occur, for example, a coordinator server timeout, a participant server timeout, a downtime restart, and the like. Different processing measures are taken by the embodiments of the present application for different abnormal situations.

[0063] When an abnormal situation occurs, there can be a pending transaction, which is mainly caused by node failure or network delay or interruption in a two-phase commit process. At present, a simple timeout detection mechanism is used to detect the pending transaction. In an incremental server, a separate timeout detection thread is used to automatically detect and process a transaction that has timed out after a certain time interval. The timeout detection thread can traverse all transaction nodes by using SessionMgr, and the transaction node is an incremental server participating in an update operation. If the transaction has timed out and has not written any log, the transaction is directly aborted, and related changes are rolled back. If the transaction has written a preparation log, the transaction needs to be processed according to whether the transaction node is a coordinator server or a participant server. See Figure 6 FIG. 1 is a flowchart of a general processing procedure in a case of a coordinator server timeout.

[0064] (1) Processing of the coordinator server timeout:

[0065] In response to the timeout of the coordinator server, the coordinator server is further configured to perform the following steps:

[0066] If the coordinator server times out, it is detected whether there is an uncompleted transaction in the coordinator server;

[0067] If there is an uncompleted transaction, it is determined whether log information has been written for the detected uncompleted transaction;

[0068] If no log has been written, the operation is aborted and rolled back;

[0069] If a prepared log has been written, the receipt of the confirmation information returned by the participant servers is stopped; it is determined whether all the confirmation information returned by the participant servers has been received;

[0070] If all the confirmation information returned by the participant servers has been received, the committed transaction state is written into the system table, and the transaction commit notification is sent to each participant server, so that each participant server sends the local transaction commit information to the fusion server.

[0071] That is, if the timeout occurs, it is determined whether the transaction of the coordinator server is completed, and if not, the coordinator server has an uncompleted transaction. When it is determined that there is an uncompleted transaction, it is determined whether log information has been written for the uncompleted transaction. If no task log has been written, the operation can be directly aborted and rolled back. If a prepared log has been written, it indicates that the first phase of the coordinator server has been executed, at this time the receipt of the confirmation information returned by the participant servers is stopped, and it is determined whether all the confirmation information returned by the participant servers has been received. If all the confirmation information returned by the participant servers has been received, and at the same time the coordinator server as one of the participants also completes the first phase, the transaction state written into the system table is committed, and each participant server is notified, so that each participant server commits the local transaction information.

[0072] Of course, the coordinator server can also be configured to, if all the confirmation information returned by the participant servers is not received, write the rolled back transaction state into the system table, and send the rollback notification to each participant server, so that each participant server sends the rollback information to the fusion server.

[0073] That is, if all the confirmation information returned by the participant servers is not received, the transaction state written into the system table is rolled back, and each participant server is notified.

[0074] As can be seen, the coordinator server will record the whole transaction state of the current update in the system table only after all participant servers confirm that the execution of the update sub-plan is successful. When the transaction state is commit, it indicates that the execution of the update sub-plan of each participant server is successful. If the transaction state is rollback, it indicates that at least one participant server fails to execute the update sub-plan or the in-memory table version is inconsistent or other conditions.

[0075] In summary, when the coordinator server times out, it stops receiving the confirmation messages returned by other participants first, and then detects whether all the confirmation messages are received. If the confirmation messages of all participant servers are received, the transaction state is recorded as commit in the system table. Otherwise, it is recorded as rollback. Since the in-memory table version is also confirmed before writing in the system table, if the transaction state record is not the same version as the transaction update part, the transaction state is recorded as rollback. Finally, according to the record in the system table, the commit or rollback command is sent to all participant servers. In this process, the system table is not detected first because it will increase two network communications. In order to ensure that the records of multiple writings are the same, the reception of the confirmation messages returned by the participant servers is stopped first, so that the inconsistent state of not receiving all the replies in the first time and receiving all the replies in the second time can be avoided.

[0076] (2) Referring to Figure 7 , the processing of the participant server timeout:

[0077] The participant server is further configured to perform the following steps:

[0078] If the participant server times out or finishes replaying the local log information after a restart, the participant server detects whether there is an uncompleted transaction;

[0079] If there is an uncompleted transaction, the participant server queries the transaction state of the uncompleted transaction from the system table for the uncompleted transaction;

[0080] If the corresponding transaction state is queried, the participant server processes according to the queried transaction state;

[0081] If the corresponding transaction state is not queried, the participant server judges whether the coordinator server is in a normal state;

[0082] If the coordinator server is in the normal state, the participant server waits for the next timeout processing;

[0083] If the coordinator server is not in the normal state, the participant server confirms again whether the corresponding transaction state is recorded in the system table;

[0084] If the corresponding transaction state is not recorded, the participant server writes the transaction state of rollback in the system table, and ends the corresponding local transaction;

[0085] If the transaction state corresponding to the query is recorded, the corresponding processing is performed according to the transaction state queried.

[0086] That is, when the participant server times out, it is detected whether the local transaction on the participant server is processed or not. If the local transaction is not processed, it is considered that there is an unprocessed transaction. The overall transaction state of the current update operation is queried in the system table for the unprocessed transaction. If the transaction state can be queried, commit or rollback processing is performed according to the transaction state queried. If the transaction state cannot be queried, it is judged whether the coordinator server is normal or not. If the coordinator server is normal, the next timeout processing is waited for. During the waiting process, the commit or rollback command sent by the coordinator server can be received. If the command is received, the command can be executed. If the coordinator server is not normal, it is further judged whether the transaction state is recorded in the system table or not. If the transaction state is not recorded, the rollback is written into the system table. If the transaction state exists at this time, commit or rollback is performed according to the transaction state.

[0087] It can be seen that in the above process, the record of the coordinator is mainly used. If the coordinator does not exist, the entire transaction rollback can be written into the system table, so that the coordinator is reselected.

[0088] In addition to the above steps performed by the participant server when it times out, the above steps can also be performed after the local log information is replayed after the participant server is restarted.

[0089] For the situation after the restart, the following processing can also be performed:

[0090] Any incremental server can also be used to replay the local log information after the restart, perform the corresponding update operation according to the preparation log if the preparation log appears during the replaying process, commit the transaction locally if the commit log appears after the preparation log, and rollback the transaction locally if the rollback log appears after the preparation log.

[0091] That is, if the preparation log appears during the replaying process, the corresponding update operation is performed according to the preparation log, but no commit operation is performed. When the commit log or the rollback log appears after the preparation log, commit or rollback is performed. After all the logs are replayed, if there are still unmatched preparation logs, the processing flow of the participant server timeout in (2) above can be used to process each transaction.

[0092] It can be seen that, in the log playback, the incremental server adopts the process of multi-thread playback and single-thread submission. For distributed transactions, the log is divided into two parts. In the playback to the preparation log, only the relevant data is written into the uncommitted list, and the subsequent submission part is not done, and the log information is saved, waiting for subsequent processing. When the corresponding submission log or rollback log is played back, the corresponding submission or abortion operation is performed. If the log is completely played back, there are still unmatched preparation logs, and the process of pending transactions after log playback is entered.

[0093] In the version freeze, a log playback point is established, which prevents the processing of new transactions and quickly ends the transactions being executed. At this time, the ending of the transaction being executed is basically the same as the timeout processing, and the difference is that the transaction is directly aborted without checking whether the transaction is processed for a timeout. The transaction with the operation log is processed in the same way as the above timeout thread, and the processing of the distributed transaction is also the same.

[0094] In the above, the system table submitted is specially set for distributed transactions, and the participant server needs to obtain the transaction state in abnormal conditions, so it is necessary to record the transaction state of the distributed transaction globally. In order to facilitate the write and read operations of the coordinator server and the participant server in the distributed transaction, the final state of the distributed transaction is recorded in a new system table. The specific structure of the system table is briefly designed as shown in Table 1:

[0095] Table 1

[0096] Field (attribute) Type Nullable(whether it can be nullable, 0 for no) default start_time int 0 null server_ip varchar(32) 0 null server_port int 0 null commit_stat bool 0 null group_info varchar(1024) 0 null

[0097] As shown in Table 1, the transaction state is recorded on the system table one by one, using the timestamp of the start of the transaction on the coordinator server, the IP address and port PORT of the coordinator server as the joint primary key of the record, marking the transaction submission or rollback state by commit_stat, and identifying the group information involved in the distributed transaction by group_info. Since group migration may be required later, the proportion of distributed transactions is reduced.

[0098] It can be understood that in the embodiment of the application, a fusion server and a plurality of incremental servers are included in the distributed transaction processing system, each incremental server is used for storing incremental data, and the incremental data stored by the plurality of incremental servers does not overlap; when the fusion server receives a data update request sent by a client, a data division rule is acquired; according to data information involved in the data request and the data division rule, an incremental server involved in this time of data update is determined; if the number of the incremental servers involved is multiple, a physical update plan corresponding to this time of data update is split into a plurality of update sub-plans; the plurality of incremental servers involved are controlled to execute respective corresponding update sub-plans, and data update is realized. Since a plurality of incremental servers are provided in the embodiment of the application, data can be orderly divided in the plurality of incremental servers through a data division rule, when high-concurrency read-write transactions come in a cluster, the single-point load of a single incremental server can be reduced, the processing efficiency of incremental data is improved, and the overall performance of the distributed database is improved.

[0099] The following experiments are performed on the distributed processing system.

[0100] Experimental environment: YaoBase server cluster deployment, the server is composed of a 1T SSD hard disk, 256GB memory, 64-core CPU, and one network card. The operating system of the server is Kylin Linux Advanced Server release V10 (Sword). The configuration information is shown in Table 2.

[0101] Table 2

[0102]

[0103] The six servers node1-node6 in Table 2 provide database services and build a YaoBase distributed cluster. We construct three kinds of distributed clusters, architecture one is a single incremental server architecture, architecture two and architecture three correspond to distributed architectures of 3 incremental servers and 5 incremental servers respectively, and node7 is used as an independent test node to run the test program. The role allocation of each node under the three architectures is shown in Table 3.

[0104] Table 3

[0105]

[0106] Experimental data: The experiment uses sysbench for benchmark testing, and the experimental data parameter settings are as follows: test table size: table_size = 1000000 (100W); test table number: tables = 30 (so the data volume is estimated as: 1W row = 2.4MB, 100*30*2.4M = 7GB); test thread number: threads = 32 / 64 / 128 / 256 / 512 / 1024; test report time: report-interval = 10 (report the result every 10 seconds); test time: time = 60 (each test for 60 seconds).

[0107] Experiment one

[0108] Experiment one uses sysbench test tool to test the read-only (read_only) scenario of YaoBase cluster under three architectures. With the increase of thread number, the TPS (Transaction Per Second), QPS (Query Per Second) and delay time performance of YaoBase under three different architectures are shown in the following table 4.

[0109] Table 4

[0110]

[0111]

[0112] Experimental analysis: The experimental results show that after applying multi-TS architecture, under the same thread number condition, the TPS and QPS of YaoBase are greatly improved, and the delay time is greatly reduced. The performance of non-transaction scenario is significantly improved. Because the single TS is transformed into multi-TS architecture, the incremental data is divided into different TS, and multiple TS provides readable service at the same time, which solves the single point bottleneck of single TS and significantly improves the read and write performance of the system.

[0113] Experiment two

[0114] Experiment two uses sysbench test tool to test the write-only (write_only) scenario of YaoBase cluster under three architectures. With the increase of thread number, the TPS, QPS and delay time performance of YaoBase under three different architectures are shown in the following table 5.

[0115] Table 5

[0116]

[0117]

[0118] Experimental analysis: The experimental results show that after applying the multi-TS architecture, the TPS and QPS of YaoBase have a certain degree of decline, and the delay time also increases under the same thread number condition. This is because the single TS is transformed into the multi-TS architecture, and the incremental data is divided into different TSs. For the extreme scenario of only writing, a large number of distributed transactions need to be processed. Compared with the single TS architecture, processing distributed transactions means that more network transmission costs need to be consumed. Although the data division strategy has been used to minimize distributed transactions, distributed transactions of multiple nodes are inevitable, so the performance of the distributed YaoBase for the only write scenario is reduced to a certain extent compared with the single TS.

[0119] Experiment three

[0120] Experiment three uses the sysbench test tool to perform non-transaction stress testing on the read-write mixed (read_rite) scenario of YaoBase clusters under three architectures. As the number of threads increases, the TPS, QPS and delay time performance of YaoBase under three different architectures are shown in Table 6.

[0121] Table 6

[0122]

[0123]

[0124] Experimental analysis: The experimental results show that after applying the multi-TS architecture, the TPS and QPS of YaoBase have a certain degree of decline, and the delay time also increases under the same thread number condition. This is because the single TS is transformed into the multi-TS architecture, and the incremental data is divided into different TSs. For the extreme scenario of only writing, a large number of distributed transactions need to be processed. Compared with the single TS architecture, processing distributed transactions means that more network transmission costs need to be consumed. Although the data division strategy has been used to minimize distributed transactions, distributed transactions of multiple nodes are inevitable, so the performance of the distributed YaoBase for the only write scenario is reduced to a certain extent compared with the single TS.

[0125] Through the above experiments, it is shown that the present application proposes a multi-TS transaction processing strategy, which improves the write efficiency of incremental data by dividing the data in multiple incremental servers in an orderly manner, thereby improving the overall performance of the distributed database.

[0126] In a second aspect, an embodiment of the present application provides a distributed transaction processing method, which is implemented based on the distributed transaction processing system provided in the first aspect, and the method comprises the following steps:

[0127] The fusion server acquires a data division rule when receiving the data update request sent by the client; determines an incremental server involved in the current data update according to data information involved in the data request and the data division rule; if the number of the incremental servers involved is N, splits a physical update plan corresponding to the current data update into N update sub-plans; controls the N incremental servers involved to execute respective update sub-plans, and realizes data update; wherein the data division rule comprises a first mapping relationship between a plurality of data tables and a plurality of groups and a second mapping relationship between the plurality of groups and the plurality of incremental servers, and N is a positive integer greater than 1.

[0128] In some embodiments, the fusion server controls the N incremental servers involved to execute respective update sub-plans, specifically comprising:

[0129] judging whether each incremental server involved has started a local transaction;

[0130] if yes, sending the corresponding update sub-plan to the incremental server, so that the incremental server executes the corresponding update sub-plan;

[0131] otherwise, sending a transaction start flag and the corresponding update sub-plan to the incremental server, so that the incremental server returns a local transaction ID to the fusion server after starting a transaction and executes the corresponding update sub-plan.

[0132] In some embodiments, the first mapping relationship is a mapping relationship obtained by dividing according to table level or record level; wherein:

[0133] in the first mapping relationship obtained by dividing according to table level, one data table corresponds to one group;

[0134] in the first mapping relationship obtained by dividing according to record level, at least one row in one data table corresponds to one group, and one data table corresponds to at least one group.

[0135] In some embodiments, one of the plurality of incremental servers is a coordinator server, and each of the incremental servers involved is a participant server; the coordinator server records list information formed by the identifiers of each participant server and the local transaction ID corresponding to each participant server; each participant server records the identifier of the coordinator server;

[0136] Correspondingly, the process of the incremental server executing the update sub-plan comprises:

[0137] the coordinator server sends a preprocessing request to each participant server;

[0138] The participant server executes a corresponding local transaction according to the update sub-plan corresponding to the participant server after receiving the preprocessing request, and sends confirmation information to the coordinator server after the transaction is executed;

[0139] The coordinator server sends a transaction commit notification to all participant servers after receiving the confirmation information returned by all participant servers;

[0140] The participant server sends local transaction commit information corresponding to the participant server to the fusion server when receiving the transaction commit notification sent by the coordinator server.

[0141] In some embodiments, the method further comprises the following steps performed by the coordinator server:

[0142] If the coordinator server times out, it is detected whether there is an outstanding transaction in the coordinator server;

[0143] If there is an outstanding transaction, it is determined whether log information has been written for the detected outstanding transaction;

[0144] If no log has been written, the operation is aborted and rolled back;

[0145] If the preparation log has been written, the reception of the confirmation information returned by the participant server is stopped, and it is determined whether all the confirmation information returned by the participant servers has been received;

[0146] If all the confirmation information returned by the participant servers has been received, the commit transaction state is written to the system table, and the transaction commit notification is sent to each participant server, so that each participant server sends the local transaction commit information to the fusion server.

[0147] In some embodiments, the method further comprises: if the coordinator server does not receive all the confirmation information returned by the participant servers, it writes the rollback transaction state to the system table, and sends a rollback notification to each participant server, so that each participant server sends rollback information to the fusion server.

[0148] In some embodiments, the method further comprises the following steps performed by the participant server:

[0149] If the participant server times out or replays the local log information after a restart, it is detected whether there is an outstanding transaction in the participant server;

[0150] If there is an outstanding transaction, the transaction state of the outstanding transaction is queried from the system table for the outstanding transaction;

[0151] If the corresponding transaction state is found, corresponding processing is performed according to the found transaction state;

[0152] If the corresponding transaction state is not found, it is determined whether the coordinator server is in a normal state;

[0153] If the coordinator server is in the normal state, the next timeout processing is waited for;

[0154] If the coordinator server is not in the normal state, it is determined again whether the corresponding transaction state is recorded in the system table;

[0155] If the corresponding transaction state is not recorded, a rolled-back transaction state is written into the system table, and the corresponding local transaction is ended;

[0156] If the corresponding transaction state is recorded, corresponding processing is performed according to the found transaction state.

[0157] In some embodiments, the method further comprises:

[0158] Any one of the incremental servers replays local log information after a restart; if a preparation log appears in the replay process, corresponding update operation is performed according to the preparation log; if a commit log appears after the preparation log, the local transaction commit information is sent to the fusion server; if a rollback log appears after the preparation log, the rollback information of the local transaction is sent to the fusion server.

[0159] In some embodiments, the method further comprises:

[0160] Each incremental server involved in the current update returns memory table version information corresponding to the local transaction to the fusion server;

[0161] The fusion server determines whether the local transactions of each incremental server are executed on the same version of the memory table according to the memory table version information sent by each incremental server.

[0162] It can be understood that the method provided by the second aspect corresponds to the system provided by the first aspect, and the related content explanation, implementation mode, example, beneficial effect and the like of the method can refer to the corresponding part in the first aspect, and will not be repeated here.

[0163] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A distributed transaction processing system, characterized in that, include: A client-facing fusion server and multiple incremental servers communicatively connected to the fusion server, each incremental server having multiple corresponding backup servers, each incremental server storing incremental data, and the incremental data stored by the multiple incremental servers not overlapping; wherein: The fusion server is configured to: upon receiving a data update request from the client, obtain data partitioning rules; determine the incremental servers involved in this data update based on the data information involved in the data request and the data partitioning rules; if the number of incremental servers involved is N, then split the physical update plan corresponding to this data update into N update sub-plans; control the N incremental servers involved to execute their respective update sub-plans to realize the data update; wherein, the data partitioning rules include a first mapping relationship between multiple data tables and multiple groups, and a second mapping relationship between the multiple groups and the multiple incremental servers, and N is a positive integer greater than 1; The steps performed by the fusion server include: controlling the N incremental servers involved to execute their respective corresponding update sub-plans, specifically including: Determine whether each incremental server involved has started a local transaction; If so, the corresponding update sub-plan is sent to the incremental server so that the incremental server executes the corresponding update sub-plan; Otherwise, the transaction start flag and the corresponding update sub-plan are sent to the incremental server so that the incremental server returns the local transaction ID to the fusion server after starting the transaction and executes the corresponding update sub-plan.

2. The system according to claim 1, characterized in that, The first mapping relationship is a mapping relationship obtained by dividing according to the table level or the record level; wherein: In the first mapping relationship obtained by dividing according to the table level, one data table corresponds to one group; In the first mapping relationship obtained according to record level, at least one row in a data table corresponds to one group, and a data table corresponds to at least one group.

3. The system according to claim 1, characterized in that, One of the multiple incremental servers is the coordinator server, and the other incremental servers involved are the participant servers. The coordinator server records a list of identifiers of each participant server and the local transaction ID corresponding to each participant server. Each participant server records the identifier of the coordinator server. The coordinator server is used to: send preprocessing requests to each participant server; The participant server is configured to: upon receiving the preprocessing request, execute the corresponding local transaction according to the update sub-plan corresponding to the participant server, and send confirmation information to the coordinator server after the transaction is completed; The coordinator server is also used to: send a transaction commit notification to all participant servers after receiving confirmation information from all participant servers; The participant server is further configured to: upon receiving a transaction commit notification from the coordinator server, send the local transaction commit information corresponding to the participant server to the fusion server.

4. The system according to claim 3, characterized in that, The coordinator server is also used to perform the following steps: If the coordinator server times out, check if there are any pending transactions on the coordinator server. If there are pending transactions, determine whether log information has been written for the detected pending transactions; If no logs have been written yet, the process will be stopped and a rollback will be performed. If a preparation log has already been written, then stop receiving confirmation information returned by the participant servers and determine whether confirmation information returned by all participant servers has been received. If confirmation information has been received from all participant servers, the committed transaction status is written to the system table, and the transaction commit notification is sent to each participant server so that each participant server sends the partial transaction commit information to the fusion server.

5. The system according to claim 4, characterized in that, The coordinator server is also configured to: if it does not receive confirmation information from all participant servers, write the rollback transaction status to the system table and send a rollback notification to each participant server so that each participant server sends rollback information to the fusion server.

6. The system according to claim 5, characterized in that, The participant server is also used to perform the following steps: If the participant server times out or finishes replaying the local log information after a crash and restart, then check whether the participant server has any pending transactions. If there are pending transactions, then for the pending transactions, query the system table to find the transaction status of the pending transactions; If a corresponding transaction status is found, then the corresponding processing will be carried out according to the found transaction status; If no corresponding transaction status is found, determine whether the coordinator server is in a normal state. If the status is normal, wait for the next timeout to be processed; If the status is abnormal, then reconfirm whether the corresponding transaction status is recorded in the system table. If no corresponding transaction status is recorded, the rollback transaction status is written to the system table, and the corresponding local transaction is terminated. If a corresponding transaction status is recorded, then the corresponding processing will be carried out according to the queried transaction status.

7. The system according to claim 1, characterized in that, Any incremental server is also used to: replay local log information after a crash and restart; if preparation logs appear during the replay process, perform corresponding update operations based on the preparation logs; If a commit log appears after the preparation log, a partial transaction commit message is sent to the fusion server. If a rollback log appears after the preparation log, then rollback information for the partial transaction is sent to the fusion server.

8. The system according to claim 1, characterized in that, Each incremental server involved in this update is also used to: return the memory table version information corresponding to the local transaction to the fusion server; Correspondingly, the fusion server is also used to: determine whether the local transactions of each incremental server are executed for the same version of the memory table based on the memory table version information sent by each of the incremental servers involved.

9. A distributed transaction processing method, characterized in that, Based on the distributed transaction processing system according to any one of claims 1 to 8, the method includes: When the fusion server receives a data update request from the client, it obtains the data partitioning rules; based on the data information involved in the data request and the data partitioning rules, it determines the incremental servers involved in this data update; if the number of incremental servers involved is N, it splits the physical update plan corresponding to this data update into N update sub-plans; it controls the N incremental servers involved to execute their respective update sub-plans to realize the data update; wherein, the data partitioning rules include a first mapping relationship between multiple data tables and multiple groups and a second mapping relationship between the multiple groups and the multiple incremental servers, and N is a positive integer greater than 1.

Images