Management Method, Device, Equipment and Medium of Transaction Processing System

Abstract

The present application provides a management method for a transaction processing system. The transaction processing system includes multiple participant nodes, each participant node stores a data partition, and each participant node is used to process multiple transactions. Among them, the data partition combination accessed by each transaction includes at least one data partition. The method includes: obtaining the number of transactions accessing each data partition combination within a period, and adjusting the storage of data partitions among multiple participant nodes according to the number of transactions accessing each data partition combination within the period. This method can reduce the number of participant nodes participating in the transaction, thereby reducing the number of interactions between the coordinator node and the participant nodes, shortening the latency of transaction submission, and improving the performance of transaction submission.

Description

[0001] This application claims the priority of a Chinese patent application with the application number 202010197554.0 and the application title "A Transaction Processing System" filed with the China National Intellectual Property Administration on March 19, 2020. The entire content thereof is incorporated herein by reference. Technical Field

[0002] This application relates to the field of computer technology, and in particular to a management method, apparatus, device, and computer-readable storage medium for a transaction processing system. Background Art

[0003] A transaction is a logical unit during the execution of a database management system, including a finite sequence of database operations. In a distributed system, when a transaction spans multiple participants for execution, each participant can determine the execution result of the transaction it executes, but does not know the execution results of other participants. To maintain the atomicity, consistency, isolation, and durability of the transaction, a coordinator needs to be introduced to manage the participants executing the transaction.

[0004] Two-phase commit is a method designed for transaction submission based on the system architecture of a distributed system. However, in this method, multiple interactions are required between the coordinator and the participants, resulting in a longer latency for transaction submission and a decline in transaction submission performance. Summary of the Invention

[0005] The embodiments of this application provide a management method for a transaction processing system, which solves the problem in the related art that multiple interactions are required between the coordinator and the participants, resulting in a longer latency for transaction submission and a decline in transaction submission performance. The embodiments of this application also provide a corresponding apparatus, device, computer-readable storage medium, and computer program product for the above method.

[0006] In a first aspect, the present application provides a management method for a transaction processing system. The transaction processing system includes multiple participant nodes, each participant node stores a data partition, and each participant node is used to process multiple transactions. Among them, the data partition combination accessed by each transaction includes at least one data partition. The transaction processing system also includes a coordinator node. The management method of the transaction processing system can be executed by the above-mentioned coordinator node. In some implementation manners, the transaction processing system may further include a router node for implementing read-write separation. Based on this, the management method of the transaction processing system can also be executed by the above-mentioned router node. In some implementation manners, the management method of the transaction processing system can also be executed by a node outside the transaction processing system. In the embodiments of the present application, the node that executes the management method of the transaction processing system is collectively referred to as a management node.

[0007] Specifically, the management node obtains the number of transactions accessing each data partition combination within a period. Among them, each data partition combination may be a combination of all data partitions stored in the transaction processing system, or a combination of some data partitions stored in the transaction processing system. Then, the management node adjusts the storage of the data partitions in the multiple participant nodes according to the number of transactions accessing each data partition combination within the period, so as to reduce the number of participant nodes participating in the transaction. In this way, the number of interactions between the coordinator node and the participant nodes can be reduced, the latency of transaction submission can be reduced, and the performance of transaction submission can be improved.

[0008] In some implementation manners, the management node may collect the number of transactions accessing each data partition combination within the period. In this way, the management node can adjust the storage of the data partitions in the multiple participant nodes according to the number of transactions obtained in real time. Since the number of transactions collected in real time is relatively accurate, the data partitions that are likely to be accessed together can be adjusted to one participant node, with a relatively high adjustment accuracy.

[0009] In some implementation manners, the management node may obtain the number of transactions accessing each data partition combination within a historical period, and predict the number of transactions accessing each data partition combination within the period according to the number of transactions accessing each data partition combination within the historical period. In this way, when the management node adjusts the storage of the data partitions in the participant nodes, it can make an adjustment in advance according to the prediction result, so as to shorten the latency of transaction submission within the period and improve the performance of transaction submission.

[0010] In some implementation manners, the management node may, according to the number of transactions accessing each data partition combination within the historical period, use any one of an autoregressive integrated moving average model, an exponential smoothing model, a nonparametric time series model, a prophet model, and a time series model based on deep learning to predict the number of transactions accessing at least one data partition combination within the period.

[0011] According to different business scenarios, the management node can adopt different models to predict the number of transactions accessing each data partition combination within the period. For example, for business scenarios such as sales forecasting, traffic flow forecasting, hospital patient number forecasting, etc., the management node can adopt an autoregressive integrated moving average (ARIMA) model for prediction. Another example is that for business scenarios such as stock volatility forecasting, factory production forecasting, etc., the management node can adopt an exponential smoothing model to predict the number of transactions accessing each data partition combination within the period. Of course, for the business scenario of sales forecasting, the management node can also adopt an exponential smoothing model for prediction.

[0012] For business scenarios such as sales forecasting and intermittent product sales forecasting, etc., the management node can also adopt a non-parametric time series (NPTS) model to predict the number of transactions accessing each data partition combination within the period based on the number of transactions accessing each data partition combination in the historical period. It should be noted that the management node can also adopt a related variant model of NPTS to predict the number of transactions accessing each data partition combination within the period.

[0013] For business scenarios such as climate forecasting, holiday traffic forecasting, transaction data forecasting, etc., the management node can also adopt a prophet model for prediction. For business scenarios such as climate forecasting, store visit volume forecasting, product promotion forecasting, and sales forecasting, the management node can also adopt a deep learning-based time series model (deep AR) for prediction.

[0014] In some implementation manners, the management node can determine a target data partition combination according to the number of transactions accessing each data partition combination within the period, and migrate the data partitions in the target data partition combination from N participating nodes to M participating nodes, where N is greater than M.

[0015] Among them, the target data partition combination can be one or more of the data partition combinations accessed within the period. Specifically, when the number of transactions accessing the data partition combination within the period reaches a preset quantity, the management node can determine this data partition combination as the target data partition combination. In some implementation manners, the management node can also determine the proportion of the number of transactions accessing the data partition combination within the period to the total number of transactions accessing each data partition combination. When this proportion is greater than a preset ratio, then this data partition combination is determined as the target data partition combination.

[0016] The target data partition combination can also be one or more of the sub - combinations of the data partition combinations accessed during the period. Specifically, when the number of transactions of the accessed sub - combination during the period reaches a preset quantity, the management node can determine this sub - combination as the target data partition combination. In some implementation manners, the management node can also determine the proportion of the number of transactions of the accessed sub - combination during the period relative to the total number of transactions of all data partition combinations. When this proportion is greater than a preset ratio, the data partition combination is determined as the target data partition combination.

[0017] In some implementation manners, considering that if a node is overloaded, it will lead to a performance decline, the management node can adjust the storage of the data partitions among the multiple participant nodes according to the number of transactions of each data partition combination accessed during the period and the load of the participant nodes.

[0018] Specifically, the management node can obtain the load of each participant node, and then migrate each data partition in the target data partition combination from N participant nodes to M participant nodes with a smaller load. To reduce the overhead generated by the migration operation, the management node can migrate each data partition in the target data partition combination from N participant nodes to M participant nodes with a smaller load among these N nodes.

[0019] In a second aspect, the present application provides a management device for a transaction processing system. The transaction processing system includes multiple participant nodes, each participant node stores data partitions, and each participant node is used to process multiple transactions. Among them, the data partition combination accessed by each transaction includes at least one data partition; the device includes:

[0020] A communication module, used to obtain the number of transactions of each data partition combination accessed during the period;

[0021] A management module, used to adjust the storage of the data partitions among the multiple participant nodes according to the number of transactions of each data partition combination accessed during the period.

[0022] In some implementation manners, the device further includes:

[0023] An acquisition module, used to acquire the number of transactions of each data partition combination accessed during the period.

[0024] In some implementation manners, the communication module is specifically used for:

[0025] Obtaining the number of transactions of each data partition combination accessed during a historical period;

[0026] The device further includes:

[0027] A prediction module, configured to predict the number of transactions accessing each data partition combination within the period according to the number of transactions accessing each data partition combination within the historical period.

[0028] In some implementation manners, the management module is specifically configured to:

[0029] Determine a target data partition combination according to the number of transactions accessing each data partition combination within the period;

[0030] Migrate the data partitions in the target data partition combination from N participating nodes to M participating nodes, where N is greater than M.

[0031] In some implementation manners, the management module is specifically configured to:

[0032] Adjust the storage of the data partitions among the multiple participating nodes according to the number of transactions accessing each data partition combination within the period and the loads of the participating nodes.

[0033] In a third aspect, the present application provides a device, which includes a processor and a memory. The processor and the memory communicate with each other. The processor is configured to execute instructions stored in the memory, so that the device executes the management method of the transaction processing system in the first aspect or any implementation manner of the first aspect.

[0034] In a fourth aspect, the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions run on a device, the device is caused to execute the management method of the transaction processing system described in the first aspect or any implementation manner of the first aspect.

[0035] In a fifth aspect, the present application provides a computer program product including instructions. When the computer program product runs on a device, the device is caused to execute the management method of the transaction processing system described in the first aspect or any implementation manner of the first aspect.

[0036] Based on the implementation manners provided in the above aspects of the present application, further combinations can be made to provide more implementation manners. Description of the Drawings

[0037] To more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below.

[0038] Figure 1 It is a system architecture diagram of a distributed system provided by an embodiment of the present application;

[0039] Figure 2 It is a flowchart of a two-phase commit provided by an embodiment of the present application;

[0040] Figure 3 A schematic diagram of submission or rollback provided by an embodiment of the present application;

[0041] Figure 4 A flowchart of a management method for a transaction processing system provided by an embodiment of the present application;

[0042] Figure 5 A schematic diagram of managing data partitioning of a transaction processing system provided by an embodiment of the present application;

[0043] Figure 6 A schematic diagram of managing data partitioning of a transaction processing system provided by an embodiment of the present application;

[0044] Figure 7 A schematic diagram of managing data partitioning of a transaction processing system provided by an embodiment of the present application;

[0045] Figure 8 A schematic diagram of migrating a partition provided by an embodiment of the present application;

[0046] Figure 9 A schematic diagram of a transaction partition diagram provided by an embodiment of the present application;

[0047] Figure 10 A schematic structural diagram of a management device for a transaction processing system provided by an embodiment of the present application;

[0048] Figure 11 A schematic structural diagram of a device provided by an embodiment of the present application. Detailed implementation manners

[0049] The terms "first" and "second" in the embodiments of the present application are only for descriptive purposes, and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features.

[0050] First, some concepts involved in the embodiments of the present application are introduced.

[0051] A transaction is a logical unit during the execution of a database management system, which includes a finite sequence of database operations. To ensure the correct and reliable data, a transaction must have four characteristics: atomicity, consistency, isolation, and durability.

[0052] Among them, atomicity means that all operations in a transaction are either all completed or all not completed, and will not end at some intermediate link. If an error occurs during the execution of a transaction, it will be rolled back to the state before the start of the transaction. Consistency means that the integrity of the database is not damaged before the start and after the end of the transaction. Isolation means that the database allows multiple concurrent transactions to read, write, and modify data simultaneously. Isolation can prevent data inconsistency caused by cross-execution when multiple transactions are executed concurrently. Durability means that after the execution of a transaction, the modification to the data is permanent and will not be lost even in case of system failure.

[0053] The coordinator plays a coordinating role during the execution of a transaction. In a distributed system, when a transaction spans multiple participants (i.e., the transaction requests included in a transaction are executed by multiple participants), that is, when the transaction is a distributed transaction, each participant can determine the result of its own execution of the transaction request, but does not know the results of the execution of the transaction requests by other participants. In order to maintain the above transaction characteristics, a coordinator is introduced to manage the results of the execution of transaction requests and indicate whether the participants should commit the results of the execution of the transaction requests, such as replacing the original data with the updated data.

[0054] The participant plays the role of an executor during the execution of a transaction, and is used to receive the transaction requests sent by the coordinator and perform specified operations on the specified data in the transaction requests. Among them, a transaction includes transaction requests for one or more pieces of data, and the transaction requests are read requests or write requests.

[0055] For the sake of easy understanding, the following uses the specific example of bank transfer to illustrate transactions, participants, and coordinators. In this example, when bank account X in place A transfers M yuan to bank account Y in place B, an operation of subtracting M (which can be called operation 1 for convenience of description) needs to be performed on account X, and an operation of adding M (which can be called operation 2 for convenience of description) needs to be performed on account Y. Since the bank account data in place A and the bank account data in place B are usually stored in different nodes, these two different nodes are the participants involved in the execution of the bank transfer transaction. The node that sends the transaction requests (write requests in this example) to these two nodes is the coordinator.

[0056] Due to the ACID characteristics of the transaction, when operation 1 is successfully executed while operation 2 is not, the participant corresponding to operation 1 can roll back operation 1, so that both operation 1 and operation 2 are either successfully executed or both fail. That is, a series of operations included in the transaction can be safely and correctly executed under any circumstances.

[0057] Exemplarily, the transaction processing system is a distributed system, which includes one or more business nodes, one or more coordinator nodes, and multiple participant nodes. Among them, the business nodes are nodes related to the business, the coordinator nodes are the nodes acting as coordinators, and the participant nodes are the nodes acting as participants. These nodes can be physical machines, or virtual machines, containers, etc. deployed on physical machines. Among them, the physical machine is specifically a device with data processing capabilities, such as a terminal or a server. Figure 1 Taking a distributed system including one business node, one coordinator node, and multiple participant nodes as an example. The coordinator node is used to receive the transaction request sent by the business node and send the transaction request to one or more participant nodes for execution.

[0058] In some implementation manners, the coordinator node can be independent. That is, the coordinator node does not execute the transaction request and only coordinates the participant nodes that execute the transaction. In other implementation manners, the coordinator node can also act as a participant node. That is, the coordinator node can also execute the transaction request. Moreover, a node can act as a coordinator node during the process of processing one transaction and can also act as a participant node during the process of processing another transaction. That is to say, a node can be both a coordinator node and a participant node.

[0059] Each participant node stores at least one data partition. Each participant node is used to process multiple transactions. Among them, the data partition combination accessed by each transaction includes at least one data partition. The data partition refers to multiple blocks obtained by splitting the data table. These blocks logically form a complete data table, but are multiple physical blocks at the bottom layer. The data partition does not result in the generation of a new data table, but distributes the data in the data table to different storage media, such as different hard disks, systems, and servers. In this way, the data retrieval efficiency can be improved, and the frequent input / output (IO) pressure value of the database can be reduced.

[0060] According to different ways of partitioning the data table, data partitioning can be divided into horizontal partitioning and vertical partitioning. Among them, horizontal partitioning divides the rows of the data table. All columns defined in the data table can be found in each data partition, and the characteristics of the data table are maintained. For example, a data table including ten-year social security records can be divided into ten different data partitions by year, and each data partition includes the social security records of one of those years. Vertical partitioning divides the columns of the data table, thereby reducing the width of a single data table. For example, for a data table including large text columns and / or binary large object (BLOB) columns, the access frequency of these text and BLOB columns is relatively low, so they can be divided into another data partition, which can improve the access speed while ensuring data relevance.

[0061] In Figure 1 the example of, participant node 1 includes at least data partitions P11, P12, and P13, participant node 2 includes at least data partitions P21, P22, and P23, and participant node 3 includes at least data partitions P31, P32, and P33. Figure 1 Taking data partitioning as vertical partitioning as an example for illustration. For example, when the data table is the salary record table of an enterprise's employees, P11, P12, and P13 can be employee ID, year, and payment month, P21, P22, and P23 are basic salary, performance bonus, and deductions for five social insurances and one housing fund. P31, P32, and P33 are gross salary, tax, and net salary.

[0062] When paying salaries, financial staff usually need to update the above salary record table. Specifically, the business node sends a transaction, the coordinator node receives the transaction, and forwards one or more transaction requests included in the transaction to the participant nodes to which the data partitions requested to be operated in each transaction request belong. The participant nodes execute the transaction requests. For example, participant node 1 executes a write request to update the payment month, participant node 2 executes a write request to update the performance bonus, and participant node 3 executes a write request to update the gross salary, tax, and net salary.

[0063] Considering that the data operated by the above transaction involves multiple nodes at the same time, a distributed transaction commit protocol can be used to ensure the ACID of the transaction. A relatively typical commit protocol for distributed transactions is the two-phase commit (2PC) protocol. The two-phase commit includes a prepare phase and a commit phase. Specifically, as Figure 2 shown, taking the transaction request included in the transaction sent by the business node as a write request as an example, the prepare phase includes:

[0064] a. After the coordinator node receives a transaction sent by a service node, it forwards one or more write requests included in the transaction to the participant nodes belonging to the data partitions of the data requested to be operated on in each write request. Figure 2 Taking the example where participant node 1 receives the first write request, participant node 2 receives the second write request, and participant node n receives the nth write request.

[0065] b. After each participant node receives a write request, it executes the write request it received. The participant node executes the write request on the original data to obtain the target data, but does not replace the original data with the target data, and only temporarily stores the target data.

[0066] c. The participant node sends an operation result to the coordinator node. If the participant node can successfully obtain the target data from the original data requested to be operated on in the write request in step b, it sends an operation result of successful operation to the coordinator node. If the participant node cannot successfully obtain the target data from the original data requested to be operated on in the write request in step b, it sends an operation result of failed operation to the coordinator node. Figure 2 Taking the example where participant node 1 generates a first operation result according to the execution situation of the first write request, participant node 2 generates a second operation result according to the execution situation of the second write request, and participant node Pn generates an nth operation result according to the execution situation of the nth write request.

[0067] The commit phase includes:

[0068] d. The coordinator node determines the execution result of the write request according to the operation results sent by each participant node, and sends a commit instruction or a rollback instruction to each participant node. Among them, when the operation result of each participant node is successful operation, the coordinator node determines that the execution result of the write request is successful execution, and the coordinator node sends a commit instruction to each participant node. When at least one of the operation results of each participant node is failed operation, the coordinator node determines that the execution result of the write request is failed execution, and the coordinator node sends a rollback instruction to each participant node. Figure 2 Taking the execution result of the write request as a commit instruction as an example.

[0069] e. Each participant node executes the received commit instruction or rollback instruction. When the instruction sent by the coordinator node is a commit instruction, each participant node performs a commit operation and replaces the original data with the above-mentioned target data; when the instruction sent by the coordinator node is a rollback instruction, each participant node performs a rollback operation and deletes the above-mentioned temporarily stored target data. Figure 3As shown, the target data obtained after performing a write operation on the original data A is B. Before receiving a commit instruction or a rollback instruction, the target data B is temporarily stored. If the participant node performs a commit operation according to the commit instruction, then B replaces A. If the participant node performs a rollback operation according to the rollback instruction, then the temporarily stored target data B is deleted.

[0070] f, each participant node sends an acknowledgment message to the coordinator node, and the acknowledgment message indicates that the commit operation is completed or the rollback operation is completed. Figure 2 In this case, taking participant node 1 generating the first acknowledgment message, participant node 2 generating the second acknowledgment message, and participant node n generating the nth acknowledgment message as an example.

[0071] After receiving the acknowledgment messages fed back by each participant node, the coordinator node returns the execution result of the transaction to the service node.

[0072] The above introduces the interaction process between the coordinator and the participants in the two-phase commit. Based on the above interaction process, it can be seen that the number of interactions between the coordinator node and the participant nodes is relatively large, and the number of interactions is related to the number of participant nodes during the execution of the transaction. The more participant nodes there are during the execution of the transaction, the more interactions there are, which in turn leads to a longer latency for transaction submission and a decline in transaction submission performance.

[0073] In view of this, the embodiments of the present application provide a management method for a transaction processing system. This method can be executed by a node within the transaction processing system, such as a coordinator node. When the transaction processing system also includes a routing (router) node, this method can also be executed by the routing node. In some implementation manners, this method can also be executed by a node outside the transaction processing system. For ease of description, the embodiments of the present application collectively refer to the nodes that execute the management method of the transaction processing system as management nodes. In some implementation manners, this method can also be executed by a management cluster formed by at least one management node, so as to improve availability.

[0074] Specifically, the management node obtains the number of transactions accessing each data partition combination within a period, and then adjusts the storage of the data partitions among the multiple participant nodes according to the number of transactions accessing each data partition combination within this period, so as to reduce the number of participant nodes participating in the transaction. In this way, the number of interactions between the coordinator node and the participant nodes can be reduced, the latency of transaction submission can be lowered, and the performance of transaction submission can be improved.

[0075] To make the technical solution of the present application clearer and easier to understand, the following will introduce the management method of the transaction processing system provided by the embodiments of the present application in conjunction with the accompanying drawings.

[0076] See Figure 4Flowchart of the management method for the transaction processing system shown, the method comprising:

[0077] S402: The management node obtains the number of transactions accessing each data partition combination within a period.

[0078] Specifically, the transactions sent by the service nodes are sent by the coordinator node to the relevant participant nodes for execution, and the distribution of the data partitions possessed by the participant nodes is known. Therefore, the coordinator node can determine the data partition combination accessed by the transactions sent by this coordinator node. This data partition combination includes at least one data partition.

[0079] Based on this, the coordinator node can, within a period, for each transaction sent, perform an increment operation on the access count of the data partition combination accessed by this transaction, so as to obtain the number of transactions accessing each data partition combination within this period. Among them, each data partition combination can be the data partition combination formed by all the data partitions within the transaction processing system, or can be the data partition combination formed by some data partitions.

[0080] The coordinator nodes when the transaction processing system processes different transactions can be different. Based on this, the management node can respectively obtain the number of transactions accessing each data partition combination within this period from each coordinator node, and then summarize the number of transactions obtained from each coordinator node according to the data partition combination respectively, so as to obtain the number of transactions accessing each data partition combination within this period.

[0081] For example, the management node obtains that the number of transactions accessing the data partition combination <P11, P21> within this period from coordinator node 1 is 3, and the number of transactions accessing the data partition combination <P21, P31> is 5, and obtains that the number of transactions accessing the data partition combination <P11, P21> within this period from coordinator node 2 is 7, and the number of transactions accessing the data partition combination <P11, P31> is 4. Then the management node summarizes the number of transactions according to the data partition combination respectively, and can obtain that the number of transactions accessing the data partition combination <P11, P21> within this period is 3 + 7 = 10, the number of transactions accessing the data partition combination <P21, P31> is 5 + 0 = 5, and the number of transactions accessing the data partition combination <P11, P31> is 4 + 0 = 4.

[0082] When the transaction processing system creates a partition view for the data partition, the management node can obtain the number of transactions accessing each data partition combination within a period through a view manager (ViewMgr).

[0083] Among them, a partitioned view is a virtual data table composed of query statements. From the perspective inside the transaction processing system, the partitioned view consists of data from one or more data tables. From the perspective outside the transaction processing system, the partitioned view is like a data table. General operations on data tables such as query, insert, modify, and delete operations can all be applied to the partitioned view. The partitioned view hides the underlying data table structure, simplifies data access operations, provides another entry point for viewing data, and improves security because users are not granted permission to access the underlying data tables. The view manager is specifically a tool for managing the partitioned view.

[0084] In some implementation manners, the management node is a node in the transaction processing system. For example, as Figure 5 shown, the management node can be the coordinator node in the transaction processing system. The view manager can be deployed in this coordinator node. In Figure 5 the example shown, the transaction processing system adopts an architecture with separated computing and storage. Under this architecture, the business node and the data only have a logical holding relationship, and the data is stored in the shared storage layer. The storage layer includes multiple data partitions, such as data partitions 1 to n, and there is a logical mapping relationship between the participant nodes 1 to x and the data partitions 1 to n. When each participant node receives a transaction request, it accesses the data partition with the logical mapping relationship to execute the transaction request. Since general operations on data tables can all be applied to the partitioned view, each participant node can execute query, insert, modify, delete, etc. operations on the partitioned view to execute the transaction request. The coordinator node can record the number of transactions accessing the combined data partitions according to the combined data partitions of the requested operations when executing the transaction.

[0085] The view manager can actively obtain the number of transactions accessing each combined data partition within a period from each coordinator node, or receive the number of transactions accessing at least one combined data partition within a period reported by each coordinator node. Then, the view manager summarizes the data obtained from each coordinator node according to the combined data partitions, so as to obtain the number of transactions accessing each combined data partition within a period.

[0086] In some other implementation manners, the transaction processing system may further include a routing node. For a read request in a transaction request, the routing node directly sends the read request to the corresponding participant node. For a write request in a transaction request, the routing node sends the write request to the coordinator node, and then the coordinator node sends it to the corresponding participant node, so as to achieve read-write separation.

[0087] As Figure 6As shown, the management node can be a routing node in the transaction processing system. The view manager can be deployed in this routing node. The view manager obtains the number of transactions accessing each data partition combination within a period from each coordinator node, and then summarizes the number of transactions separately according to the data partition combinations.

[0088] In some other implementation manners, the management node can also be a node outside the transaction processing system. This management node can be a node in the management cluster. As Figure 7 shown, the view manager is deployed in the management node outside the transaction processing system. The view manager communicates with the coordinator nodes in the transaction processing system, obtains the number of transactions accessing each data partition combination within a period from each coordinator node, and then summarizes the number of transactions separately according to the data partition combinations.

[0089] The period can be set according to actual business requirements. For example, the period can be set to 1 minute, 1 day, 1 week, and so on. The number of transactions accessing the data partition combination obtained by the view manager can be represented by {<partitions_related>, txnCnt}. Among them, <partitions_related> represents the data partition combination accessed by the transaction, and txnCnt represents the number of transactions accessing the data partition combination. For example, {<Px, Py, …>, xx} means that there are xx transactions accessing <Px, Py, …> within this period.

[0090] S404: The management node adjusts the storage of the data partitions in the multiple participant nodes according to the number of transactions accessing each data partition combination within a period.

[0091] The number of transactions accessing each data partition combination can reflect the affinity of the data partitions in the data partition combination. Affinity is an index for measuring the closeness of data partitions. The higher the affinity, the more likely the data partitions in the data partition combination will be accessed simultaneously, and the lower the affinity, the less likely the data partitions in the data partition combination will be accessed simultaneously. Based on this, the management node can adjust the storage of the data partitions in the participant nodes according to the number of transactions accessing the data partition combination, thereby reducing the number of participant nodes participating in the transaction.

[0092] Specifically, the management node determines the target data partition combination according to the number of transactions accessing each data partition combination within the period, and then migrates the data partitions in the target data partition combination from N participant nodes to M participant nodes, where N is greater than M.

[0093] The target data partition combination can be one or more of the data partition combinations accessed during the period. Specifically, when the number of transactions accessing the data partition combination during the period reaches a preset quantity, the management node can determine this data partition combination as the target data partition combination. In some implementation manners, the management node can also determine the proportion of the number of transactions accessing the data partition combination during the period to the total number of transactions accessing each data partition combination. When this proportion is greater than a preset ratio, then this data partition combination is determined as the target data partition combination.

[0094] The target data partition combination can also be one or more of the sub - combinations of the data partition combinations accessed during the period. Specifically, when the number of transactions accessing the sub - combination during the period reaches a preset quantity, the management node can determine this sub - combination as the target data partition combination. In some implementation manners, the management node can also determine the proportion of the number of transactions accessing the sub - combination during the period to the total number of transactions accessing each data partition combination. When this proportion is greater than a preset ratio, this data partition combination is determined as the target data partition combination.

[0095] For ease of understanding, the process of determining the target data partition combination is described below with specific examples. For example, if the number of transactions accessing the data partition combination <P1, P2, P5> reaches 10, <P1, P2, P5> can be determined as the target data partition combination. Another example, if the number of transactions accessing the data partition combination <P1, P2, P5> is 6, and the number of transactions accessing the data partition combination <P1, P2, P4> is 4. It can be seen that the number of transactions accessing the sub - combination <P1, P2> is 6 + 4 = 10. <P1, P2> can be determined as the target data partition combination.

[0096] In some implementation manners, the management node can migrate the data partitions in the target data partition combination from N participant nodes to one participant node. In this way, it can significantly reduce the number of participant nodes participating in the transaction, thereby reducing the number of interactions between the coordinator node and the participant nodes, reducing the latency of transaction submission, and improving the performance of transaction submission.

[0097] Of course, considering that if a node is overloaded, it will lead to a performance decline, the management node can also adjust the storage of the data partitions on the participating nodes according to the number of transactions accessing each data partition combination within the period and the load of the participating nodes, in combination with the load balancing strategy. Specifically, the management node can obtain the load of each participating node, and then migrate each data partition in the target data partition combination from N participating nodes to M participating nodes with a smaller load. To reduce the overhead generated by the migration operation, the management node can migrate each data partition in the target data partition combination from N participating nodes to M participating nodes with a smaller load among these N nodes.

[0098] For ease of understanding, a specific example is given below for illustration.

[0099] As Figure 8 shown, the current distribution of data partitions is that participating node 1 stores data partitions P0 and P3, participating node 2 stores data partitions P1 and P6, participating node 3 stores data partitions P2 and P4, and participating node 4 stores data partition P5. The management node determines the target data partition combination {P1, P2, P5} according to the number of transactions accessing each data partition combination, and then considering that the load of participating node 4 is smaller, the management node migrates data partition P1 from participating node 2 to participating node 4, and migrates data partition P2 from participating node 3 to participating node 4. In this way, data partitions P1, P2, and P5 are located on the same participating node, that is, participating node 4. When the transaction processing system processes transactions, the coordinator node only interacts with participating node 4, reducing the number of interactions between the coordinator node and the participating nodes, reducing the latency of transaction submission, and improving the performance of transaction submission.

[0100] Furthermore, when the transaction processing system adopts an architecture with separated computing and storage, since the participating nodes and the data partitions have a logical mapping relationship, the management node can modify this logical mapping relationship to achieve the migration of data partitions between participating nodes. In this way, the migration cost is reduced.

[0101] In Figure 4 the illustrated embodiment, the management node can collect the number of transactions accessing each data partition combination within the period, so as to adjust the storage of data partitions on the participating nodes based on the real-time collected number of transactions accessing each data partition combination, enabling the data partitions participating in a transaction together to be on the same node, reducing the number of participating nodes during transaction execution, thereby reducing the number of interactions in 2PC and improving the transaction submission efficiency.

[0102] Considering that some business access data exhibits specific patterns, the management node can also use these patterns to predict the number of transactions accessing each data partition combination within a period. Specifically, some business access data exhibits seasonal trends. Seasonality refers to a pattern that repeats within a fixed time period, specifically manifesting as the same direction of change and roughly the same magnitude of change within the same time period, while a trend refers to the basic trend of an indicator. For example, a website has higher traffic on weekends, so the access data exhibits a seasonal trend with a seven-day cycle. For another example, a company's employees always swipe their work cards between 8:00 and 9:00 a.m. and their get off work cards between 5:00 and 6:00 p.m., so the swipe time exhibits a seasonal trend with a daily cycle.

[0103] Based on this, the management node can obtain the number of transactions that accessed each data partition combination within a historical period. Then, based on the number of transactions that accessed each data partition combination within the historical period, it can predict the number of transactions that will access each data partition combination within the period. For example, the management node can obtain yesterday's card swipe time and predict today's card swipe time based on yesterday's card swipe time. In this way, the management node can perform data partition migration in advance based on the prediction results, thereby reducing the number of participating nodes during transaction execution, improving transaction submission efficiency, and increasing throughput.

[0104] In some implementations, the management node may use at least one model to predict the number of transactions accessing each data partition combination within the period. Depending on the business scenario, the management node may use different models to predict the number of transactions accessing each data partition combination within the period.

[0105] For example, for business scenarios such as sales forecasting, traffic flow forecasting, and hospital patient population forecasting, the management node can use the autoregressive integrated moving average (ARIMA) model to predict the number of transactions accessing each data partition combination within the historical period based on the number of transactions accessing each data partition combination within the period.

[0106] For another example, for business scenarios such as stock price fluctuation prediction and factory production prediction, the management node can use an exponential smoothing model to predict the number of transactions accessing each data partition combination within the period based on the number of transactions accessing each data partition combination in the historical period. Of course, for the business scenario of sales prediction, the management node can also use an exponential smoothing model for prediction. When using the exponential smoothing model for prediction, the management node can draw a corresponding curve based on the number of transactions accessing each data partition combination in the historical period, and then use an exponential smoothing model such as the single exponential smoothing prediction method or the double exponential smoothing prediction method to predict the number of transactions accessing each data partition combination within the period.

[0107] For business scenarios such as sales prediction and intermittent product sales prediction, the management node can also use a non-parametric time series (NPTS) model to predict the number of transactions accessing each data partition combination within the period based on the number of transactions accessing each data partition combination in the historical period. It should be noted that the management node can also use a related variant model of NPTS to predict the number of transactions accessing each data partition combination within the period.

[0108] For business scenarios such as climate prediction, holiday traffic prediction, and transaction data prediction, the management node can also use a prophet model to predict the number of transactions accessing each data partition combination within the period based on the number of transactions accessing each data partition combination in the historical period.

[0109] For business scenarios such as climate prediction, store visit volume prediction, product promotion prediction, and sales prediction, the management node can also use a deep learning-based time series model (DeepAR) to predict the number of transactions accessing each data partition combination within the period based on the number of transactions accessing each data partition combination in the historical period.

[0110] After obtaining the number of transactions accessing each data partition combination within the period, the management node can use each partition as a vertex of the transaction partition graph. When two partitions appear in the partitions_related field at the same time, an edge is added, and the weight of the edge is txncnt. For existing edges, the weights are accumulated on the existing basis, thereby generating a transaction partition graph. Figure 9 An example of the transaction partition graph is shown.

[0111] Next, the management node traverses the transaction partition graph and selects the edges whose weights exceed a preset threshold (preset quantity or preset ratio). For example, if the weight / total number of transactions ≥ P, then this edge is selected, and the affinity of the two nodes (i.e., two partitions) on this edge meets the requirements. The management node migrates the two partitions on this edge to the same node.

[0112] As shown in the example above: Assume the preset threshold is 50%, the transaction processing system executed 20 transactions during this statistical period, p1 and p2 participated in 17 together, p1 and p5 participated in 13 together, and p2 and p5 participated in 10 together. Then according to the above logic, the three partitions p1, p2, and p5 should be migrated to the same node. Based on the load of each participating node and other situations, the management node can decide to migrate p1, p2, and p5 to a certain node.

[0113] In the above text, in combination with Figures 1 to 9 , the management method of the transaction processing system provided by this application is described in detail. Next, the apparatus and device according to this application will be described in combination with the accompanying drawings.

[0114] Refer to Figure 10 the structural schematic diagram of the management apparatus of the transaction processing system shown. The transaction processing system includes multiple participating nodes, each participating node stores a data partition, and each participating node is used to process multiple transactions. Among them, the data partition combination accessed by each transaction includes at least one data partition; the apparatus 1000 includes:

[0115] A communication module 1002, configured to obtain the number of transactions accessing each data partition combination within a period.

[0116] A management module 1004, configured to adjust the storage of the data partitions in the multiple participating nodes according to the number of transactions accessing each data partition combination within the period.

[0117] In some implementation manners, the apparatus 1000 further includes:

[0118] An acquisition module 1006, configured to acquire the number of transactions accessing each data partition combination within the period.

[0119] In some implementation manners, the communication module 1002 is specifically configured to:

[0120] Obtain the number of transactions accessing each data partition combination within a historical period;

[0121] The apparatus 1000 further includes:

[0122] A prediction module 1008 is configured to predict the number of transactions accessing each data partition combination within the period based on the number of transactions accessing each data partition combination in the historical period.

[0123] In some implementation manners, the management module 1004 is specifically configured to:

[0124] Determine a target data partition combination according to the number of transactions accessing each data partition combination within the period;

[0125] Migrate the data partitions in the target data partition combination from N participating nodes to M participating nodes, where N is greater than M.

[0126] In some implementation manners, the management module 1004 is specifically configured to:

[0127] Adjust the storage of the data partitions among the multiple participating nodes according to the number of transactions accessing each data partition combination within the period and the loads of the participating nodes.

[0128] The management apparatus 1000 of the transaction processing system according to the embodiments of the present application may correspondingly execute the methods described in the embodiments of the present application, and the above and other operations and / or functions of each module of the management apparatus 1000 of the transaction processing system respectively implement Figure 4 the corresponding processes of the respective methods in, and for the sake of brevity, will not be described in detail herein.

[0129] The above management apparatus 1000 of the transaction processing system may be deployed in a coordinator node, a routing node, or a node outside the transaction processing system. The node implementing the management apparatus 1000 of the transaction processing system is also referred to as a management node. The management node may be a physical device, or a virtual machine, a container, etc. on the physical device.

[0130] Embodiments of the present application further provide a physical device (hereinafter simply referred to as a device). Figure 11 A structural schematic diagram of the device is provided, as Figure 11 described, the device is used to implement the functions of the above management apparatus 1000 of the transaction processing system. The device 1100 includes a bus 1101, a processor 1102, a communication interface 1103, and a memory 1104. The processor 1102, the memory 1104, and the communication interface 1103 communicate with each other through the bus 1101. The bus 1101 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation,Figure 11 It is represented by only one thick line, but it does not mean that there is only one bus or one type of bus. The communication interface 1103 is used for external communication. For example, it obtains the number of transactions accessing the combination of each data partition during a period.

[0131] Among them, the processor 1102 can be a central processing unit (CPU). The memory 1104 may include volatile memory, such as random access memory (RAM). The memory 1104 may also include non-volatile memory, such as read-only memory (ROM), flash memory, HDD or SSD.

[0132] The memory 1104 stores executable code, and the processor 1102 executes the executable code to execute the management method of the foregoing transaction processing system.

[0133] Specifically, in the case of implementing Figure 10 the embodiment shown, and Figure 10 when each module of the management device 1000 of the transaction processing system described in the embodiment is implemented by software, the software or program code required to execute the functions of the management module 1004, the acquisition module 1006, and the prediction module 1008 in Figure 10 is stored in the memory 1104. The function of the communication module 1002 can be implemented through the communication interface 1103.

[0134] The processor 1102 can execute the program code corresponding to the acquisition module 1006 to collect the number of transactions accessing the combination of each data partition during the period. The communication interface 1103 obtains the number of transactions accessing the combination of each data partition during the period and transmits it to the processor 1102 through the bus 1101. The processor 1102 executes the program code corresponding to the management module 1004 to adjust the storage of the data partition in the multiple participant nodes according to the number of transactions accessing the combination of each data partition during the period.

[0135] In some implementations, the communication interface 1103 can also obtain the number of transactions accessing each data partition combination within a historical period, and transmit it to the processor 1102 via the bus 1101. The processor 1102 executes the program code corresponding to the prediction module 1008, and predicts the number of transactions accessing each data partition combination within the period based on the number of transactions accessing each data partition combination within the historical period. Then the processor 1102 executes the program code corresponding to the management module 1004 to adjust in advance the storage of the data partition at the multiple participant nodes according to the predicted number of transactions accessing each data partition combination within the period.

[0136] An embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium includes instructions that, when running on a computer, cause the computer to execute the management method of the transaction processing system of the management device 1000 applied to the transaction processing system as described above.

[0137] An embodiment of the present application also provides a computer program product. When the computer program product is executed by a computer, the computer executes any one of the foregoing management methods of the transaction processing system. The computer program product can be a software installation package. In the case where any one of the foregoing management methods of the transaction processing system needs to be used, the computer program product can be downloaded and executed on the computer.

[0138] In addition, it should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the accompanying drawings of the device embodiments provided in the present application, the connection relationship between the modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines.

[0139] Through the description of the above embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general hardware. Of course, it can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. Generally, functions accomplished by computer programs can easily be implemented by corresponding hardware, and the specific hardware structures for implementing the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for this application, software program implementation is a better embodiment in more cases. Based on such understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disc, etc., and includes several instructions for causing a computer device (which can be a personal computer, training device, or network device, etc.) to execute the methods described in various embodiments of this application.

[0140] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product.

[0141] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are generated in whole or in part. The computer can be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, training device, or data center to another website, computer, training device, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store, or a data storage device such as a training device or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

Claims

1. A management method for a transaction processing system, characterized in that, The transaction processing system includes multiple participant nodes, each participant node stores a data partition, and each participant node is used to process multiple transactions. Among them, the data partition combination accessed by each transaction includes at least one data partition; the method includes: Obtaining the number of transactions accessing each data partition combination within a period; Adjusting the storage of the data partitions among the multiple participant nodes according to the number of transactions accessing each data partition combination within the period; the transaction processing system adopts an architecture with separated computing and storage, and adjusting the storage of the data partitions among the multiple participant nodes includes modifying the logical mapping relationship between the participant nodes and the data partitions.

2. The method according to claim 1, characterized in that, The obtaining the number of transactions accessing each data partition combination within a period includes: During the period, collecting the number of transactions accessing each data partition combination.

3. The method according to claim 1, wherein The obtaining the number of transactions accessing each data partition combination within a period includes: Obtaining the number of transactions accessing each data partition combination within a historical period; Predicting the number of transactions accessing each data partition combination within the period according to the number of transactions accessing each data partition combination within the historical period.

4. The method according to any one of claims 1 to 3, characterized in that, The adjusting the storage of the data partitions among the multiple participant nodes according to the number of transactions accessing each data partition combination within the period includes: Determining a target data partition combination according to the number of transactions accessing each data partition combination within the period; Migrating the data partitions in the target data partition combination from N participant nodes to M participant nodes, where N is greater than M.

5. The method according to any one of claims 1 to 3, characterized in that The adjusting the storage of the data partitions among the multiple participant nodes according to the number of transactions accessing each data partition combination within the period includes: Adjusting the storage of the data partitions among the multiple participant nodes according to the number of transactions accessing each data partition combination within the period and the load of the participant nodes.

6. A management device for a transaction processing system, characterized in that, The transaction processing system includes multiple participant nodes, each participant node stores a data partition, and each participant node is used to process multiple transactions. Among them, the data partition combination accessed by each transaction includes at least one data partition; the device includes: A communication module, configured to obtain the number of transactions accessing each data partition combination within a period; A management module, configured to adjust the storage of the data partitions among the multiple participant nodes according to the number of transactions accessing each data partition combination within the period; the transaction processing system adopts an architecture with separated computing and storage, and adjusting the storage of the data partitions among the multiple participant nodes includes modifying the logical mapping relationship between the participant nodes and the data partitions.

7. The device according to claim 6, characterized in that, The device further includes: A collection module, configured to collect the number of transactions accessing each data partition combination during the period.

8. The device according to claim 6, characterized in that, The communication module is specifically configured to: Obtain the number of transactions accessing each data partition combination within a historical period; The device further includes: A prediction module, configured to predict the number of transactions accessing each data partition combination within the period according to the number of transactions accessing each data partition combination within the historical period.

9. The device according to any one of claims 6 to 8, characterized in that, The management module is specifically used for: Determine a target data partition combination according to the number of transactions accessing each data partition combination within the period. Migrate the data partitions in the target data partition combination from N participating nodes to M participating nodes, where N is greater than M.

10. The device according to any one of claims 6 to 8, characterized in that, The management module is specifically configured to: Adjust the storage of the data partitions among the multiple participating nodes according to the number of transactions accessing each data partition combination within the period and the loads of the participating nodes.

11. A device, characterized in that, The device includes a processor and a memory; The processor is configured to execute the instructions stored in the memory, so that the device executes the management method of the transaction processing system according to any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that, Includes instructions that, when running on the device, cause the device to execute the management method of the transaction processing system according to any one of claims 1 to 5.

Images