Transaction processing method and cloud-native database system

Abstract

The embodiments of the present application belong to the technical field of databases. Disclosed are a transaction processing method and a cloud-native database system. The method comprises: a first database computing node responding to a transaction request of a user and starting to process a target transaction; the first database computing node storing state information of the target transaction in a database state node during the processing of the target transaction; during the process of the first database computing node processing the target transaction, the first database computing node suspending the processing of the target transaction; and a second database computing node acquiring the state information of the target transaction from the database state node, and continuing, on the basis of the state information of the target transaction, to process the target transaction. In the present application, after a database computing node is switched, the processing of a transaction can be continued from a breaking point, such that time resource overheads brought about by the switching of the database computing node are reduced, and thus the switching of the database computing node is not perceived by a user.

Description

Transaction processing method and cloud-native database system

[0001] This application claims priority to Chinese patent application No. 202310693898.4 filed on June 12, 2023, with the invention name “A database system and transaction processing method”, and claims priority to Chinese patent application No. 202410543821.3 filed on April 30, 2024, with the invention name “Transaction processing method and cloud native database system”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of database technology, and in particular to a transaction processing method and a cloud-native database system. Background Art

[0003] When using cloud-native database systems, in many cases, a transaction may be switched from one database compute node to another. For example, when scaling, a new database compute node is added. Consequently, pending transactions on a congested database compute node may be switched to the new compute node. Furthermore, if a downtime or network outage occurs, the problematic database compute node in the cloud-native database system will be taken offline, and pending transactions on the node will be switched to another compute node. Therefore, when using cloud-native database systems, database compute node switching may occur in both planned and unplanned scenarios. Making these node switching invisible to users and minimizing the time and resource overhead associated with switching between compute nodes is a major challenge.

[0004] Summary of the Invention

[0005] This application provides a transaction processing method and a cloud-native database system, which can ensure that the time and resource overhead caused by switching database computing nodes does not affect the user experience, making the switching of database computing nodes imperceptible to the user. The technical solution is as follows:

[0006] In a first aspect, a transaction processing method is provided, which is applied to a cloud-native database system, the cloud-native database system including a first database computing node, a second database computing node, a database status node, and a database storage node; the first database computing node and the second database computing node are used to process transactions requested by users, the database status node is used to store transaction status information, and the database storage node is used to store database data; the method includes: the first database computing node responds to the user's transaction request and starts processing a target transaction, the target transaction is used to perform a read operation and / or a write operation on data in the database storage node; the first database computing node stores the status information of the target transaction to the database status node during processing of the target transaction; the first database computing node suspends processing of the target transaction during processing of the target transaction by the first database computing node; the second database computing node obtains the status information of the target transaction from the database status node, and continues processing the target transaction based on the status information of the target transaction.

[0007] The status information of the target transaction may include the lock table status of the write operation being executed, the log of the target transaction, the query plan tree and intermediate results of the read operation statements in the target transaction, etc.

[0008] Among them, the lock table status of the write operation means that the data table being used by the write operation is in a locked state, and other transactions cannot operate on the data table being used by the write operation; the log of the target transaction records the operation status of the write operation in the target transaction; the query plan tree of a read operation statement is used to guide the execution steps of the statement, and the intermediate results refer to the results obtained from some execution steps of the query plan tree of the statement (excluding the final execution result).

[0009] By storing the target transaction's state information in a database state node and having the second database compute node retrieve the target transaction's state information from the database state node, the second database compute node can continue processing the target transaction from the state it was in before it was suspended, thus enabling breakpoint resumption at the transaction granularity. Compared to rollback or redo, this reduces the time and resource overhead associated with switching database compute nodes and improves the user experience.

[0010] Optionally, the target transaction includes at least one structured query language (SQL) statement, and the status information of the target transaction is used to record an execution status of the SQL statement being executed before the target transaction is paused; the second database computing node continues to process the target transaction based on the status information of the target transaction, including: the second database computing node continues to execute the SQL statement being executed in the target transaction based on the execution status of the SQL statement being executed before the target transaction is paused.

[0011] The execution status of a SQL statement in the target transaction, i.e., the status information related to the SQL statement in the target transaction's status information. If the SQL statement is a write statement, the execution status of the SQL statement may include the corresponding write operation status (recorded in the target transaction's log) and the table lock status of the write operation. If the SQL statement is a read statement, the execution status of the SQL statement may include the corresponding query plan tree and intermediate results.

[0012] By storing the execution status of the SQL statement being executed before the pause in the target transaction in the database status node and the second database computing node continuing to execute the SQL statement based on the execution status of the SQL statement, breakpoint resumption at the statement granularity is actually achieved.

[0013] Optionally, the first database computing node stores the status information of the target transaction to a database status node during processing of the target transaction, including: the first database computing node stores the log of the target transaction to the database status node; the second database computing node continues to process the target transaction based on the status information of the target transaction, including: the second database computing node continues to execute the write operation that was being executed before the suspension in the target transaction based on the log of the target transaction in the database status node.

[0014] It should be noted that during the execution of the target transaction by the first database computing node, the operation status of each write operation is recorded in the target transaction log and stored in the database status node. In this way, the second database computing node can obtain the location of the write operation being executed before the suspension of the target transaction and the operation status of the write operation based on the target transaction log, and thus resume the write operation being executed before the suspension of the target transaction.

[0015] Therefore, even if the first database computing node suddenly fails during a write operation, the operation status of the write operation has been recorded in the log of the target transaction and stored in the database status node. Therefore, for the case where the write operation was being executed before the target transaction was suspended, whether it is a node switch in an unplanned scenario or a node switch in an unplanned scenario, the embodiments of the present application can achieve breakpoint resumption at the transaction granularity or even the statement granularity.

[0016] Optionally, the first database computing node stores the status information of the target transaction to the database status node during processing of the target transaction, further comprising: the first database computing node stores the lock table status of the write operation being executed before the suspension in the target transaction to the status node; the second database computing node continues to process the target transaction based on the status information of the target transaction, comprising: the second database computing node continues to execute the write operation being executed before the suspension in the target transaction based on the log of the target transaction in the database status node and the lock table status of the write operation being executed before the suspension in the target transaction.

[0017] It should be noted that the lock table state corresponding to a write operation is actually generated before the write operation is executed. The first database compute node actually stores the lock table state corresponding to a write operation immediately after it is generated in the database status node. Therefore, even if the first database compute node suddenly fails during a write operation, the lock table state of the write operation will have been stored in the database status node.

[0018] Optionally, the first database computing node stores the status information of the target transaction to a database status node during processing of the target transaction, including: the first database computing node stores the query execution plan tree and intermediate results of the SQL statement corresponding to the read operation being executed before the suspension in the target transaction to the database status node; the second database computing node continues to process the target transaction based on the status information of the target transaction, including: the second database computing node continues to execute the read operation being executed before the suspension in the target transaction based on the query execution plan tree and intermediate results of the SQL statement corresponding to the read operation being executed before the suspension in the target transaction in the database status node.

[0019] It should be noted that the above-mentioned stored intermediate results are the latest intermediate results before the target transaction is suspended.

[0020] Because the first database compute node is required to save the SQL statements and latest intermediate results corresponding to the read operations currently executing in the target transaction before the pause to the database status node before switching, the first database must be aware of the switch plan. This shows that the above method is applicable to switching database compute nodes within a planned scenario. In other words, if the target transaction was executing a read operation before the pause, this method can also be used to resume work at the transaction or even statement granularity after switching the database compute node within a planned scenario.

[0021] Optionally, the first database computing node stores the status information of the target transaction to the database status node during processing of the target transaction, including: the first database computing node stores the SQL statement corresponding to the read operation currently being executed in the target transaction to the database status node during processing of the target transaction; the second database computing node continues to process the target transaction based on the status information of the target transaction, including: the second database computing node re-executes the SQL statement corresponding to the read operation being executed in the target transaction before suspension.

[0022] Because when switching database computing nodes in an unplanned scenario, the first database computing node does not know when the database computing node will be switched, and therefore cannot know when the latest intermediate results should be stored. If you want to be able to resume read operations as in a planned scenario, you need to save the intermediate results regularly, but this will bring about a large resource overhead. Therefore, in this application, the first database computing node can store a SQL statement being executed at each moment in the process of processing the target transaction to the database status node. After the database computing node is switched in an unplanned scenario, the second database computing node can obtain the SQL statement that was being executed before the target transaction was paused, and thus redo the statement.

[0023] Optionally, the first database computing node stores the status information of the target transaction to the database status node during processing of the target transaction, further comprising: the first database computing node stores the execution result of the SQL statement corresponding to the read operation that has been executed in the target transaction to the database status node during processing of the target transaction; the second database computing node continues to process the target transaction based on the status information of the target transaction, further comprising: the second database computing node continues to process the target transaction based on the execution result of the SQL statement corresponding to the read operation that has been executed in the target transaction in the database status node after re-executing the SQL statement corresponding to the read operation currently being executed in the target transaction.

[0024] In this way, if the execution result of the SQL statement corresponding to the previous read operation is needed in the subsequent processing of the target transaction, it can be used directly, which helps to achieve breakpoint resumption at the transaction granularity.

[0025] Optionally, the database status node uses persistent memory to store transaction status information.

[0026] The database status node uses persistent memory to store transaction status information. Once the transaction log is written from the first database compute node to the database status node, the transaction can be committed without having to wait for the log to be flushed to the database storage node. This minimizes the increase in client response time for the target transaction and avoids performance degradation in the cloud-native database system caused by the target transaction log having to traverse two layers of network links before committing.

[0027] Optionally, the method further includes: the first database computing node pushes down the target operator of the SQL statement being executed in the target transaction before the suspension to the database storage node for execution, the target operator being a partial operator included in the query execution plan tree of the SQL statement being executed; the database storage node stores the execution result of the target operator; after the first database computing node suspends processing of the target transaction, the second database computing node obtains the status information of the target transaction from the database status node, and continues to process the target transaction based on the status information of the target transaction, including: after the first database computing node suspends processing of the target transaction, the second database computing node obtains the status information of the target transaction from the database status node and obtains the execution result of the target operator from the database storage node; the second database computing node continues to execute the SQL statement being executed in the target transaction before the suspension based on the status information of the target transaction and the execution result of the target operator.

[0028] By pushing down some operators to let the database storage node execute and save the execution results, the amount of data transmitted between the database storage node and the first database computing node can be reduced, and the speed at which the entire cloud-native database system processes the target transaction can also be accelerated.

[0029] Optionally, the state information of the target transaction also includes the session state of the target transaction. That is, the first database computing node may store the session state of the target transaction in the database state node during processing of the target transaction.

[0030] In this way, taking the implementation environment including a scheduling node as an example, when the target transaction needs to be switched to other database computing nodes for execution, the scheduling node can obtain the session status of the target transaction from the database status node, and the scheduling node can know the client identifier of the transaction request that initiated the target transaction, the identifier of the target transaction and other information. Therefore, the scheduling node can switch the target transaction from the first database computing node to other database computing nodes without switching the wrong transaction or switching to an inappropriate computing node. The client does not need to resend the transaction request to successfully establish a connection between the client and the database computing node to which the target transaction is switched, which can make the user unaware of the node switching.

[0031] Optionally, the method includes: the first database computing node stores the status information of the target transaction to the database status node based on remote direct memory access (RDMA) or compute express link (CXL) technology; and / or, the second database computing node obtains the status information of the target transaction from the database status node based on RDMA or CXL technology.

[0032] RDMA technology allows user programs to bypass the operating system kernel and directly interact with the network card for network communication. CXL technology also offers high bandwidth and low latency, and is often used in high-performance computing and cloud computing. Transmitting state information using RDMA or CXL can increase network bandwidth and reduce latency during the transmission process, thereby improving the performance of cloud-native database systems.

[0033] Optionally, the first database computing node suspends processing of the target transaction when cross-node expansion or cross-node contraction is required or a failure occurs.

[0034] Among them, the switching of database computing nodes caused by cross-node expansion and contraction is considered as the switching of database computing nodes in the planned scenario, and the switching of database computing nodes caused by failure is considered as the switching of database computing nodes in the unplanned scenario.

[0035] In a second aspect, a cloud-native database system is provided. The cloud-native database system includes a first database compute node, a second database compute node, a database status node, and a database storage node. The first and second database compute nodes are configured to process user-requested transactions, the database status node is configured to store transaction status information, and the database storage node is configured to store database data. The cloud-native database system implements the transaction processing method described in the first aspect.

[0036] In a third aspect, a computer device cluster is provided, wherein the computer device cluster includes at least one computing device, each computing device includes a processor and a memory, and the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the transaction processing method provided by the first aspect above.

[0037] In a fourth aspect, a computer-readable storage medium is provided, wherein the storage medium includes computer program instructions. When the computer program instructions are executed by a computer device cluster, the computer device cluster executes the transaction processing method described in the first aspect.

[0038] In a fifth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computing device cluster, the computing device cluster performs the steps of the transaction processing method described in the first aspect. Alternatively, a computer program is provided. When the computer program is executed on the computing device cluster, the computer performs the transaction processing method described in the first aspect.

[0039] The technical effects obtained in the above-mentioned second, third, fourth and fifth aspects are similar to those obtained by the corresponding technical means in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] FIG1 is a schematic diagram of the structure of a traditional database system provided in an embodiment of the present application;

[0041] FIG2 is a schematic diagram of the structure of a database system with storage and computing separation provided in an embodiment of the present application;

[0042] FIG3 is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0043] FIG4 is a schematic diagram of the structure of a cloud native database system provided in an embodiment of the present application;

[0044] FIG5 is a flow chart of a transaction processing method provided in an embodiment of the present application;

[0045] FIG6 is a schematic diagram of a database computing node switching process provided by an embodiment of the present application;

[0046] FIG7 is a schematic diagram of another database computing node switching process provided in an embodiment of the present application;

[0047] FIG8 is a schematic diagram of a process of executing multiple operators when processing a target transaction provided by an embodiment of the present application;

[0048] FIG9 is a schematic diagram of another database computing node switching process provided in an embodiment of the present application;

[0049] FIG10 is a schematic diagram of another database computing node switching process provided in an embodiment of the present application;

[0050] FIG11 is a schematic structural diagram of a computer device provided in an embodiment of the present application;

[0051] FIG12 is a schematic diagram of the structure of a computer device cluster provided in an embodiment of the present application;

[0052] FIG13 is a schematic diagram of a connection method between two computer devices provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0054] First, the application scenarios and implementation environment involved in the embodiments of the present application are introduced.

[0055] Traditional database system architectures typically employ a tightly coupled design, with compute and storage integrated. As shown in Figure 1, each database compute node has its own corresponding database storage node, and these database storage nodes are not shared. This results in a very lengthy scaling process for traditional database systems, and a difficult scaling process after peak business hours, often resulting in significant resource waste. This is one of the most significant drawbacks of traditional database system architectures. Unlike traditional database architectures, a database system architecture with separate storage and compute separates compute and storage. The storage layer utilizes distributed shared storage, and each database compute node in the compute layer shares the storage layer's content. Therefore, scaling up and down a database system architecture with separate storage and compute is much easier than with traditional database architectures.

[0056] Current cloud-native database systems often adopt a database system architecture with storage and computing separation, as shown in Figure 2. Under this architecture, the cloud-native database system includes a computing layer and a storage layer. The computing layer can include multiple database computing nodes, and the storage layer can include multiple database storage nodes (not shown in the figure). All database computing nodes in the computing layer share the storage content of the storage layer. When necessary, the computing layer and storage layer can be quickly expanded or reduced in capacity.

[0057] In this cloud-native database system, database storage nodes are used to store databases, and database computing nodes are used to analyze and execute SQL statements included in transactions. The local storage space of the database computing nodes can temporarily store the query execution plan tree of SQL statements, intermediate results of SQL statements, execution results of SQL statements, and so on.

[0058] Cloud native database systems can scale the computing layer by directly adding or deleting database compute nodes. Assuming that the computing layer is to be scaled down, if the database compute node to be deleted has an ongoing transaction, the ongoing transaction will be suspended and switched to another database compute node for continued execution. When scaling up the computing layer, it is also possible that a transaction will be suspended and then switched to another database compute node for continued execution. In addition, when such a cloud native database system suddenly encounters a downtime, network outage, etc., it will directly shut down the problematic database compute node and switch the transaction currently being processed by the database compute node to another database compute node for execution. In other words, when scaling up or down a database compute node or encountering a downtime, network outage, etc., transactions may be switched from one database compute node to another for processing. In this application, for convenience of description, the switching caused by cross-node expansion or cross-node reduction of database compute nodes will be referred to as switching in a planned scenario; the switching caused by unexpected downtime or failure of a database compute node or network outage will be referred to as switching in an unplanned scenario.

[0059] In current cloud-native database systems, write operations must be rolled back or redone at the database compute node switched to after a transaction is paused. Some cloud-native database systems perform redo operations on reads, while others, while capable of resuming operations after a planned database compute node switch, must perform redo operations after an unplanned database compute node switch. However, with increasing user business volume and complexity, long transactions are increasingly processing transactions in cloud-native database systems. Rolling back or redoing these long transactions can take a significant amount of time, preventing clients from obtaining the latest SQL statement execution results. Therefore, reducing the overhead associated with transaction rollbacks or redone operations, ensuring that node switching is imperceptible to users and that the time and resource overhead associated with node switching does not impact the user experience, is a major challenge.

[0060] Based on this, an embodiment of the present application provides a cloud native database system, which separates the state information generated by the database computing node in the process of processing a transaction on the basis of the existing cloud native database system separating computing and storage, that is, the architecture of the cloud native database system is a storage, computing, and state separation architecture. In addition to the database computing node and the database storage node, the cloud native database system also includes a database state node. In the cloud native database system, after the first database computing node generates state information in the process of executing a transaction, it will send the state information to the database state node, and the database state node will store the state information. In this way, when the first database computing node suspends the execution of the transaction and the transaction is switched to the second database computing node for execution, the second database computing node can obtain the state information of the transaction from the database state node, such as the query execution plan tree and intermediate results of the SQL statement being executed before the transaction is suspended, and the log of the transaction (recording the operations performed on the data by the SQL statement being executed before the transaction is suspended). In this way, for read operations, after completing the database computing node switch in the planned scenario, the second database computing node can continue to execute the SQL statement currently being executed by the transaction, that is, the transaction can be resumed from a breakpoint, even the SQL statement being executed before the pause. For write operations, after completing the database computing node switch in various scenarios, there is no need to roll back or redo the transaction. Instead, the transaction can be resumed from a breakpoint, even the SQL statement being executed before the pause. This can greatly save the extra time overhead caused by node switching and improve the user experience.

[0061] Please refer to Figure 3, which is a schematic diagram of an implementation environment according to an embodiment of the present application. The implementation environment includes at least one client 301, a scheduling node 302, and a cloud-native database system 303. Client 301 can establish a communication connection with cloud-native database system 303 through scheduling node 302. This communication connection can be a wired or wireless connection, which is not limited in this embodiment of the present application.

[0062] The client 301 is used to send the SQL statements included in the target transaction to the cloud native database system 303 through the scheduling node 302.

[0063] The scheduling node 302 is used to receive a transaction processing request sent by the client 301 and establish a communication connection between the client 301 and the cloud native database system 303.

[0064] The cloud native database system 303 is used to process the target transaction, execute the SQL statements included in the target transaction, and return the intermediate results obtained in the process of processing the target transaction, the final results obtained in processing the target transaction, etc. to the client 303 through the scheduling node 302.

[0065] As shown in Figure 4, the cloud-native database system 303 may include a first database computing node 3031, a second database computing node 3032, a database status node 3033, and a database storage node 3034. The first database computing node 3031 and the second database computing node 3032 are used to process transactions requested by users, the database status node 3033 is used to store transaction status information, and the database storage node 3034 is used to store database data.

[0066] For example, as shown in FIG4 , the first database computing node 3031 , the second database computing node 3032 and the database storage node 3034 may be directly connected through a bypass in addition to being indirectly connected through the intermediate database status node 3032 .

[0067] The client 301 may be any electronic product that can interact with a user through one or more methods such as a keyboard, touchpad, touch screen, remote control, voice interaction, or handwriting device, such as a personal computer (PC), a mobile phone, a smart phone, a personal digital assistant (PDA), a wearable device, a pocket PC (PPC), a tablet computer, a smart car computer, a smart TV, a smart speaker, etc.

[0068] The above-mentioned scheduling node 302, database status node 3033, and database storage node 3034 can all be a server, a virtual machine, a container, etc.

[0069] The first database computing node 3031 and the second database computing node 3032 are servers, virtual machines, containers, etc. that can deploy database instances.

[0070] In actual applications, the database status node 3032 can be deployed in the same computing device cluster (the computing device cluster is, for example, a server cluster, a virtual machine cluster, or a container cluster) together with the first database computing node 3031, or the second database computing node 3032, or the database storage node 3034, or can be independently deployed in a separate computing device cluster. Those skilled in the art should understand that the above-mentioned client, scheduling node, database status node, database storage node, first database computing node, and second database computing node are only examples. Any other existing or future forms that may appear, if applicable to the embodiments of the present application, should also be included in the scope of protection of the embodiments of the present application and are hereby incorporated by reference.

[0071] It should be noted that the application scenarios and implementation environments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Persons skilled in the art will appreciate that, as new application scenarios emerge and implementation environments evolve, the technical solutions provided by the embodiments of this application will also be applicable to similar technical problems. For example, the implementation environment described above may not include a scheduling node.

[0072] Figure 5 is a flowchart of a transaction processing method provided in an embodiment of the present application. The method is applied to a cloud-native database system, which may be the cloud-native database system described above. That is, the cloud-native database system includes a first database computing node, a second database computing node, a database status node, and a database storage node. The first and second database computing nodes are used to process transactions requested by users, the database status node is used to store transaction status information, and the database storage node is used to store database data. Referring to Figure 5, the method includes the following steps.

[0073] Step 501: A first database computing node responds to a transaction request from a user and starts processing a target transaction, where the target transaction is used to perform a read operation and / or a write operation on data in a database storage node.

[0074] A database storage node can store multiple databases, and a database can include one or more data tables. A target transaction usually performs read and / or write operations on data in one or more data tables.

[0075] In some embodiments, when a user initiates a transaction request for a target transaction on a client, the scheduling node receives the transaction request and sends it to the first database computing node. The first database computing node then responds to the transaction request and begins processing the target transaction. Alternatively, if the aforementioned implementation environment does not include a scheduling node, the client can also send the transaction request to the first database computing node via other means. The following description uses the aforementioned implementation environment including a scheduling node as an example.

[0076] Database computing nodes in a cloud-native database system are typically of two types: read-only nodes and read-write nodes. In an embodiment of the present application, when the target transaction corresponds to a read-only operation, the first database computing node can be either a read-only node or a read-write node. When the target transaction corresponds to a write operation, the first database computing node is a read-write node.

[0077] The target transaction includes at least one SQL statement. The process of the first database computing node processing the target transaction is the process of the first database computing node executing the SQL statement included in the target transaction. The SQL statement included in the target transaction is sent by the client to the scheduling node, and then sent by the scheduling node to the first database computing node.

[0078] In addition, the target transaction can be a deterministic load or an uncertain load, which is not limited in the embodiments of the present application.

[0079] When the target transaction is a deterministic workload, the SQL statements included in the target transaction are deterministic. In this case, the first database computing node can receive all the SQL statements included in the target transaction at once. When the target transaction is an uncertain workload, the SQL statements included in the target transaction are uncertain. For example, a user can enter statements instantly through the client's interactive interface. In this case, the first database computing node can receive SQL statements one by one. Obviously, whether receiving multiple SQL statements at once or receiving SQL statements one by one, the first database computing node executes the SQL statements one by one in the order in which the SQL statements are included in the target transaction.

[0080] Furthermore, a data page is the basic unit of database storage. When a first database compute node executes an SQL statement, it reads the entire data page containing the data to be operated on by the SQL statement from the database storage node and places the read data page into the buffer pool of the first database compute node's local memory. When executing a target transaction, the first database compute node actually reads and / or writes the corresponding data page in the buffer pool. This reduces the number of accesses the first database compute node makes to the database storage node, improving data processing efficiency.

[0081] Step 502: The first database computing node stores the state information of the target transaction to the database state node during the process of processing the target transaction.

[0082] As can be seen from the above description, the process of the first database computing node processing the target transaction is the process of sequentially executing the SQL statements included in the target transaction. The SQL statements included in the target transaction may include write operation statements (such as addition, deletion, and modification) and / or read operation statements (such as query). When the SQL statement executed by the first database computing node is a read operation statement, the first database computing node performs a read operation, which can also be said to be the target transaction performing a read operation; when the SQL statement executed by the first database computing node is a write operation statement, the first database computing node performs a write operation, which can also be said to be the target transaction performing a write operation.

[0083] In some embodiments, the status information of the target transaction includes the lock table status of the write operation being executed, the log of the target transaction, the query plan tree and intermediate results of the read operation statement in the target transaction, etc.

[0084] When the target transaction performs a write operation, it locks the data table being used by the write operation and records the lock status on the data table in the local buffer pool. This prevents other transactions except the target transaction from performing operations on the data table being used by the write operation. Simultaneously, when performing a write operation, the first database computing node also records the write operation details in the log of the target transaction.

[0085] When preparing to execute a read statement, the first database computing node generates a query plan tree for the read statement and then executes the read statement according to the query plan tree. During the execution of the read statement by the first database computing node according to the query plan tree, intermediate results are generated, such as the current data cursor (i.e., the data page and location of the record being read), and the index result obtained at the end of indexing (i.e., when searching for data, indexing can be performed first and then the specific data can be determined).

[0086] The storage of the above-mentioned status information will be described below in step 404 and will not be expanded here.

[0087] Optionally, the state information of the target transaction also includes the session state of the target transaction.

[0088] Each time a client opens a transaction, it initiates a transaction request, establishing a connection with the database compute node through the scheduling node. Each connection corresponds to a session state. For a target transaction, the session state may include the target transaction's identifier, execution parameters, isolation level, the identifier of the client initiating the transaction request, and language preference.

[0089] In some embodiments, the first database computing node stores the session state of the target transaction in a database state node during processing of the target transaction.

[0090] In this way, when the target transaction needs to be switched to another database computing node for execution, the scheduling node can obtain the session status of the target transaction from the database status node, and the scheduling node can know the client identifier of the transaction request that initiated the target transaction, the identifier of the target transaction and other information. Therefore, the scheduling node can switch the target transaction from the first database computing node to other database computing nodes without switching the wrong transaction or switching to an inappropriate computing node. The client does not need to resend the transaction request to successfully establish a connection between the client and the database computing node to which the target transaction is switched, which can make the user unaware of the node switching.

[0091] In some embodiments, the first database computing node stores the state information of the target transaction to the database state node based on RDMA or CXL technology.

[0092] RDMA technology allows user programs to bypass the operating system kernel and directly interact with the network card for network communication. CXL technology also offers high bandwidth and low latency, and is often used in high-performance computing and cloud computing. Transmitting state information using RDMA or CXL can increase network bandwidth and reduce latency during the state information transmission process, thereby improving the efficiency of transmitting the target transaction's state information from the first database compute node to the database state node, thereby enhancing the performance of the cloud-native database system.

[0093] In some embodiments, the first database computing node stores the state information of the target transaction to the database state node based on RDMA write.

[0094] Information transmission via RDMA technology includes two transmission modes: RDMA write and RDMA read. The RDMA write mode has higher throughput, i.e., higher network bandwidth, than the RDMA read mode. Therefore, when the first database compute node stores the target transaction's status information on the database status node via RDMA technology, the target transaction's status information is transmitted via RDMA write. This can further increase the network bandwidth during the status information transmission process, thereby further improving the efficiency of transmitting the target transaction's status information from the first database compute node to the database status node.

[0095] The state information is transmitted from the first database computing node to the database state node through RDMA technology or CXL technology. For the database state node, it can be implemented through an application program interface including an RDMA library or a CXL library, which can be called a state disaggregated application program interface (state disaggregated API).

[0096] In some embodiments, the database status node uses persistent memory to store transaction status information.

[0097] In current cloud-native database technologies, when a transaction is ready to commit, the transaction log must be written to the disk of the database storage node for persistent storage before the transaction can be committed. If the database state node uses volatile memory, then when a transaction commits, the transaction log is written to the database storage node before the transaction is committed. Therefore, during the transaction processing, the transaction log must first be written from the first database compute node to the database state node, and then from the database state node to the database storage node. This requires the transaction log to traverse two layers of network links, which can degrade the performance of the cloud-native database system.

[0098] Therefore, the database state node uses persistent memory to store transaction status information. After the first database compute node writes the transaction log to the state layer, the transaction log is asynchronously flushed to the database state node at a later time. After the transaction log is written from the first database compute node to the database state node, the transaction can be committed without having to wait until the transaction log is flushed to the database storage node. This prevents increased response time for the target transaction to the client and avoids performance degradation in the cloud-native database system caused by the target transaction log having to traverse two layers of network links before it can be committed.

[0099] In addition, if the database state node uses volatile memory (i.e., non-persistent memory) to store transaction state information, the first database compute node can also bypass the transaction log and directly store it on the database storage node. In this way, the transaction can also be committed after the transaction log only passes through one network link. It should be noted that in this implementation, at some later time, the transaction log will be transferred from the database storage node to the database state node, so that the compute node after the switch can also obtain the transaction log status information from the state node.

[0100] Step 503: While the first database computing node is processing the target transaction, the first database computing node suspends processing the target transaction.

[0101] The first database computing node may suspend processing the target transaction if cross-node scaling is required or a failure occurs. The target transaction will then be switched to the second database computing node by the scheduling node. In addition, in some cases, the scheduling node may switch the target transaction from the first database computing node to another database computing node for processing in order to balance the resources used by each database computing node. In this case, the first database computing node will also suspend processing the target transaction.

[0102] Among them, database compute node switching caused by cross-node scaling and switching due to scheduling nodes to balance the resources used by each database compute node are considered as planned database compute node switching scenarios, while database compute node switching caused by failures is considered as unplanned database compute node switching. In other words, the planned scenario refers to the scenario where the first database compute node is aware of the database compute node switching plan in advance, and the unplanned scenario refers to the scenario where the first database compute node is not aware of the database compute node switching plan in advance.

[0103] It should be noted that the node type of the second database computing node may be the same as or different from that of the first database computing node. When the target transaction corresponds to a read-only operation, the second database computing node can be a read-only node or a read-write node. When the target transaction corresponds to a write operation, the first database computing node is a read-write node.

[0104] In some embodiments, when switching database computing nodes in a planned scenario, the scheduling node will disconnect from the first database computing node and establish a connection with the second database computing node, thereby establishing a connection between the client and the second database computing node.

[0105] In some embodiments, if the scheduling node detects a failure in the first database compute node, it will disconnect from the first database compute node and establish a connection with the second database compute node, thereby reestablishing a connection between the client and the second database compute node. This section describes switching database compute nodes in an unplanned scenario.

[0106] Step 504: The second database computing node obtains the status information of the target transaction from the database status node, and continues to process the target transaction based on the status information of the target transaction.

[0107] In some embodiments, the second database compute node obtains the target transaction's status information from the database status node based on RDMA or CXL technology. This improves network bandwidth and reduces latency in transmitting the status information, thereby increasing the efficiency with which the second database compute node obtains the target transaction's status information. This reduces the extra time overhead associated with switching database compute nodes and improves user experience.

[0108] In some embodiments, the second database compute node obtains the target transaction's status information from the database status node using RDMA writes. This can further improve network bandwidth during the state information transmission process and further improve the efficiency of the second database compute node in obtaining the target transaction's status information, thereby further reducing the additional time overhead caused by switching database compute nodes and improving user experience.

[0109] The state information is transmitted from the database state node to the second database computing node through RDMA technology or CXL technology. For the database state node, it can also be implemented through the state disaggregated API described above.

[0110] In some embodiments, before switching the database computing node, the scheduling node records the pause position cursor of the target transaction, where the pause position cursor indicates the SQL statement being executed in the target transaction before the pause. After the scheduling node establishes a connection between the client and the second database computing node, the scheduling node sends the SQL statement being executed in the target transaction before the pause and the SQL statement to be executed in the target transaction to the second database computing node based on the pause position cursor, so that the second database computing node can continue to execute the target transaction.

[0111] By storing the status information of the target transaction in the database status node and the second database computing node obtaining the status information of the target transaction from the database status node, the second database computing node can continue to process the target transaction according to the state before the target transaction was suspended. In other words, the embodiment of the present application can achieve breakpoint resumption at the transaction granularity.

[0112] In some embodiments, the status information of the target transaction is used to record the execution status of the SQL statement being executed in the target transaction before it is paused; the second database computing node continues to process the target transaction based on the status information of the target transaction, including: the second database computing node continues to execute the SQL statement being executed in the target transaction based on the execution status of the SQL statement being executed in the target transaction before it is paused.

[0113] The execution status of a SQL statement in the target transaction, i.e., the status information related to the SQL statement in the target transaction's status information. If the SQL statement is a write statement, the execution status of the SQL statement may include the corresponding write operation status (recorded in the target transaction's log) and the table lock status of the write operation. If the SQL statement is a read statement, the execution status of the SQL statement may include the corresponding query plan tree and intermediate results.

[0114] Among them, an SQL statement is a read operation statement, and the query plan tree of the SQL statement is the tree structure of the query plan of the SQL statement, which is used to guide the execution steps of the SQL statement. The intermediate result of the SQL statement corresponding to a read operation refers to the results obtained by some execution steps of the query plan tree of the SQL statement (excluding the final execution result of the SQL statement).

[0115] By storing the execution status of the SQL statement being executed before the pause in the target transaction in the database status node and the second database computing node continuing to execute the SQL statement based on the execution status of the SQL statement, breakpoint resumption at the statement granularity is actually achieved.

[0116] The SQL statement being executed before the target transaction is suspended may be a write operation statement or a read operation statement. In these two cases, the possible implementation methods of step 502 and step 503 are different. The following describes three implementation methods of the above steps 502 and 504:

[0117] In a first implementation, step 502 includes: the first database computing node stores the log of the target transaction to the database status node; and step 504 includes: the second database computing node continues to execute the write operation that was being executed before the suspension in the target transaction based on the log of the target transaction in the database status node.

[0118] It should be noted that during the execution of the target transaction by the first database computing node, the operation status of each write operation is recorded in the target transaction log and stored in the database status node. In this way, the second database computing node can obtain the location of the write operation being executed before the suspension of the target transaction and the operation status of the write operation based on the target transaction log, and thus resume the write operation being executed before the suspension of the target transaction.

[0119] Optionally, the database storage node includes a log buffer for storing transaction logs. In this case, step 502 includes: the first database computing node stores the target transaction log in the log buffer of the database status node.

[0120] In addition, if the SQL statement being executed in the target transaction before the suspension is a statement for creating an index, step 502 may further include: the first database computing node also storing the index file corresponding to the data table involved in the SQL statement being executed in the target transaction before the suspension in the database status node; and step 504 may further include: the second database computing node continuing to execute the write operation being executed in the target transaction before the suspension based on the log of the target transaction in the database status node and the index file corresponding to the data table involved in the SQL statement being executed in the target transaction before the suspension. In this way, the index can be continued to be constructed and merged after the database computing node is switched.

[0121] For example, assume that the SQL statement being executed in the target transaction before suspension is "Create Index index_1On R(A)", that is, the target transaction is executing the operation of creating index index_1 of the data on column A of table R before suspension, as shown in Figure 6. Assume that the index file corresponding to the data table is the index file INDEX (the triangle in Figure 6), then the first database node needs to store the log of the target transaction (the circle in Figure 6) and the index file INDEX to the database status node. In this way, after the database computing node is switched, the second database computing node can pull out the index file INDEX and the log of the target transaction that records the execution status of the SQL statement from the database status node, so that the SQL statement can be continued to be executed and the index continuation and merging can be completed.

[0122] In some embodiments, step 502 also includes: the first database computing node stores the lock table status of the write operation being executed before the suspension in the target transaction to the database status node; step 504 includes: the second database computing node continues to execute the write operation being executed before the suspension in the target transaction based on the log of the target transaction in the database status node and the lock table status of the write operation being executed before the suspension in the target transaction.

[0123] It should be noted that the lock table state corresponding to a write operation is actually generated before the write operation is executed, and the first database computing node actually stores the lock table state corresponding to a write operation to the database state node immediately after it is generated.

[0124] Optionally, the database storage node includes an active transaction list (active tran list), which is used to store transaction logs. In this case, step 502 includes: the first database computing node stores the lock table status of the write operation being executed in the target transaction before the pause in the active tran list of the database status node.

[0125] The second database computing node will record the lock table status of the write operation being executed before the target transaction is suspended in the corresponding data table of the buffer pool of the second database computing node. In this way, the second database computing node does not need to lock the data table again, saving the time of another table lock operation. At the same time, it can also ensure that when the second database computing node continues to execute the target transaction, other transactions executed by the second database computing node cannot operate on the data table being used by the target transaction write operation, which can prevent multiple transactions from concurrently writing to a data table, thereby ensuring data consistency and validity.

[0126] For example, assume that the target transaction includes the following SQL statements:

[0127] BEGIN TRANSACTION;

[0128] INSERT INTO R VALUES(100,'xxx');

[0129] COMMIT;

[0130] Assume that the second database computing node also executes another transaction, and the SQL statements included in the transaction are as follows:

[0131] BEGIN TRANSACTION;

[0132] INSERT INTO R VALUES(200,'aaa');

[0133] COMMIT;

[0134] It can be seen that both transactions need to insert certain content into table R, and both include write operations on table R. As shown in Figure 7, assuming that the statement being executed before the target transaction is paused is INSERT INTO R VALUES(100,'xxx'), that is, the write operation on table R is being executed before the target transaction is paused, the first database computing node will store the lock table status of table R to the database status node. Then, after the target transaction switches to the second database computing node, the second database computing node can obtain the lock table status from the database status node. When the second database computing node continues to execute the write operation, there is no need to lock the table R again. At the same time, it can also ensure that when the second database computing node continues to execute the write operation, the other transaction executed by the second database computing node cannot write to table R, thereby ensuring the consistency and validity of the data in table R.

[0135] In the first implementation, since the log of the target transaction is recorded in real time and stored in the database status node throughout the entire process of executing the target transaction, and the lock table state corresponding to a write operation is actually generated and stored in the database status node before the write operation is performed, even if the first database computing node suddenly fails during the execution of a write operation, the operation status of the write operation has been recorded in the log of the target transaction and stored in the database status node, and the lock table state of the write operation has also been stored in the database status node. Therefore, for the situation where the write operation is being executed before the target transaction is suspended, whether it is a node switch in an unplanned scenario or a node switch in an unplanned scenario, the embodiments of the present application can achieve breakpoint resumption at the transaction granularity or even the statement granularity.

[0136] In a second implementation, step 502 includes: the first database computing node stores the query execution plan tree and intermediate results of the SQL statement corresponding to the read operation being executed in the target transaction before the pause to the database status node; and step 504 includes: the second database computing node continues to execute the read operation being executed before the pause in the target transaction based on the query execution plan tree and intermediate results of the SQL statement corresponding to the read operation being executed in the target transaction before the pause in the database status node.

[0137] It should be noted that the above-mentioned stored intermediate results are the latest intermediate results before the target transaction is suspended.

[0138] Optionally, the active tran list included in the database status node is further used to store the query plan tree and intermediate results of the SQL statement corresponding to the write operation being executed in the target transaction before the target transaction was suspended. In this case, step 502 includes: the first database computing node storing the query execution plan tree and intermediate results of the SQL statement corresponding to the read operation being executed in the target transaction before the target transaction was suspended in the active tran list of the database status node.

[0139] That is, in a planned scenario, before switching database compute nodes, the first database compute node will save the query plan tree and the latest intermediate results of the SQL statement corresponding to the read operation currently being executed in the target transaction to the database storage node. This way, after switching nodes, the second database compute node no longer needs to regenerate the query plan tree or re-execute the SQL statement corresponding to the read operation. Instead, it can directly continue executing the read operation that was being executed before the target transaction was paused based on the query plan tree and the latest intermediate results obtained from the database storage node. This avoids the time and resource overhead of regenerating the query plan tree and re-doing the operation, improving the user experience.

[0140] For example, assume that the target transaction includes the following SQL statements:

[0141] BEGIN TRANSACTION;

[0142] SELECT RB,sum(SD)

[0143] FROM R,S

[0144] WHERE RA=SC AND RB>10AND SD>20

[0145] GROUP BY RB;

[0146] COMMIT;

[0147] As shown in Figure 8, Figure 8 shows the process of the target transaction executing each operator in sequence according to the query plan tree of the SELECT statement. These operators mainly include the selection operator (select), the hash join operator (hash join), and the grouping operator (group-by). The triangles in Figure 8 represent some intermediate results obtained by executing each operator. Assuming that before the database computing node switches, the first database computing node has just completed executing operator 81 in Figure 8 and is executing operator 83, then the first database computing node will store the query plan tree of the SELECT statement and the intermediate result 82 in Figure 8 in the database status node. In this way, after the database computing node switches, the second database computing node can directly obtain the query plan tree of the SELECT statement and the intermediate result 82 in Figure 8 from the database status node, and the second database computing node can continue to execute operator 83 in Figure 8, thereby continuing to execute the read operation that was being executed before the pause in the target transaction.

[0148] In the second implementation, the first database compute node is required to save the SQL statements and the latest intermediate results corresponding to the read operation being executed in the target transaction before the pause to the database status node before switching. Therefore, the first database needs to be aware of the switching plan. Therefore, the above method is applicable to switching database compute nodes within a planned scenario. That is, if the target transaction was executing a read operation before the pause, after switching the database compute node within the planned scenario, the embodiments of the present application can also achieve breakpoint resumption at the transaction granularity, or even the statement granularity.

[0149] In a third implementation, step 502 includes: the first database computing node, in the process of processing the target transaction, stores the SQL statement corresponding to the read operation currently being executed in the target transaction to the status node; and step 504 includes: the second database computing node re-executes the SQL statement corresponding to the read operation being executed before the pause in the target transaction.

[0150] Optionally, the active tran list included in the database status node is also used to store the SQL statement corresponding to the read operation currently being executed in the target transaction. In this case, step 502 includes: the first database computing node stores the SQL statement corresponding to the read operation currently being executed in the target transaction in the active tran list of the database status node.

[0151] As can be seen from the above description, the scheduling node can record the pause cursor position of the target transaction, so that after the database computing node switches, the SQL statement that was being executed in the target transaction before the pause and the SQL statement to be executed in the target transaction are sent to the second database computing node. Therefore, the first database computing node can also not store the SQL statement corresponding to the currently executing read operation in the target transaction. After the database computing node switches, the second database computing node directly obtains the SQL statement that was being executed in the target transaction before the pause sent by the scheduling node, and then re-executes the SQL statement corresponding to the currently executing read operation in the target transaction.

[0152] Since the first database computing node does not know when the database computing node will be switched in an unplanned scenario, it is impossible to know when the latest intermediate results should be stored. Therefore, the second implementation method mentioned above cannot be used. In an unplanned scenario, if you want to be able to resume the read operation as in a planned scenario, you need to save the intermediate results regularly, but this will bring about a large resource overhead. Therefore, in an embodiment of the present application, the first database computing node stores a SQL statement being executed at each moment in the process of processing the target transaction to the database status node, so that the subsequent second database computing node can obtain the SQL statement being executed before the target transaction is suspended, thereby redoing the statement. In other words, for the case where a read operation is being performed before the target transaction is suspended, after the database computing node is switched in an unplanned scenario, the embodiment of the present application can redo the SQL statement corresponding to the read operation.

[0153] As shown in Figure 9, Figure 9 shows a possible situation in which switching is performed in a planned scenario when the target transaction is performing a read operation before being suspended. In Figure 9, the dotted triangles in the first database computing node represent some operators that were not executed before the target transaction was suspended. As can be seen from the figure, after the second database computing node obtains the query plan tree and intermediate results stored in the active transaction list in the database status node, it can continue to execute some unfinished operators, thereby realizing breakpoint resumption of the SQL statement corresponding to the query plan tree. As shown in Figure 10, Figure 10 shows a possible situation in which switching is performed in an unplanned scenario when the target transaction is performing a read operation before being suspended. In Figure 10, the dotted triangles in the first database computing node represent some operators that were not executed before the target transaction was suspended. As can be seen from the figure, before the target transaction was suspended, the first database computing node only sent statement 1 to the database status node. After obtaining statement 1 stored in the active transaction list in the database status node, the second database computing node can re-execute statement 1.

[0154] In some embodiments, step 502 also includes: the first database computing node stores the execution result of the SQL statement corresponding to the read operation that has been executed in the target transaction to the database status node during the process of processing the target transaction; step 504 also includes: the second database computing node continues to process the target transaction based on the execution result of the SQL statement corresponding to the read operation that has been executed in the target transaction in the database status node after re-executing the SQL statement corresponding to the read operation currently being executed in the target transaction.

[0155] In this way, if the execution result of the SQL statement corresponding to the previous read operation is needed in the subsequent processing of the target transaction (which is rare), it can be used directly, which helps to achieve breakpoint resumption at the transaction granularity.

[0156] In summary, in this embodiment of the present application, the target transaction status information that the database status node needs to maintain may include: the target transaction log, the lock table status of the write operation being executed before the target transaction is suspended, the query plan tree and intermediate results of the SQL statement corresponding to the read operation being executed before the target transaction is suspended, the SQL statement being executed before the target transaction is suspended, the execution results of the SQL statement corresponding to the read operation that has been executed by the target transaction, and the session status of the target transaction. Table 1 below summarizes the functions / definitions, application scenarios, and common application statements of this status information:

[0157] Table 1

[0158] In some embodiments, the first database computing node pushes down the target operator of the SQL statement being executed before the suspension in the target transaction to the database storage node for execution, and the target operator is a partial operator included in the query execution plan tree of the SQL statement being executed.

[0159] By pushing down some operators to the database storage node for execution, the amount of data transmitted between the database storage node and the first database computing node can be reduced, and the speed at which the entire cloud-native database system processes the target transaction can also be accelerated.

[0160] In some embodiments, after executing the target operator, the database storage node stores the execution results of the target operator. This allows the second database compute node to obtain the execution results of the target operator directly from the database storage node when it needs them, avoiding the need to transfer state information from the database storage node to the database state node.

[0161] In this case, after the first database computing node suspends processing of the target transaction, the second database computing node obtains the status information of the target transaction from the database status node and obtains the execution result of the target operator from the database storage node; based on the status information of the target transaction and the execution result of the target operator, the second database computing node continues to execute the SQL statement that was being executed before the suspension in the target transaction.

[0162] For example, for the statement "select xx from t1, t2 where t1.c = t2.c", a select operation can be performed on the database storage node to select the two data tables t1 and t2, and then a join operation can be performed on the data in the two tables on the first database computing node.

[0163] Finally, it should be noted that embodiments of the present application can implement SQL statement recovery during database compute node switching through a fault-tolerant daemon in the database state node. The fault-tolerant daemon is located in the application programming interface of the database state node. For example, the fault-tolerant daemon is located in the aforementioned state disaggregated API.

[0164] It should be understood that in various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0165] In an embodiment of the present application, during the execution of the target transaction, the first database computing node will send some status information of the target transaction to the database status node, and the database status node will store this status information. In this way, when the first database computing node suspends the execution of the target transaction and the target transaction is switched to the second database computing node for execution, the second database computing node can obtain the status information of the target transaction from the database status node. In this way, through the corresponding status information, when the operation being executed before the suspension in the target transaction is a write operation, there is no need to roll back or redo after completing the node switch in both planned and unplanned scenarios, but it is possible to achieve breakpoint continuation of the transaction granularity or even the statement granularity; when the operation being executed before the suspension in the target transaction is a read operation, after completing the node switch in the planned scenario, the second database computing node can continue to execute the read operation. If it is a node switch in an unplanned scenario, the second database computing node can re-execute the SQL statement corresponding to the read operation. In this way, the extra time overhead caused by node switching can be saved to a large extent, and the user experience can be improved. In addition, since the session state of the target transaction is also stored in the database state node, the scheduling node can directly switch nodes without the client having to resend the transaction request to successfully establish a connection between the client and the database computing node to which the target transaction switches, making the user unaware of the node switch.

[0166] An embodiment of the present application also provides a cloud-native database system, which includes a first database computing node, a second database computing node, a database status node and a database storage node; the first database computing node and the second database computing node are used to process transactions requested by users, the database status node is used to store transaction status information, and the database storage node is used to store database data.

[0167] Among them, the first database computing node is specifically used to respond to the user's transaction request and start processing the target transaction, and the target transaction is used to perform read operations and / or write operations on the data in the database storage node; the first database computing node is also specifically used to store the status information of the target transaction to the database status node during the process of processing the target transaction; during the process of the first database computing node processing the target transaction, the first database computing node is also specifically used to suspend the processing of the target transaction; the second database computing node is specifically used to obtain the status information of the target transaction from the database status node, and continue to process the target transaction based on the status information of the target transaction.

[0168] Optionally, the target transaction includes at least one structured query language SQL statement, and the status information of the target transaction is used to record the execution status of the SQL statement being executed before the target transaction is paused; the second database computing node is specifically used to continue executing the SQL statement being executed in the target transaction based on the execution status of the SQL statement being executed before the target transaction is paused.

[0169] Optionally, the first database computing node is specifically used to store the log of the target transaction to the database status node; the second database computing node is specifically used to continue executing the write operation being executed before the pause in the target transaction based on the log of the target transaction in the database status node.

[0170] Optionally, the first database computing node is further specifically used to store the lock table status of the write operation being executed before the suspension in the target transaction to the status node; the second database computing node is further specifically used to continue executing the write operation being executed before the suspension in the target transaction based on the log of the target transaction in the database status node and the lock table status of the write operation being executed before the suspension in the target transaction.

[0171] Optionally, the first database computing node is specifically used to store the query execution plan tree and intermediate results of the SQL statement corresponding to the read operation being executed before the pause in the target transaction to the database status node; the second database computing node is specifically used to continue executing the read operation being executed before the pause in the target transaction based on the query execution plan tree and intermediate results of the SQL statement corresponding to the read operation being executed before the pause in the target transaction in the database status node.

[0172] Optionally, the first database computing node is specifically used to store the SQL statement corresponding to the read operation currently being executed in the target transaction to the database status node during the process of processing the target transaction; the second database computing node is specifically used to re-execute the SQL statement corresponding to the read operation being executed in the target transaction before it was paused.

[0173] Optionally, the first database computing node is further specifically used to store the execution result of the SQL statement corresponding to the read operation that has been executed in the target transaction to the database status node during the process of processing the target transaction; the second database computing node is further specifically used to continue processing the target transaction based on the execution result of the SQL statement corresponding to the read operation that has been executed in the target transaction in the database status node after re-executing the SQL statement corresponding to the read operation currently being executed in the target transaction.

[0174] Optionally, the database status node uses persistent memory to store transaction status information.

[0175] Optionally, the first database computing node is further specifically used to push down the target operator of the SQL statement being executed in the target transaction before the suspension to the database storage node for execution, and the target operator is a partial operator included in the query execution plan tree of the SQL statement being executed; the database storage node is further specifically used to store the execution result of the target operator; when the first database computing node suspends processing of the target transaction, the second database computing node is further specifically used to obtain the status information of the target transaction from the database status node, and obtain the execution result of the target operator from the database storage node; the second database computing node is further specifically used to continue executing the SQL statement being executed in the target transaction before the suspension based on the status information of the target transaction and the execution result of the target operator.

[0176] Optionally, the state information of the target transaction also includes the session state of the target transaction.

[0177] Optionally, the first database computing node is specifically configured to store the state information of the target transaction to the database state node based on RDMA or CXL technology;

[0178] And / or, the second database computing node is specifically configured to obtain the status information of the target transaction from the database status node based on RDMA or CXL technology.

[0179] Optionally, the first database computing node is specifically configured to suspend processing of the target transaction when cross-node expansion or cross-node contraction is required or a failure occurs.

[0180] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the cloud-native database system described above and the specific implementation of various operations of the transaction processing method in the execution method embodiment can refer to the corresponding processes in the aforementioned method embodiment and will not be repeated here.

[0181] In an embodiment of the present application, the architecture of the cloud native database system is a storage, calculation, and state separation architecture. In addition to database computing nodes and database storage nodes, the cloud native database system also includes a database status node. During the execution of the target transaction, the first database computing node will send some status information of the target transaction to the database status node, and the database status node will store this status information; when the first database computing node suspends the execution of the target transaction and the target transaction is switched to the second database computing node for execution, the second database computing node can obtain the status information of the target transaction from the database status node. In this way, through the corresponding status information, when the operation being executed before the suspension in the target transaction is a write operation, there is no need to roll back or redo after the node switch is completed, but it is possible to achieve breakpoint continuation of the transaction granularity or even the statement granularity; when the operation being executed before the suspension in the target transaction is a read operation, after the node switch is completed, the second database computing node can continue to execute the read operation or re-execute the SQL statement corresponding to the read operation. In this way, the extra time overhead caused by node switching can be saved to a large extent, and the user experience can be improved.

[0182] The first database computing node, second database computing node, database status node, and database storage node described above can all be implemented via software or hardware. As an example, the implementation of the first database computing node is described below. Similarly, the implementation of the other nodes can refer to the implementation of the first database computing node.

[0183] Taking a node as an example of a software functional unit, the first database computing node may include code running on a computing instance. The computing instance may be at least one of a physical host (computing device), a virtual machine, a container, and other computing devices. Furthermore, the above-mentioned computing device may be one or more. For example, the first database computing node may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the application may be distributed in the same region or in different regions. The multiple hosts / virtual machines / containers used to run the code may be distributed in the same AZ or in different AZs, and each AZ includes one data center or multiple data centers with close geographical locations. Generally, a region may include multiple AZs.

[0184] Similarly, the multiple hosts / virtual machines / containers running the code can be distributed within the same VPC or across multiple VPCs. Typically, a VPC is located within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0185] As an example of a hardware functional unit, the first database computing node may include at least one computing device, such as a server. Alternatively, the first database computing node may be implemented using an ASIC or a PLD. The PLD may be implemented using a CPLD, FPGA, GAL, or any combination thereof.

[0186] The multiple computing devices included in the first database compute node can be distributed in the same region or in different regions. The multiple computing devices included in the first database compute node can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the first database compute node can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0187] An embodiment of the present application also provides a computing device 1100. Referring to Figure 11 , computing device 1100 includes a bus 1102, a processor 1104, a memory 1106, and a communication interface 1108. Processor 1104, memory 1106, and communication interface 1108 communicate with each other via bus 1102. Computing device 1100 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 1100.

[0188] Bus 1102 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG11 illustrates a single bus line, but this does not imply a single bus or type of bus. Bus 1102 may include a path for transmitting information between various components of computing device 1100 (e.g., memory 1106, processor 1104, and communication interface 1108).

[0189] The processor 1104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0190] The memory 1106 may include volatile memory, such as random access memory (RAM). The processor 1104 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0191] Memory 1106 stores executable program code, and processor 1104 executes the executable program code to implement the functions of the first database computing node, the second database computing node, the database status node, and the database storage node, thereby implementing the aforementioned transaction processing method. In other words, memory 1106 stores instructions for executing the aforementioned transaction processing method.

[0192] Alternatively, the memory 1106 stores executable code, and the processor 1104 executes the executable code to respectively implement the functions of any of the aforementioned nodes, thereby implementing the aforementioned transaction processing method. In other words, the memory 1106 stores instructions for executing the aforementioned transaction processing method.

[0193] The communication interface 1108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1100 and other devices or a communication network.

[0194] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0195] As shown in Figure 12, the computing device cluster includes at least one computing device 1100. The memory 1106 in one or more computing devices 1100 in the computing device cluster may store the same instructions for executing the aforementioned transaction processing method.

[0196] In some possible implementations, the memory 1106 of one or more computing devices 1100 in the computing device cluster may also store some instructions for executing the aforementioned transaction processing method. In other words, the combination of one or more computing devices 1100 can jointly execute the instructions for executing the aforementioned transaction processing method.

[0197] It should be noted that the memory 1106 in different computing devices 1100 in the computing device cluster can store different instructions, each used to perform part of the functions of any of the aforementioned nodes. In other words, the instructions stored in the memory 1106 in different computing devices 1100 can implement the functions of one or more of the first database computing node, the second database computing node, the database status node, and the database storage node.

[0198] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network. The network may be a wide area network (WAN) or a local area network (LAN), etc. FIG13 shows a possible implementation. As shown in FIG13 , two computing devices 1100A and 1100B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 1106 in the computing device 1100A stores instructions for executing the functions of the first database computing node and the second database computing node. At the same time, the memory 1106 in the computing device 1100B stores instructions for executing the functions of the database status node and the database storage node. Alternatively, the memory 1106 in the computing device 1100A stores instructions for executing the functions of the database status node and the database storage node. At the same time, the memory 1106 in the computing device 1100B stores instructions for executing the functions of the first database computing node.

[0199] The connection method between the computing device clusters shown in Figure 13 can be that considering that the transaction processing method provided in this application requires a large amount of data storage and calculation execution, etc., it is considered to hand over the functions implemented by the above-mentioned first database computing node and the second database computing node to the computing device 1100B for execution, or the functions implemented by the above-mentioned database status node and database storage node to the computing device 1100B for execution.

[0200] It should be understood that the functionality of the computing device 1100A shown in FIG13 may also be implemented by multiple computing devices 1100. Similarly, the functionality of the computing device 100B may also be implemented by multiple computing devices 1100.

[0201] The present application also provides another computing device cluster. The connection relationship between the computing devices in this computing device cluster can be similar to the connection method of the computing device cluster shown in FIG13 . However, the memory 1106 in one or more computing devices 1100 in this computing device cluster can store the same instructions for executing the aforementioned transaction processing method.

[0202] In some possible implementations, the memory 1106 of one or more computing devices 1100 in the computing device cluster may also store some instructions for executing the aforementioned transaction processing method. In other words, the combination of one or more computing devices 1100 can jointly execute the instructions for executing the aforementioned transaction processing method.

[0203] It should be noted that the memory 1106 in different computing devices 1100 in the computing device cluster can store different instructions for executing some of the functions of the cloud-native database system provided in the embodiments of the present application. In other words, the instructions stored in the memory 1106 in different computing devices 1100 can implement the functions of one or more of the aforementioned first database computing node, second database computing node, database status node, and database storage node.

[0204] Embodiments of the present application also provide a computer program product containing instructions. The computer program product may be software or a program product containing instructions that can be executed on a computing device or stored on any available medium. When the instructions are executed by a computing device cluster, the computing device cluster performs the aforementioned transaction processing method.

[0205] The present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the aforementioned transaction processing method.

[0206] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, or a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiments of the present application may be a non-volatile storage medium, in other words, a non-transient storage medium.

[0207] It should be understood that the "plurality" mentioned herein refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0208] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0209] The above description is an embodiment provided for this application and is not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application should be included in the scope of protection of this application.

Claims

1. A transaction processing method, It is characterized in that Applied to a cloud native database system, the cloud native database system includes a first database computing node, a second database computing node, a database status node and a database storage node; the first database computing node and the second database computing node are used to process transactions requested by users, the database status node is used to store transaction status information, and the database storage node is used to store database data; the method includes: The first database computing node responds to a transaction request from a user and starts processing a target transaction, wherein the target transaction is used to perform a read operation and / or a write operation on data in the database storage node; The first database computing node stores the state information of the target transaction to the database state node during processing of the target transaction; During the process of the first database computing node processing the target transaction, the first database computing node suspends processing the target transaction; The second database computing node obtains the status information of the target transaction from the database status node, and continues to process the target transaction based on the status information of the target transaction.

2. The method according to claim 1, It is characterized in that The target transaction includes at least one structured query language SQL statement, and the state information of the target transaction is used to record the execution state of the SQL statement being executed before the target transaction is suspended; The second database computing node continues to process the target transaction based on the state information of the target transaction, including: The second database computing node continues to execute the SQL statement being executed in the target transaction based on the execution status of the SQL statement being executed before being suspended in the target transaction.

3. The method according to claim 1 or 2, It is characterized in that The first database computing node stores the state information of the target transaction to the database state node during processing of the target transaction, including: the first database computing node stores the log of the target transaction to the database state node; The second database computing node continues to process the target transaction based on the status information of the target transaction, including: the second database computing node continues to execute the write operation being executed before the suspension in the target transaction based on the log of the target transaction in the database status node.

4. The method according to claim 3, It is characterized in that The first database computing node stores the state information of the target transaction to the database state node during processing of the target transaction, further comprising: the first database computing node stores the lock table state of the write operation being executed before the suspension in the target transaction to the state node; The second database computing node continues to process the target transaction based on the status information of the target transaction, including: the second database computing node continues to execute the write operation being executed before the suspension in the target transaction based on the log of the target transaction in the database status node and the lock table status of the write operation being executed before the suspension in the target transaction.

5. The method according to claim 1 or 2, It is characterized in that The first database computing node stores the state information of the target transaction to the database state node during the process of processing the target transaction, including: the first database computing node stores the query execution plan tree and intermediate results of the SQL statement corresponding to the read operation being executed before the suspension in the target transaction to the database state node; The second database computing node continues to process the target transaction based on the status information of the target transaction, including: the second database computing node continues to execute the read operation being executed before the suspension in the target transaction based on the query execution plan tree and intermediate results of the SQL statement corresponding to the read operation being executed in the target transaction before the suspension in the database status node.

6. The method according to claim 1, It is characterized in that The first database computing node stores the state information of the target transaction to the database status node during the process of processing the target transaction, including: the first database computing node stores the state information of the target transaction currently being processed in the target transaction during the process of processing the target transaction The SQL statement corresponding to the executed read operation is stored in the database status node; The second database computing node continues to process the target transaction based on the state information of the target transaction, including: the second database computing node re-executes the SQL statement corresponding to the read operation being executed before the suspension in the target transaction.

7. The method according to claim 6, It is characterized in that The first database computing node stores the state information of the target transaction to the database state node during processing of the target transaction, further comprising: the first database computing node stores the execution result of the SQL statement corresponding to the read operation that has been executed in the target transaction to the database state node during processing of the target transaction; The second database computing node continues to process the target transaction based on the status information of the target transaction, and also includes: after the second database computing node re-executes the SQL statement corresponding to the read operation currently being executed in the target transaction, the second database computing node continues to process the target transaction based on the execution result of the SQL statement corresponding to the read operation that has been executed in the target transaction in the database status node.

8. The method according to any one of claims 1 to 7, It is characterized in that The database status node uses persistent memory to store transaction status information.

9. The method according to any one of claims 1 to 8, It is characterized in that The method further comprises: The first database computing node pushes down the target operator of the SQL statement being executed before the suspension in the target transaction to the database storage node for execution, wherein the target operator is a partial operator included in the query execution plan tree of the SQL statement being executed; The database storage node stores the execution result of the target operator; After the first database computing node suspends processing of the target transaction, the second database computing node obtains the status information of the target transaction from the database status node, and continues to process the target transaction based on the status information of the target transaction, including: After the first database computing node suspends processing of the target transaction, the second database computing node obtains the status information of the target transaction from the database status node, and obtains the execution result of the target operator from the database storage node; The second database computing node continues to execute the SQL statement being executed before the suspension in the target transaction based on the state information of the target transaction and the execution result of the target operator.

10. The method according to any one of claims 1 to 9, It is characterized in that The state information of the target transaction also includes the session state of the target transaction.

11. The method according to any one of claims 1 to 10, It is characterized in that The method comprises: The first database computing node stores the state information of the target transaction to the database state node based on remote direct data access RDMA or memory interconnect CXL technology; And / or, the second database computing node obtains the status information of the target transaction from the database status node based on RDMA or CXL technology.

12. The method according to claim 1, It is characterized in that The first database computing node suspends processing of the target transaction when cross-node expansion or cross-node reduction is required or a failure occurs.

13. A cloud-native database system, It is characterized in that The cloud native database system includes a first database computing node, a second database computing node, a database status node and a database storage node; the first database computing node and the second database computing node are used to process transactions requested by users, the database status node is used to store transaction status information, and the database storage node is used to store database data; The first database computing node is specifically used to respond to a transaction request of a user and start processing a target transaction, wherein the target transaction is used to perform a read operation and / or a write operation on data in the database storage node; The first database computing node is further configured to store the state information of the target transaction to the database state node during processing of the target transaction; During the process of the first database computing node processing the target transaction, the first database computing node is further configured to suspend processing. handling the target affairs; The second database computing node is specifically configured to obtain the status information of the target transaction from the database status node, and continue to process the target transaction based on the status information of the target transaction.

14. The system according to claim 13, It is characterized in that The target transaction includes at least one structured query language SQL statement, and the state information of the target transaction is used to record the execution state of the SQL statement being executed before the target transaction is suspended; The second database computing node is specifically configured to continue executing the SQL statement being executed in the target transaction based on the execution status of the SQL statement being executed before being suspended in the target transaction.

15. A system according to claim 13 or 14, It is characterized in that The first database computing node is specifically used to store the log of the target transaction to the database status node; The second database computing node is specifically configured to continue executing the write operation being executed before the suspension in the target transaction based on the log of the target transaction in the database status node.

16. The system according to claim 15, It is characterized in that The first database computing node is further specifically configured to store the lock table state of the write operation being executed before the target transaction is suspended in the state node; The second database computing node is further specifically configured to continue executing the write operation being executed before the target transaction is suspended based on the log of the target transaction in the database status node and the lock table status of the write operation being executed before the target transaction is suspended.

17. The system according to claim 13 or 14, It is characterized in that The first database computing node is specifically used to store the query execution plan tree and intermediate results of the SQL statement corresponding to the read operation being executed before the suspension in the target transaction to the database status node; The second database computing node is specifically used to continue executing the read operation being executed before the suspension in the target transaction based on the query execution plan tree and intermediate results of the SQL statement corresponding to the read operation being executed before the suspension in the target transaction in the database status node.

18. The system according to claim 13, It is characterized in that The first database computing node is specifically configured to store, in the process of processing the target transaction, the SQL statement corresponding to the read operation currently being executed in the target transaction to the database status node; The second database computing node is specifically used to re-execute the SQL statement corresponding to the read operation being executed before the suspension in the target transaction.

19. The system according to claim 18, It is characterized in that The first database computing node is further specifically configured to store, in the process of processing the target transaction, the execution result of the SQL statement corresponding to the read operation that has been executed in the target transaction to the database status node; The second database computing node is also specifically used to continue processing the target transaction based on the execution results of the SQL statements corresponding to the read operations that have been executed in the target transaction in the database status node after re-executing the SQL statements corresponding to the read operations currently being executed in the target transaction.

20. A system according to any one of claims 13 to 19, It is characterized in that The database status node uses persistent memory to store transaction status information.

21. A system according to any one of claims 13 to 20, It is characterized in that The first database computing node is further specifically used to push down the target operator of the SQL statement being executed before being suspended in the target transaction to the database storage node for execution, wherein the target operator is a partial operator included in the query execution plan tree of the SQL statement being executed; The database storage node is specifically further used to store the execution result of the target operator; When the first database computing node suspends processing of the target transaction, the second database computing node is further configured to obtain status information of the target transaction from the database status node, and obtain an execution result of the target operator from the database storage node; The second database computing node is further specifically configured to continue executing the SQL statement being executed before the suspension in the target transaction based on the state information of the target transaction and the execution result of the target operator.

22. A system according to any one of claims 13 to 21, It is characterized in that The state information of the target transaction also includes the session state of the target transaction.

23. A system according to any one of claims 13 to 22, It is characterized in that The first database computing node is specifically used to store the state information of the target transaction to the database state node based on remote direct data access RDMA or memory interconnect CXL technology; And / or, the second database computing node is specifically used to obtain the status information of the target transaction from the database status node based on RDMA or CXL technology.

24. The system according to claim 13, It is characterized in that The first database computing node is specifically used to suspend processing of the target transaction when cross-node expansion or cross-node reduction is required or a failure occurs.

25. A computing device cluster, It is characterized in that comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 12.

26. A computer-readable storage medium, It is characterized in that The method comprises computer program instructions, and when the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 12.

27. A computer program product comprising instructions, It is characterized in that When the instructions are executed by a computing device cluster, the computing device cluster is caused to perform the method according to any one of claims 1 to 12.

Images