Data management method and system, computing device, storage medium, program product

Abstract

The embodiments of this specification provide a data management method and system, computing device, storage medium, and program product, the method comprising: after executing a target statement, the first database stores target data in a log buffer of a second database based on the binary log generated by executing the target statement; the target data includes the binary log corresponding to the target statement, the transaction identifier of the multi-statement transaction to which the target statement belongs, and the location information of the target statement in the multi-statement transaction; the second database determines the target transaction supporting the resume transmission and the corresponding target resume transmission location based on the data in the log buffer in response to the database switching instruction; the second database restores the binary log before the target resume transmission location corresponding to the target transaction in the log buffer to the target binary log of the target transaction, and continues to execute the target transaction based on the target binary log. This method can improve the data management effect.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and in particular to a data management method, a data management system, a computing device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the development of computer technology, various data are increasing, and the requirements for data management are also getting higher and higher. At present, databases can be used for data management, and various data required by client devices can be stored in the database. The client device can send a transaction request to the database so that the database executes the request to query data, change the stored data, or perform other data operations.

[0003] Usually, a primary database and a standby database can be set up, and the primary and standby databases synchronize the stored data. When the primary database can no longer provide services to the client device, the database is switched to connect the client device to the standby database, and the standby database continues to serve the client device. When switching the primary and standby databases, the relevant data of the executed transactions can be synchronized, and the transactions being executed by the primary database at the time of switching will be rolled back after the database switch, resulting in the transaction request issued by the client device may not receive a response.

[0004] Therefore, the data management effect during database switching still needs to be improved. Summary of the invention

[0005] In view of this, an embodiment of this specification provides a data management method that can improve the data management effect. One or more embodiments of this specification also relate to a data management system, a computing device, a computer-readable storage medium, and a computer program product. The data management method includes:

[0006] After executing the target statement, the first database stores the target data in the log buffer area of ​​the second database based on the binary log generated by executing the target statement; wherein the target statement belongs to any data operation statement in a multi-statement transaction, and the target data includes the binary log corresponding to the target statement, the transaction identifier of the multi-statement transaction to which the target statement belongs, and the location information of the target statement in the multi-statement transaction;

[0007] The second database determines, in response to the database switching instruction, a target transaction supporting resume transmission and a corresponding target resume transmission location based on the data in the log buffer area;

[0008] The second database restores the binary log before the target resume point corresponding to the target transaction in the log buffer area as the target binary log of the target transaction, and continues to execute the target transaction based on the target binary log.

[0009] In one embodiment of the present specification, in the process of executing a data operation statement (such as a target statement) using a primary database (such as a first database), the target data can be stored in the log buffer of a standby database (such as a second database) based on the binary log generated by executing the target statement, so as to synchronize the binary log of the transaction being executed and the execution site of the transaction between the primary and standby databases. When performing a master-standby switchover of the database, the target transaction that supports the continuation and the corresponding target continuation site in the transaction being executed by the primary database can be determined based on the data stored in the log buffer of the standby database, and then the target binary log of the target transaction can be restored. In this way, after the master-standby database is switched, the standby database can continue to execute the target transaction from the execution site of the primary database based on the target binary log, which can reduce the impact of the master-standby database switchover on the transaction execution, reduce the situation where the transaction request issued by the client device cannot receive a response due to the switchover of the master-standby database, and thus improve the data management effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a schematic diagram of a data management scenario provided in an embodiment of this specification;

[0011] Figure 2 is a flow chart of a data management method provided in an embodiment of this specification;

[0012] Figure 3 This is a schematic diagram of a log buffer area provided in an embodiment of this specification;

[0013] Figure 4 This is a schematic diagram of the layout of the log buffer area of ​​the first database and the second database provided in an embodiment of this specification;

[0014] Figure 5 This is an architecture diagram of a synchronous binary log provided in an embodiment of this specification;

[0015] Figure 6 is a processing flow chart of a first database after receiving a data operation statement provided by an embodiment of this specification;

[0016] Figure 7 This is a flowchart of a database switching and transaction continuation after switching provided by an embodiment of this specification;

[0017] Figure 8is a structural diagram of a first data management device provided in an embodiment of this specification;

[0018] Fig. 9 is a structural diagram of a second data management device provided in an embodiment of this specification;

[0019] Fig.10 is a structural diagram of a third data management device provided in an embodiment of this specification;

[0020] Fig.11 It is a structural block diagram of a computing device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0021] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0022] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms of "one", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items, where " / " means "or". The term "at least one" in one or more embodiments of this specification refers to "one or more", and "multiple" refers to "two or more". The term "including" is an open description and should be understood as "including but not limited to", and may include other content on the basis of the content already described.

[0023] It should be understood that although the terms "first", "second", etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, "first" may also be referred to as "second", and similarly, "second" may also be referred to as "first". Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0024] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0025] At present, it is becoming more and more common to use databases for data management, and the requirements for data management effects are becoming higher and higher. For example, a client device can be connected to a database, data required by the client device can be stored in the database, and data management can be implemented in the database. Figure 1 1 is a schematic diagram of a data management scenario provided by an embodiment of the present specification, the scenario includes a data management system 10 and a client device 20 served by the data management system 10, the client device 20 can send a data operation request to the data management system 10, so that the data management system 10 performs a corresponding data operation based on the data operation request. The client device 20 can correspond to some applications, and different data operation requests can be sent to the data management system 10 through different applications to operate different data in the data management system.

[0026] like Figure 1 As shown, the data management system 10 may include a first database 101, a second database 102 and a database middleware 103. The first database 101 is connected to the second database 102, and the database middleware 103 is used to control the first database 101 or the second database 102 to be connected to the client device 20 to execute the data operation request sent by the client device 20. One of the first database 101 and the second database 102 can be used as a primary database, and the other can be used as a standby database. The primary database provides services for the client device, and the standby database is used to back up the data in the primary database. In the case that the primary database causes an abnormal operation or database failure due to hardware or software failure, the original standby database can be automatically switched to the primary database to provide services for the client device. In the embodiment of this specification, the first database 101 is used as the primary database and the second database 102 is used as the standby database as an example for explanation.

[0027] In the data management system 10, the first database 101 and the second database 102 can be cloud native databases or some open source databases (such as MySQL database). The database middleware 103 can correspond to Proxy, which refers to a middleware program between the database instance and the application in the client device 20, which is used to assist in achieving the connection maintenance and transaction maintenance functions between the client device 20 and the database, which can help reduce the unavailable time when the database is switched or restarted, and keep abnormal operations transparent to users.

[0028] A transaction refers to a program execution unit that accesses and may update various data items in a database. The client device 20 can trigger the database to execute a certain transaction, which corresponds to the aforementioned data operation request. A transaction may include one or more data operation statements, and the database executes each data operation statement in the transaction to complete the execution of the transaction. The data operation statement may be a structured query language (SQL) statement, and SQL may include multiple types, such as data definition language (DDL), data manipulation language (DML), data query language (DQL), and other types. According to the number of SQL statements in a transaction, transactions can be divided into single-statement transactions and multi-statement transactions.

[0029] For a single-statement transaction, the database directly executes a data operation statement, and the transaction is completed and the execution result of the transaction can be obtained. For a multi-statement transaction, the database needs to execute each data operation statement in the transaction in sequence to obtain the transaction execution result. Each data operation statement in a multi-statement transaction will correspond to a certain start flag and end flag, such as its start flag "begin", after which each data operation statement is listed, and finally ends with an end flag, which can be "commit". The database can start the execution of the transaction based on the start flag, and commit the execution result of the transaction based on the end flag, indicating that the execution of the transaction is completed.

[0030] The data management system 10 can be connected to one or more client devices 20 through a local area network connection, a wide area network connection, an Internet connection or other types of data networks. The client devices 20 here may include but are not limited to: smart phones, tablet computers, laptops, PDAs, personal computers, smart home devices, vehicle-mounted devices, etc. The client device 20 can interact with the user through a user graphical interface to trigger the sending of corresponding data operation requests to the data management system 10, so that the data management system 10 performs corresponding data operations.

[0031] The database in the data management system can be controlled by using the database instance. Usually, when an abnormal situation occurs in the database, such as the switch or restart of the master and standby databases of the database instance, the database instance will disconnect from the client device, such as the connection of the transmission control protocol (TCP), the user datagram protocol (UDP), or the remote direct memory access (RDMA). This will cause the application on the client device to perceive these abnormalities. The application needs to have a mechanism to rebuild the connection and retry the transaction, which will lead to a poor user experience. The application needs to adapt to this "disturbance" that may occur at any time, but often due to various reasons (such as short development time, reference to third-party libraries, etc.), it is difficult for the application to handle short service interruptions well, which will cause the failure to expand and eventually cause corresponding losses.

[0032] In a related technology, in order to solve the problem of application reestablishment of connection, the Proxy uses the connection maintenance function. After an exception occurs in the database instance, only the connection between the Proxy and the database is disconnected, but the connection between the Proxy and the client device on the user side is not disconnected until the database instance is switched or started. However, this solution can only solve the problem of the application in the client device reestablishing the connection when it is idle. If the database has been started but some transactions have not been completed when the database is switched, the transaction will be rolled back after the database is switched, resulting in the loss of some processed data. Therefore, in this solution, if there are transactions that have been opened and have not yet ended when the database is switched, these transactions will be rolled back, and the Proxy needs to re-establish the connection between the client device and the database so that the client device can re-initiate the request for the transaction through the application, which will cause a waste of resources.

[0033] In another related technology, when switching between the primary and standby databases, the driver layer (equivalent to Proxy) waits for the execution of the ongoing transaction to complete before performing the primary and standby data switch. This method allows the driver layer to block requests, and cannot essentially achieve the continuation of the ongoing transaction. Transaction continuation means that when the database is restarted or switched, the transaction can be restored to the state before the restart or switch state, so that the transaction triggered by the user can continue to execute without interruption. This solution greatly limits the usage scenarios. If the execution time of the transaction is slightly longer, it will block the subsequent operation of the entire system, making the user obviously feel the system lag.

[0034] Binary log (Binlog, Binary Log) is a logical log in the database system, mainly used to record all database transactions that modify data. These records can be used to replicate data changes, restore the database to a certain state, or audit the change history of the database. Almost all analytical databases are downstream of Binlog, and obtain Binlog data from the MySQL database for synchronization. In the MySQL ecosystem, the flow of data between different products (such as the flow of data from the MySQL database to the StarRocks database, data warehouse, or data lake products) is realized through Binlog data. Each transaction will generate its own private Binlog file when it is executed. When the transaction is committed, the Binlog file will be copied to the global Binlog file of the MySQL database for persistence. The private Binlog file of the transaction being executed does not require persistence and is only retained in the database memory. After the database instance switches between the primary and standby databases, the private Binlog file of the transaction stored in the memory will be lost. There is no historical Binlog data of the transaction in the new primary database after the switch, and accordingly, the recovery of the executing transaction cannot be completed.

[0035] The database kernel is designed to be divided into an SQL layer and an engine layer. The SQL layer is used to parse and optimize transactions triggered by client devices, and the engine layer is responsible for transaction processing and data storage. In another related technology, transaction information is synchronized to the standby database based on the engine layer log of the primary database. The engine layer log includes a redo log (Redo log, also known as a physical log) and an undo log (Undo log, also written as Undolog). After the primary and standby databases are switched, the standby database can restore the status of the transaction based on the Redo log and the number of undo logs (Undo log number, referred to as Undo_no in this article) in the Undo log. However, since the Binlog data of the executing transaction is private and does not exist in the engine layer log, this solution cannot support databases with Binlog data enabled.

[0036] Among them, Redo log is used to record the physical modification of data pages to ensure the persistence and consistency of data. When a transaction modifies the database, the storage engine in the database will record the detailed information of the modification in the Redo log file on the disk. Even in the case of a system crash, the unfinished transaction can be restored through the Redo log to ensure the integrity of the data. Before each modification of the data in the database, Undolog and Redo log need to be written to record the data before the modification. One Undolog can correspond to the change of one row of data, and the execution of a data operation statement may correspond to the change of multiple rows of data. When the transaction is rolled back or the old version is deleted, the Undolog and Redo log can be used for data recovery. Undo log number is used to record the number of all Undologs generated since the start of the transaction, which is an increasing value. For example, Undo_no is 0 at the beginning of a transaction. After executing a data operation statement, Undo_no increments to 1. After executing another data operation statement, Undo_no increments to 3. This value indicates that this transaction has generated 3 Undologs so far. If you need to roll back the transaction, you only need to read the contents of the three Undologs to build the old version and roll back to the previous state.

[0037] Currently, various database products do not support the continuation of executing transactions when switching between the primary and standby databases, nor do they support the synchronization of binlog data of executing transactions. This greatly limits the application scenarios of the database and makes it difficult to transfer data in the database between different database products.

[0038] This manual provides a data management method that can resume the transmission of ongoing transactions when the primary and standby databases switch, and synchronize the binlog data of ongoing transactions, which can improve data management effects and enrich database application scenarios. Figure 1 The present specification also relates to a data management device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0039] Figure 2 is a flowchart of a data management method provided in an embodiment of this specification. The data management method can be applied to Figure 1 In the data management system 10, the specific different steps can be executed by different devices in the data management system 10 (such as the first database 101, the second database 102 and the database middleware 103). Figure 2 As shown, the data management method specifically includes the following steps 202 to 206.

[0040] Step 202: After executing the target statement, the first database stores target data in the log buffer area of ​​the second database based on the binary log generated by executing the target statement; wherein the target statement belongs to any data operation statement in a multi-statement transaction, and the target data includes the binary log corresponding to the target statement, the transaction identifier of the multi-statement transaction to which the target statement belongs, and the location information of the target statement in the multi-statement transaction.

[0041] In the embodiments of this specification, in the initial state, the first database is used as the main database, connected to the client device through the database middleware, and the second database is used as the standby database. The first database can receive the data operation statement to be executed sent by the client device through the database middleware, and the data operation statement can be an SQL statement. In the embodiments of this specification, the data operation statement is taken as a DML statement as an example, and the data in the database can be queried, added, updated, and deleted through the DML statement.

[0042] A multi-statement transaction includes multiple data operation statements. During the switching process between the primary and standby databases, the execution of the multi-statement transaction may be interrupted; while a single-statement transaction directly obtains the execution result (such as direct execution success or execution failure) without interruption, so the transaction continuation in the embodiments of this specification is only for multi-statement transactions. The transaction information of a single-statement transaction can be restored through the engine layer log, and no additional elaboration is made in the embodiments of this specification. In the embodiments of this specification, the target statement in any multi-statement transaction is taken as an example to introduce the continuation method of a multi-statement transaction. The target statement can be any data operation statement in the multi-statement transaction.

[0043] When the first database receives a data operation statement, it can first determine whether the data operation statement belongs to a multi-statement transaction, that is, determine whether the data operation statement is a target statement. In the case where it is determined that the data operation statement is a target statement, step 202 is executed. If it is determined that the data operation statement belongs to a single-statement transaction, the data management method provided in the embodiment of this specification is not executed (such as not performing steps such as synchronizing the binary log), and the data is directly processed in a traditional manner. For example, the first database can determine whether the data operation statement belongs to a multi-statement transaction by determining whether the transaction to which the data operation statement belongs contains the start mark "begin".

[0044] After the first database determines that it has received the target statement, it executes the target statement, and if the execution of the target statement changes the data in the database, a corresponding binary log will be generated. The binary log can be automatically stored in the private binary log file of the transaction to which the target statement belongs in the first database. In this case, the first database can synchronize the binary log to the second database. For example, a section of memory can be allocated in the second database to store binary logs, such as calling the section of memory a log buffer (also called a Binlog buffer). The first database can store the generated binary log in the log buffer of the second database.

[0045] In an embodiment of the present specification, the first database will also record the transaction identifier (such as trx_id) of the multi-statement transaction to which the target statement belongs, as well as the location information of the target statement in the multi-statement transaction to which it belongs, and store the transaction identifier and location information together with the binary log corresponding to the target statement in the log buffer area of ​​the second database. The location information can characterize the position of the target statement in the multi-statement transaction to which it belongs, and can reflect its order in each data operation statement. For example, the location information can be represented by the number of undo logs (Undo_no) after the execution of the target statement. In some embodiments, the location information can also be represented by the sorting information or other information of the statement. In an embodiment of the present specification, the data stored in the log buffer area of ​​the second database for the target statement is collectively referred to as target data. The target data may include the aforementioned binary log, transaction identifier and location information. Optionally, the first database can execute multiple transactions simultaneously, and the target in each transaction has its corresponding target data.

[0046] In some implementations, the target data may also include verification information for the aforementioned binary log, transaction identifier, and location information. For example, the verification information may be a crc32 verification value.

[0047] Step 204: The second database determines a target transaction supporting resume transmission and a corresponding target resume transmission location based on the data in the log buffer area in response to the database switching instruction.

[0048] When an exception occurs in the first database, resulting in the inability to provide normal services to the client device, a database switch can be performed. A device that monitors whether an exception occurs in each database may be present in the data management system. When the device detects an exception in the first database, it may issue a database switch instruction to the first database, the second database, and the database middleware. Optionally, each database in the data management system may also monitor whether an exception occurs in itself, and issue a database switch instruction to the backup database and the database middleware in the event of an exception. In response to the database switch instruction, each device in the data management system may cooperate to perform a database switch operation.

[0049] After receiving the database switching instruction, the second database can restore the information of the transactions that have been executed in the first database, and also restore the information of the transactions that are being executed by the first database. For example, the second database can determine the transactions that are being executed by the first database based on the data stored in the log buffer area, so as to restore the binary logs that have been generated for these determined transactions, and ensure that the second database can continue to execute these transactions from the interruption position. The transactions that are being executed by the first database determined by the second database based on the data stored in the log buffer area can be the target transactions that support resumed transmission.

[0050] The second database can also determine, for each target transaction, a target resuming location corresponding to the target transaction, which can represent the execution location of the target transaction when the database is switched, and the target resuming location is the location of the information supporting resuming in the target transaction. The target resuming location can be represented by location information (such as Undo_no), and the target resuming location can correspond to a data operation statement in the target transaction.

[0051] In response to the database switching instruction, the database middleware can switch the client device from connecting to the first database to connecting to the second database.

[0052] Step 206: The second database restores the binary log before the target resume point corresponding to the target transaction in the log buffer area as the target binary log of the target transaction, and continues to execute the target transaction based on the target binary log.

[0053] After the second database determines the target transaction that supports resuming and its corresponding target resuming point, it can recover the information of the target transaction. The second database can recover the binary log of the target transaction from the data stored in its log buffer, and restore the binary log before the target resuming point of the stored target transaction to the private binary log file of the target transaction as the target binary log of the target transaction. This is equivalent to generating the target binary log of the target transaction in the second database, and then the second database can directly continue to execute the target transaction from the target resuming point, and replace the database providing services to the client device without the client device being aware of it.

[0054] For other committed (i.e., completed) transactions other than the target transaction, the second database can recover them through the engine layer log of the first database. For example, the first database can store the engine layer log of the committed transaction in a shared disk between the first database and the second database, and the second database can obtain the engine layer log from the shared disk to recover the transaction information.

[0055] The database switch in the embodiment of this specification can resume the transaction being executed, and supports the synchronization of the binary log data of the transaction. In this way, after the master-slave database is switched, the standby database can continue to execute the target transaction from the execution site of the master database based on the target binary log, which can reduce the impact of the master-slave database switch on the transaction execution, and reduce the situation where the transaction request issued by the client device cannot receive a response due to the switch of the master-slave database, thereby improving the data management effect. And this can ensure that the data management system using this method can support data flow between different database products and enrich the application scenarios of the data management system.

[0056] Some optional implementations of the data management method provided in the embodiments of this specification are introduced below.

[0057] In some embodiments, in the log buffer area of ​​the second database, the target data of the target statement in each transaction can be stored in a transaction-exclusive slot manner, and the data of different transactions can be stored in different slots in the log buffer area. In this way, each transaction can be stored in the memory without locks under high concurrency conditions, reducing the competition when writing data into the memory. Correspondingly, in the above step 202, after executing the target statement, the first database stores the target data in the log buffer area of ​​the second database based on the binary log generated by executing the target statement, which can include: after executing the target statement, the first database stores the target data in the target slot in the log buffer area of ​​the second database based on the binary log generated by executing the target statement. Among them, different slots in the binary log buffer area are used to store binary logs of different multi-statement transactions, and the target slot corresponds to the multi-statement transaction to which the target statement belongs.

[0058] For each received transaction, a slot can be allocated for it in the log buffer area, such as allocating a target slot for the target transaction. The target slot is any slot in the log buffer area. After the first database executes each target statement for the target transaction, the target data corresponding to the target statement can be stored in the target slot. Each slot can store target data corresponding to multiple data operation statements. For example, Figure 3 Schematic diagram of a log buffer area provided in an embodiment of this specification. Figure 3 As shown, the log buffer area may include 8 slots (slot1~slot8), and each column is a slot. The number of slots may be pre-set, and the number of slots may limit the maximum concurrent number of operations for storing target data in the log buffer area. The maximum size of each slot (that is, the corresponding maximum amount of stored data) may also be set. In some embodiments, if the remaining storage space required for actual storage is insufficient, the storage space may be expanded.

[0059] The data operation statements sent by the client device are usually executed concurrently. When the data corresponding to multiple statements need to be stored in the same memory interval, it is necessary to avoid data storage errors caused by storage conflicts of different data. Usually, a simple way to avoid storage conflicts is to lock the memory, and only locked data storage operations can be performed, but this will reduce the concurrency of the system. In the embodiments of this specification, the data of each transaction is stored in the corresponding slot, avoiding competition for the memory when the data of each transaction is stored, and there is no need to lock the memory, and the data of each transaction can be stored without conflict, ensuring high concurrency of data storage and improving data storage efficiency.

[0060] Each slot in the log buffer may include multiple sub-blocks. In some embodiments, the target data corresponding to each target statement in the target transaction may be stored in a sub-block (such as a target sub-block) in the target slot of the log buffer of the second database. Accordingly, after executing the target statement, the first database stores the target data in the target slot in the log buffer of the second database based on the binary log generated by executing the target statement, which may include: after executing the target statement, the first database stores the target data in the target sub-block in the target slot in the log buffer of the second database based on the binary log generated by executing the target statement. Among them, different sub-blocks in each slot in the binary log buffer are used to store data corresponding to different statements, the header of the target sub-block is used to store the transaction identifier and location information of the multi-statement transaction, the data part of the target sub-block is used to store the binary log corresponding to the target statement, and the tail of the target sub-block is used to store the verification information of the header and the data part.

[0061] Since each sub-block in the slot represents a binary log event (Binlogevents) generated by a target statement, and the number of binary log events generated by different target statements is uncertain, the amount of data stored in each sub-block in the target slot is uncertain, and the size of each sub-block can be different. For example, the first database can store the target data corresponding to each target statement in the target transaction in the target slot in the log buffer area of ​​the second database in sequence, and the sub-blocks corresponding to these target data can be arranged in sequence in the target slot.

[0062] Please continue to refer to Figure 3 , wherein the structure of the sub-block storing the target data of a target statement is illustrated. Each slot in the log buffer can be divided into multiple sub-blocks, and the lengths of different sub-blocks are different. The sub-block includes a header (Header), a data part (Binlog Data), and a trailer (trailer). The header includes multiple fields, two of which are the transaction identifier (trx_id) of the multi-statement transaction to which the target statement belongs and the location information (Undo_no) of the target statement in the multi-statement transaction. The other fields are not illustrated. The data part includes the original binary log data generated by the execution of the target statement (such as n Binlog events). The tail part can store the crc32 checksum of the header and data parts.

[0063] In some optional implementations, the layout of the log buffer area may not adopt the transaction exclusive slot method, but each transaction adds Binlog to the log buffer area in the form of append write. In this way, when storing the data of each transaction in the log buffer area, the entire log buffer area needs to be exclusively occupied to avoid write conflicts. This method is relatively simple to implement.

[0064] In some embodiments, a log buffer area may also be set in the first database. After the first database generates a binary log after executing the target statement, the binary system may be first stored in its own log buffer area, and then copied from the log buffer area to the log buffer area in the second database. The structure and data storage format of the log buffer area in the first database and the second database may be the same. Accordingly, in the above step 202, after executing the target statement, the first database stores the target data in the log buffer area of ​​the second database based on the binary log generated by executing the target statement, including: after executing the target statement, the first database stores the target data in the log buffer area of ​​the first database based on the binary log generated by executing the target statement; copying the target data stored in the log buffer area of ​​the first database to the log buffer area of ​​the second database; wherein the data storage format in the log buffer area of ​​the first database and the log buffer area of ​​the second database is the same.

[0065] Figure 4 1 is a schematic diagram of the layout of the log buffer area in the first database and the second database provided in an embodiment of this specification. Figure 4 As shown, the log buffer areas in the first database and the second database each include 8 slots, and the structure of each sub-block in each slot may also be the same.

[0066] In some implementations, the first database may utilize a thread to perform full-time synchronization of the binary logs corresponding to each target statement to the second database as a whole. Accordingly, in the above step 202, after executing the target statement, the first database stores the target data in the log buffer of the second database based on the binary log generated by executing the target statement, including: after executing the target statement, the first database generates a synchronization task corresponding to the target statement based on the binary log generated by executing the target statement, and adds the synchronization task to the task queue; using the synchronization thread in the first database, reading the synchronization task in the task queue, and based on the read synchronization task, storing the binary log corresponding to the target statement in the log buffer of the second database.

[0067] A synchronization thread can be set in the first database, and the synchronization thread can be a background thread that runs continuously in the background of the first database. The first database can generate a synchronization task corresponding to the target statement each time a target statement is executed and a corresponding binary log is generated. A task queue can also be set in the first database, and the generated synchronization tasks can be added to the task queue one by one for processing by the synchronization thread. Each synchronization task can correspond to its task information, such as the task information including the transaction identifier corresponding to the task, the information of the target statement corresponding to the task, and the storage location of the binary log corresponding to the task.

[0068] The synchronization thread can continuously obtain synchronization tasks from the task queue. For example, the synchronization thread can obtain each synchronization task in the order in which the tasks are added from the front to the back, and then store the binary log of the corresponding target statement in the log buffer of the second database based on the task information of the synchronization task. The synchronization thread can obtain the target data of the target statement from the log buffer of the first database, and then copy it to the same position in the log buffer of the second database. Optionally, the synchronization thread can also obtain multiple synchronization tasks at one time and execute the multiple synchronization tasks in parallel.

[0069] Figure 5 This is an architecture diagram of a synchronous binary log provided in an embodiment of this specification. Figure 5 As shown, after being executed by the first database, each target statement (such as corresponding THD1~THDn) sent by the client device can send corresponding instructions to the task queue and the management module (MNGR) of the first database. The synchronization thread can execute the tasks in the task queue in sequence and remotely write data to the memory of the second database.

[0070] In the embodiments of this specification, a synchronization thread is used to synchronize binary logs to a standby database, so that batch synchronization operations can be easily performed, and multiple user threads can be prevented from competing for link resources, thereby reducing the delay and contention loss during synchronization.

[0071] Optionally, the first database may not use a synchronization thread to synchronize the binary log, but after the binary log is generated by the execution of each transaction, the thread corresponding to the transaction synchronizes to the second database by itself.

[0072] In some embodiments, the first database can synchronize binary logs to the second database by remote direct memory access (RDMA), without performing too many data input and output (IO) operations, which can improve data synchronization efficiency. In addition, the RDMA kernel by pass can be used to remotely write to the second database to synchronize data, avoiding the second database from wasting CPU to process massive messages. Accordingly, in the above step 202, after executing the target statement, the first database stores the target data in the log buffer area of ​​the second database based on the binary log generated by executing the target statement, which may include: after executing the target statement, the first database uses the remote direct memory access component to store the target data in the log buffer area of ​​the second database based on the binary log generated by executing the target statement. By way of example, the aforementioned synchronization thread can use the remote direct memory access component to store the target data in the log buffer area of ​​the second database.

[0073] Optionally, the first database can also synchronize the private binary log of the transaction through the shared disk between the first database and the second database. When the transaction is executed, the thread corresponding to the transaction can persist its own private binary log to the shared disk. After the primary and standby databases are switched, the standby database can recover the private binary log of the executing transaction by scanning these files. In this way, a file synchronization (sync) operation is called once after each target statement is successfully executed. When the standby database performs data recovery, all files need to be loaded, and more IO operations are required.

[0074] Optionally, the first database may also synchronize the private binary log of the transaction through TCP. In this manner, the second database needs to use a central processing unit (CPU) to process TCP messages when performing data recovery.

[0075] In some implementations, after the first database executes the target statement in step 202, based on the binary log generated by executing the target statement, the target data is stored in the log buffer of the second database. The data management method provided by the embodiment of this specification also includes: the first database sends the transaction identifier, location information and execution result of the target statement of the multi-statement transaction to the database middleware; the database middleware records the transaction identifier and location information of the multi-statement transaction, and sends the execution result of the target statement to the client device. The transaction identifier and location information recorded by the database middleware can be used in the subsequent determination of the actual transaction information to be continued. The specific method of use will be described in detail later and will not be described here for the time being.

[0076] Figure 6 This is a flowchart of a process of a first database after receiving a data operation statement provided by an embodiment of this specification. The process includes the process of synchronizing binary logs from the first database to the second database. The process can be cross-referenced with the above content. Figure 6As shown, when the database instance is running normally (such as the first database is running normally), before executing each SQL statement, the first database determines whether the current statement is in the state of a multi-statement transaction. If not, the current statement is a single-statement transaction, and there is no need to resume the transaction, so there is no need to synchronize the binary log to the second database. The first database directly executes the statement and returns the execution result to the client device through the database middleware Proxy. If it is determined that the statement is in a multi-statement transaction (that is, the transaction started by the begin display), the first database can first record the position of the private binary log corresponding to the multi-statement transaction before the statement starts to execute. After the statement is executed, the first database records the Undo_no after the statement is executed in the engine layer log (such as the Undo log). Then, it is determined whether the execution of the statement generates a new Binlog. When the private Binlog of this statement is generated in the first database, the new Binlog generated is determined based on the position of the private binary log corresponding to the statement before the statement starts to execute. This part of the Binlog is copied to the background synchronization thread. The synchronization thread efficiently synchronizes these Binlogs to the remote end (such as the log buffer area of ​​the second database) through RDMA. For example, the target data (including Binlog, transaction identifier trx_id and Undo_no) corresponding to the statement stored in the log buffer area of ​​the first database is copied to the synchronization thread, and the synchronization thread synchronizes the target data to the log buffer area of ​​the second database.

[0077] In some embodiments, after waiting for Binlog synchronization to complete, the thread executing the current SQL statement returns the previously recorded Undo_no, trx_id, and the execution result of the statement to the Proxy. When the execution of the SQL statement does not generate a new private Binlog, the Undo_no, trx_id, and the execution result of the statement corresponding to the statement are also returned to the Proxy. The Proxy records and saves the trx_id and Undo_no corresponding to the last successfully executed statement of the current transaction, and returns the execution result to the client device. When the Proxy receives the trx_id and Undo_no corresponding to the next successfully executed statement of a transaction, it can update the recorded trx_id and Undo_no, and only record the trx_id and Undo_no corresponding to the latest successfully executed statement of a transaction. When the transaction is executed, that is, after it is committed, the Proxy can delete the recorded trx_id and Undo_no related to the transaction.

[0078] The following is an introduction to the implementation method of determining the target transaction that supports resume transmission and the corresponding target resume transmission site in step 204. In some embodiments, the second database can combine the data in the engine layer log and the data stored in the log buffer area to jointly determine the target transaction and the corresponding target resume transmission site. In this way, even if the data in the engine layer log and the log buffer area are inconsistent when the database is switched due to certain reasons (such as network delays, interference or abnormal operations during storage), the second database can also determine the appropriate target transaction and target resume transmission site to keep the engine layer log and binary log used after the database switch consistent, thereby avoiding data loss after transaction resumption and improving data management effects.

[0079] Accordingly, after executing the target statement, the first database may record the location information (such as Undo_no) of the target statement in the multi-statement transaction to which it belongs in its engine layer log (such as Undolog). In the above step 204, the second database responds to the database switching instruction and determines the target transaction supporting the resume and the corresponding target resume location based on the data in the log buffer area, which may include: the second database responds to the database switching instruction and obtains the transaction information executed by the first database based on the engine layer log of the first database; the second database determines the target transaction supporting the resume and the corresponding target resume location based on the transaction information and the data in the log buffer area.

[0080] The first database can store its engine layer log in the shared disk, and the second database can also read the data in the shared disk at any time. When the second database receives the database switching instruction, it can scan the engine layer log of the first database in the shared disk and restore the transaction information corresponding to the transaction executed by the first database. In some embodiments, when the second database starts to switch, it can first replay the physical log (that is, Redo log) of the first database, and restore all transaction information of the first database before the database switch (that is, before the first database crashes) to the Undolog of the shared disk. Since the latest data in the engine layer log of the first database (such as the latest data on the Undo log page) may not be written to the disk before the first database crashes, the second database cannot obtain it. The latest data is located in the physical log of the first database. The second database can restore the Undolog page data that has not been written to the disk of the first database to the shared disk by replaying the physical log. The second database scans the Undo log in the shared disk to restore all transaction information before the first database crashes. The transaction information also includes information about the execution position in the transaction being executed when the first database performs a database switch, and the execution position is represented by the site information corresponding to the target statement.

[0081] After the transaction information is restored, the second database can traverse all the private Binlog data of the executing transactions stored in its log buffer area to obtain the Binlog of the executing transaction and the corresponding site information. The second database compares the information of the executing transaction recorded in the transaction information and the information of the executing transaction recorded in the log buffer area to determine the target transaction that supports the resume and the corresponding target resume site. By way of example, the second database determines the transaction whose transaction information is consistent with the information in the log buffer area as the target transaction that supports the resume. Accordingly, the above-mentioned second database determines the target transaction that supports the resume and the corresponding target resume site based on the transaction information and the data in the log buffer area, including: the second database determines the transaction that exists in both the transaction information and the log buffer area as the target transaction that supports the resume; for the target transaction, compares the site information in the transaction information with the site information in the log buffer area to determine the target resume site corresponding to the target transaction.

[0082] The second database can determine whether there is a transaction identifier recorded in the log cache in the transaction information obtained from the shared disk (hereinafter referred to as the engine layer transaction information). If so, the transaction corresponding to the transaction identifier is determined as the target transaction that supports resuming; otherwise, it is determined that the transaction corresponding to the transaction identifier does not support resuming, and the data corresponding to the transaction identifier recorded in the log cache can also be deleted. For the target transaction that exists in both the engine layer transaction information and the log cache, the second database can also determine and compare the location information of the target transaction in the engine layer transaction information and the location information of the target transaction in the log cache, and determine the target resuming location based on the location information that exists in both the engine layer transaction information and the log cache. The target resuming location determined by the second database needs to match the statement execution result received by the client device to ensure that after the subsequent transaction resuming is completed, the second database can continue to provide services to the client device without any sense.

[0083] In the first comparison result, the location information recorded for the same target transaction in the engine layer transaction information and the log buffer area is the same. Accordingly, for the target transaction, the location information in the transaction information is compared with the location information in the log buffer area to determine the target continuation location corresponding to the target transaction, including: for the target transaction, when it is determined that each location information in the transaction information is the same as each location information in the log buffer area, the last two locations of the target transaction are determined as the target continuation locations.

[0084] When the location information for the same target transaction in the engine layer transaction information and the log buffer is the same, it means that the database has completed the execution of all statements in the target transaction. At this time, there may be the following two situations. One is that the execution results of all statements have been returned to the client device. In this case, it is necessary to resume the binary logs of all statements in the target transaction. The other is that the execution result of the last statement in the target transaction has not been returned to the client device. In this case, the client device task has not yet executed the last statement, so it is necessary to resume the binary logs of the statements before the last statement, and the last statement should be re-executed. Therefore, when the location information of the same target transaction in the engine layer transaction information and the log buffer is the same, the last two locations of the target transaction can be determined as the target resume locations, and then the specific final resume is determined according to the execution result reception of the client device. Which of the two locations is used for the final resume.

[0085] In the second comparison result, the site information recorded for the same target transaction in the engine layer transaction information and the log buffer area is not completely consistent. For example, it is possible that the site information in the engine layer transaction information is more than the site information in the log buffer area, or the site information in the log buffer area is more than the site information in the transaction information. In this case, the target resume site can be determined based on the engine layer transaction information and the last consistent site information in the log buffer area. Accordingly, the above-mentioned comparison of the site information in the transaction information with the site information in the log buffer area for the target transaction to determine the target resume site corresponding to the target transaction includes: for the target transaction, when it is determined that the site information in the transaction information is more than the site information in the log buffer area, the last site of the target transaction in the log buffer area is determined as the target resume site; for the target transaction, when it is determined that the site information in the log buffer area is more than the site information in the transaction information, the last site of the target transaction in the transaction information is determined as the target resume site.

[0086] Since the engine-layer transaction information and the log buffer both record the location information corresponding to each statement during the execution of the same target transaction in sequence, after each statement is executed normally, the location information recorded for the statement in the engine-layer transaction information and the log buffer must be the same. If the information recorded in the engine-layer transaction information and the log buffer is inconsistent for the same target transaction, the inconsistency will be located at the location of the last statement executed by the target transaction. The cause of the inconsistency is usually that the information of some statements is not recorded in time. The inconsistency can be reflected in that more information is recorded in one storage location, and some information is missing in another storage location. Therefore, there may be a situation where the location information in the above-mentioned engine-layer transaction information is more than the location information in the log buffer, or the location information in the log buffer is more than the location information in the transaction information. The above-mentioned inconsistency of location information can also be called location information asynchrony.

[0087] After the database is switched in the embodiment of this specification, to achieve the normal execution of a transaction, the engine layer transaction information corresponding to the transaction needs to be consistent with the binary log, and the second database performs transaction continuation to restore the target continuation site and its previous data. Therefore, the second database needs to determine the site corresponding to the consistent site information recorded in the engine layer transaction information and the log buffer as the target continuation site to ensure that the engine layer transaction information corresponding to the target transaction and the binary log remain consistent after the transaction is resumed. In addition, the second database also needs to resume as much information as possible to ensure that the transaction after the continuation matches the execution status of the transaction in the client device to avoid affecting the normal transaction feedback of the client device. Therefore, the second database can determine the last consistent site recorded for the target transaction in the engine layer transaction information and the log buffer as the target continuation site. The "last" here corresponds to the last statement executed in the currently executed statement in the target transaction.

[0088] In some embodiments, when the location information for the same target transaction record in the engine layer transaction information and the log buffer is not completely consistent, the second database can also adjust the data in the engine layer transaction and the log buffer to adapt to the subsequent data storage situation. Accordingly, when the last location of the target transaction in the log buffer is determined as the target continuation location, the transaction information is rolled back to the state where the statement corresponding to the target continuation location has been executed; when the last location of the target transaction in the transaction information is determined as the target continuation location, the target data corresponding to the statement after the target continuation location in the log buffer is deleted. The second database can delete the relevant data of the statement after the target continuation location in the engine layer transaction information and the log buffer to ensure the consistency of the data in the engine layer transaction information and the log buffer.

[0089] The following is a further explanation of the comparison of the aforementioned engine-layer transaction information and the data in the log buffer (Binlog buffer). The second database uses Undo_no to verify whether the execution sites of the transactions in the engine-layer transaction information and Binlog are consistent. When the target transaction is executed, the Undo_no of the transaction (recorded as engine_Undo_no) will be stored in the Undo log, and the log buffer also stores the transaction identifier trx_id and Undo_no field (recorded as Binlog_Undo_no) of the target transaction. Only the Undo_no with the same corresponding trx_id can be compared. There are the following three situations for the comparison of the Undo_no in the engine-layer transaction information and the log buffer.

[0090] In the first case, the engine layer transaction information and the log buffer area keep consistent with the Undo_no of the target transaction, that is, Binlog_Undo_no=engine_Undo_no, which corresponds to the first comparison result mentioned above. In this case, the statement execution result returned to the client device before the database switch may correspond to the last Undo_no in the engine layer transaction information, or the second to last Undo_no in the engine layer transaction information.

[0091] In the case of the last Undo_no, it means that the last statement sent by the client device for the target transaction has been successfully executed in the first database and fully synchronized with the second database. The database middleware has received the execution result returned by the database and returned it to the client device. In this case, the second database maintains the current state and all Binlogs in the log buffer area can be restored.

[0092] In the case of the penultimate Undo_no, it means that the first database successfully executed the last statement sent by the client device for the target transaction and fully synchronized with the second database, but the database middleware did not receive the execution result of the database for the last statement. From the perspective of the database middleware and the client device, a database switch occurred when the last statement was being executed. At this time, the engine layer can roll back the last statement, and roll back the engine layer transaction information to the state where the penultimate statement has been executed. The Binlog corresponding to the last statement can be deleted from the log buffer. In this case, the target transaction maintains the state where the penultimate statement has been executed, and the data middleware can send the last statement that has not been executed to the second database so that the second database can re-execute the statement and return the execution result to the client device.

[0093] In the second scenario, the transmission of Binlog in the log buffer lags behind the storage of transaction information in the engine layer, that is, Binlog_Undo_no < engine_Undo_no. This scenario corresponds to the situation in the second comparison result above where the site information in the engine layer transaction information is more than the site information in the log buffer. Table 1 below shows the comparison of the status of the engine layer transaction information and Binlog in such a scenario provided by the embodiments of this specification. As shown in Table 1, the last statement is statement 2, corresponding to Undo_no 2, and the penultimate statement is statement 1, corresponding to Undo_no 1. The engine layer transaction information and Binlog of statement 1 have been synchronized, and the engine layer transaction information of statement 2 has been synchronized, but the Binlog has not been synchronized.

[0094] Table 1

[0095]

[0096] This scenario indicates that the first database is executing statement 2, and the corresponding engine layer transaction information has been synchronized to the second database through physical logs, but the Binlog has not been synchronized to the second database yet. Since statement 2 is being executed, the statement whose execution result the client device can receive can only be statement 1, and correspondingly, the target resume transfer site is the site corresponding to Undo_no 1. In this case, the engine layer transaction information needs to be rolled back to the information corresponding to statement 1, and the Binlog is restored to the state where statement 1 has been executed.

[0097] In the third scenario, the storage of transaction information in the engine layer lags behind the transmission of Binlog in the log buffer, that is, Binlog_Undo_no > engine_Undo_no. This scenario corresponds to the situation in the second comparison result above where the site information in the log buffer is more than the site information in the engine layer transaction information. Table 2 below shows the comparison of the status of the engine layer transaction information and Binlog in such a scenario provided by the embodiments of this specification. As shown in Table 2, the engine layer transaction information and Binlog of statement 1 have been synchronized, the engine layer transaction information of statement 2 has not been synchronized, but the Binlog has been synchronized.

[0098] Table 2

[0099]

[0100] This situation indicates that the first database is executing statement 2, and the corresponding engine-layer transaction information has not been synchronized to the second database through the physical log, but the Binlog has been synchronized to the second database. Since statement 2 is being executed, the client device can only receive statement 1 as the execution result, and the corresponding target resume point is the point corresponding to Undo_no 1. In this case, the engine-layer transaction information needs to maintain the current state of statement 1 execution, delete the Binlog of statement 2 in the log buffer, and restore to the state of maintaining the execution of statement 1.

[0101] In some implementations, after determining the target transaction and the corresponding target resuming location, the second database may start a background thread, and the background thread may perform a specific transaction resuming process. The information of each determined target transaction and the corresponding target resuming location may form a resumable transaction list, and the background thread may resume the transactions in the resumable transaction list based on the resumable transaction list, and directly roll back the transactions outside the list.

[0102] In the embodiments of the present specification, database middleware can be used to assist in database switching. For example, when the database instance is operating normally, when returning the statement execution result to the client device, if the statement belongs to a multi-statement transaction, the transaction identifier of the transaction to which the statement belongs and the location information corresponding to the statement are also sent to the database middleware, and the database middleware can record the transaction identifier and location information corresponding to each latest executed statement. The location information recorded by the database middleware can correspond to the statement execution result received by the client device, and the transaction continuation can be ensured to be consistent with the transaction execution status perceived by the client device based on the location information. When switching the database, the database middleware can assist the second database in transaction continuation based on the transaction identifier and location information recorded by it.

[0103] Correspondingly, in the above step 206, the second database restores the binary log before the target resume point corresponding to the target transaction in the log buffer area as the target binary log of the target transaction, including: when the transaction identifier recorded by the database middleware corresponds to the target transaction, and the point information recorded by the database middleware corresponds to the target resume point, the second database restores the binary log before the target resume point corresponding to the point information in the target transaction in the log buffer area as the target binary log of the target transaction. For example, the database middleware or the second database can determine whether the transaction identifier recorded by it corresponds to the target transaction, and determine whether the point information recorded by it corresponds to the target resume point.

[0104] In an implementation manner in which the above-mentioned judgment is performed by the second database, the second database restores the binary log before the target resume point corresponding to the point information in the target transaction in the log cache area to before the target binary log of the target transaction (such as before step 206). The data management method provided in the embodiment of this specification also includes: the database middleware sends a resume request to the second database, wherein the resume request carries the transaction identifier and point information recorded by the database middleware; the second database determines, based on the resume request, whether the transaction identifier recorded by the database middleware corresponds to the target transaction, and whether the point information recorded by the database middleware corresponds to the target resume point.

[0105] The database middleware can also send a resume request to the second database based on the recorded transaction identifier and site information, so that the second database executes step 206 to resume the transaction in response to the resume request. The second database can determine whether the transaction identifier recorded by the database middleware corresponds to a target transaction that supports resume. If it is determined that the transaction identifier corresponds to the target transaction, it can further determine whether the site information matches the target resume site corresponding to the target transaction, that is, determine whether the site information corresponds to the target resume site. When it is determined that the site information matches the target resume site corresponding to the target transaction, the transaction is resumed. During the transaction resumption process, the second database restores the binary log before the target resume site corresponding to the site information in its log cache area to continue executing the target transaction based on the restored binary log.

[0106] For the different target resuming locations mentioned above, the binary logs restored when resuming transactions may be different. For the case where the second database determines a target resuming location for the target transaction (i.e. the second and third cases mentioned above), please refer to the previous introduction. After determining the target resuming location, corresponding adjustments can be made to keep the engine layer transaction information consistent with the data in the log buffer. In this case, the second database can directly restore all binary logs in the log buffer as target binary logs to achieve transaction resuming and continue to execute the target transaction.

[0107] In the case where the second database determines the last two locations of the target transaction as the target retransmission locations (i.e. the first case mentioned above), if the location information recorded by the database middleware corresponds to the last location of the target transaction, the second database can directly restore all binary logs in the log buffer area to the target binary logs, and then continue to execute the target transaction. If the location information recorded by the database middleware corresponds to the second-to-last location of the target transaction, the second database can adjust both the engine layer transaction information and the log buffer area to maintain the second-to-last location, and then resume the transaction.

[0108] Correspondingly, in the case where the transaction identifier recorded by the database middleware corresponds to the target transaction, and the site information recorded by the database middleware corresponds to the target resume site, the second database restores the binary log before the target resume site corresponding to the site information in the target transaction in the log buffer area as the target binary log of the target transaction, including: when the transaction identifier recorded by the database middleware corresponds to the target transaction, the target resume site includes the last two sites of the target transaction, and the site information recorded by the database middleware corresponds to the second to last site of the target transaction, the second database deletes the binary log corresponding to the last site of the target transaction stored in the log buffer area; the second database restores the remaining binary log of the target transaction in the log buffer area as the target binary log of the target transaction. The second data volume also rolls back the engine layer transaction information to the state corresponding to the second to last site.

[0109] After the transaction is resumed, the database middleware can resend the data operation statement corresponding to the last site to the second database so that the second database executes the statement, regenerates the engine layer log and binary log corresponding to the statement, and feeds back the execution result of the statement to the client device.

[0110] The data management method provided by the embodiments of the present specification may further include: the database middleware connects the second database to the client device in response to a database switching instruction; when the transaction identifier recorded by the database middleware does not correspond to the target transaction, or the location information recorded by the database middleware does not correspond to the target resume transmission location, the database middleware disconnects the second database from the client device.

[0111] After the second database is ready to provide services, the database middleware can respond to the database switching instruction to connect the second database to the client device, and then assist the second database in transaction resuming. If the second database determines that the transaction identifier recorded by the database middleware does not correspond to the target transaction, or the location information recorded by the database middleware does not correspond to the target resuming location, it can report an error to the database middleware.

[0112] If the transaction identifier recorded by the database middleware does not correspond to the target transaction, it can be determined that the transaction corresponding to the transaction identifier does not support retransmission, such as the database middleware record may be incorrect, or the transaction information corresponding to the transaction identifier is missing, and the retransmission of the transaction information can be prohibited. If the location information recorded by the database middleware does not correspond to the target retransmission location, it may be that the location information record is incorrect or the corresponding transaction information is missing, and the corresponding transaction cannot be completely resumed. In both cases, the database middleware can disconnect the second database from the client device, such as only disconnecting the connection based on the transaction. The database middleware can also reconnect the second database with the client device, and then the client device can resend the unfinished transaction request to the second database so that the second database re-executes the unfinished transaction.

[0113] Figure 7 1 is a flowchart of a database switching and transaction continuation after switching provided by an embodiment of this specification. This flow can be cross-referenced with the above-mentioned introduction on database switching and transaction continuation. Figure 7 As shown in the figure, when the database performs active / standby switching, the database middleware Proxy and the database instance kernel need to work together to complete the transaction continuation, ultimately achieving lossless switching for users.

[0114] In the database switching process, the second database starts switching in response to the database switching instruction, and the executed transaction can be reconstructed through the Undolog. For example, the second database first replays the physical log of the first database and restores all transaction information before the first database crashes to the Undolog. After that, the second database scans the Undolog to determine whether each transaction has been committed or rolled back. If the transaction has been committed or rolled back, the transaction information of the transaction can be directly synchronized, so as to restore all transaction information before the first database crashes. After the transaction information is restored, the second database traverses the Binlog (that is, the data in the log buffer area) transmitted before the first database, which contains the private Binlog data of all executing transactions, and obtains the Binlog site information of the executing transaction. Compare the engine layer transaction information with the Binlog data in the log buffer area to check whether the two are consistent, that is, determine whether the engine layer transaction information and the data in the log buffer area are complete. Based on this, the list of transactions executing before the database switch can be determined, and the target transaction supporting the resuming and the corresponding target resuming site can be obtained. This list is also the list of resumable transactions. After obtaining the transaction list, a background thread can be started to wait for the Proxy to initiate a transaction retransmission request. For transactions that cannot be resumed, the database will be rolled back directly.

[0115] In the transaction retransmission process, after the second database completes crash recovery and can provide services, the Proxy re-initiates a new connection, connects the client device to the second database, and sends its recorded trx_id and the last successfully received Undo_no to the second database. The second database checks whether the trx_id and Undo_no issued by the Proxy are allowed to be resumed based on the list of resumable transactions determined in the database switching step. If not, an error is reported to the Proxy, and the Proxy disconnects from the user; if so, the second database restores the transaction information corresponding to the trx_id to the current link with the client device, and finds the private Binlog data of this transaction in the log buffer area, and restores it to the private Binlog file of the new link. Finally, success is returned to the Proxy, and the Proxy can wait for the client device to continue to send new statements of the transaction. In some embodiments, the Proxy replays the last statement before the first database is switched.

[0116] In the embodiment of this specification, the physical log can be used to synchronize the physical information of the engine layer in the master and standby databases, and the physical information includes the storage information of the data on the disk and the transaction information. In the embodiment of this specification, the first database uses RDMA to transmit the private Binlog data of the executing transaction to the log buffer area of ​​the second database in real time, which effectively takes into account the distributed data sharing and performance issues. When performing the master-standby database switch, the second database recovers the executing transaction information from the physical log and recovers the executing Binlog data from the log buffer area. By adding the two fields trx_id and Undo_no, the engine layer transaction information and the Binlog data location information are checked to see if they match, and the transaction that can support the continuation is determined to ensure the data accuracy of the transaction continuation. In this way, the lossless continuation of the executing transaction during the database switch can be achieved, which can improve the data management effect and the application scenario of the data management system. The proxy cooperates with the database kernel to perform the continuation operation of the specific transaction, which can realize the retention of the transaction under abnormal circumstances and realize the user-insensitive switching of the database master and standby.

[0117] The embodiments of this specification can solve the problem that when the database product performs a fast master-slave switch, the executing Binlog transaction will be interrupted and rolled back, causing the user link to report an error and exit. The embodiments of this specification can ensure that the user link is not interrupted during active database operation and maintenance (such as kernel minor version upgrades, computing node migration, first database machine downtime, serverless cross-machine pop-up, etc.), greatly improving the stability of user transaction processing and user experience.

[0118] In summary, in the data management method provided by the embodiment of the present specification, in the process of executing a data operation statement (such as a target statement) using the main database (such as the first database), the target data can be stored in the log buffer area of ​​the standby database (such as the second database) based on the binary log generated by executing the target statement, so as to synchronize the binary log of the transaction being executed and the execution site of the transaction between the main and standby databases. When performing a master-standby switchover of the database, the target transaction that supports the continuation and the corresponding target continuation site in the transaction being executed by the main database can be determined based on the data stored in the log buffer area of ​​the standby database, and then the target binary log of the target transaction can be restored. In this way, after the master-standby database is switched, the standby database can continue to execute the target transaction from the execution site of the main database based on the target binary log, which can reduce the impact of the master-standby database switchover on the transaction execution, reduce the situation where the transaction request issued by the client device cannot receive a response due to the switchover of the master-standby database, and thus improve the data management effect.

[0119] The data management system in the embodiment of this specification may include a first data management device, a second data management device and a third data management device, and the three data management devices may correspond to the first database, the second database and the database middleware respectively. Corresponding to the above data management method embodiment, this specification also provides a data management device embodiment.

[0120] Figure 8 This is a structural diagram of a first data management device provided in an embodiment of the present specification. The first data management device is used to: after executing a target statement, store target data in a log buffer area of ​​a second database based on a binary log generated by executing the target statement; wherein the target statement belongs to any data operation statement in a multi-statement transaction, and the target data includes the binary log corresponding to the target statement, the transaction identifier of the multi-statement transaction to which the target statement belongs, and the location information of the target statement in the multi-statement transaction.

[0121] Optionally, the first data management device is used to: after executing the target statement, store the target data in a target slot in the log buffer area of ​​the second database based on the binary log generated by executing the target statement; wherein different slots in the binary log buffer area are used to store binary logs of different multi-statement transactions, and the target slot corresponds to the multi-statement transaction to which the target statement belongs.

[0122] Optionally, the first data management device is used to: after executing the target statement, based on the binary log generated by executing the target statement, store the target data in the target sub-block in the target slot in the log buffer area of ​​the second database; wherein different sub-blocks in each slot in the binary log buffer area are used to store data corresponding to different statements, the header of the target sub-block is used to store the transaction identifier and location information of the multi-statement transaction, the data part of the target sub-block is used to store the binary log corresponding to the target statement, and the tail of the target sub-block is used to store the verification information of the header and the data part.

[0123] Alternatively, if Figure 8 As shown, the first data management device includes: a storage module 802, which is used to store target data in the log buffer area of ​​the first database based on the binary log generated by executing the target statement after the first database executes the target statement; a copy module 804, which is used to copy the target data stored in the log buffer area of ​​the first database to the log buffer area of ​​the second database; wherein the data storage format in the log buffer area of ​​the first database and the log buffer area of ​​the second database is the same.

[0124] Optionally, the first data management device includes: a task generation module, which is used to generate a synchronization task corresponding to the target statement based on the binary log generated by executing the target statement after executing the target statement, and add the synchronization task to the task queue; a task execution module, which is used to use the synchronization thread in the first database to read the synchronization task in the task queue, and based on the read synchronization task, store the binary log corresponding to the target statement in the log cache area of ​​the second database.

[0125] Optionally, the first data management device is used to: after executing the target statement, based on the binary log generated by executing the target statement, use the remote direct memory access component to store the target data in the log buffer area of ​​the second database.

[0126] Optionally, the first data management device further includes: a recording module, configured to record the location information corresponding to the target statement in the engine layer log after the first database executes the target statement.

[0127] Optionally, the first data management device also includes: a first sending module, which is used to store the target data in the log buffer area of ​​the second database based on the binary log generated by executing the target statement, and the first database sends the transaction identifier, location information and execution result of the multi-statement transaction to which the target statement belongs to the database middleware.

[0128] Fig. 9This is a schematic diagram of the structure of a second data management device provided in an embodiment of the present specification. The second data management device includes: a determination module 902, which is used to determine the target transaction that supports the resume transmission and the corresponding target resume transmission point based on the data in the log buffer area in response to the database switching instruction; a recovery module 904, which is used to restore the binary log before the target resume transmission point corresponding to the target transaction in the log buffer area to the target binary log of the target transaction, and continue to execute the target transaction based on the target binary log.

[0129] Optionally, the determination module 902 includes: an acquisition sub-module, which is used by the second database to obtain transaction information executed by the first database based on the engine layer log of the first database in response to the database switching instruction; and a determination sub-module, which is used by the second database to determine the target transaction that supports resumption and the corresponding target resumption location based on the transaction information and the data in the log cache area.

[0130] Optionally, the determination submodule is used for: the second database determines the transaction existing in both the transaction information and the log cache area as the target transaction supporting resumed transmission; for the target transaction, compares the site information in the transaction information with the site information in the log cache area to determine the target resumed transmission site corresponding to the target transaction.

[0131] Optionally, the determination submodule is used to: for the target transaction, when it is determined that each point information in the transaction information is the same as each point information in the log buffer area, determine the last two points of the target transaction as the target resume points.

[0132] Optionally, the determination submodule is used to: for the target transaction, when it is determined that the site information in the transaction information is more than the site information in the log buffer area, determine the last site of the target transaction in the log buffer area as the target resume site; for the target transaction, when it is determined that the site information in the log buffer area is more than the site information in the transaction information, determine the last site of the target transaction in the transaction information as the target resume site.

[0133] Optionally, the second data management device further includes:

[0134] The information rollback module is used to roll back the transaction information to the state where the statement corresponding to the target resume transmission point has been executed when the last point of the target transaction in the log buffer area is determined as the target resume transmission point;

[0135] The information deletion module is used to delete the target data corresponding to the statement after the target resume transmission point in the log buffer area when the last point of the target transaction in the transaction information is determined as the target resume transmission point.

[0136] Optionally, the recovery module 904 is used to: when the transaction identifier recorded by the database middleware corresponds to the target transaction, and the location information recorded by the database middleware corresponds to the target resume location, restore the binary log before the target resume location corresponding to the location information in the target transaction in the log cache area to the target binary log of the target transaction.

[0137] Optionally, the recovery module 904 is used to: when the transaction identifier recorded by the database middleware corresponds to the target transaction, the target resume location includes the last two locations of the target transaction, and the location information recorded by the database middleware corresponds to the penultimate location of the target transaction, delete the binary log corresponding to the last location of the target transaction stored in the log buffer area; and restore the remaining binary log of the target transaction in the log buffer area to the target binary log of the target transaction.

[0138] Fig.10 1 is a schematic diagram of the structure of a third data management device provided in an embodiment of the present specification. The third data management device includes: a receiving module 1002, which is used to receive the transaction identifier, location information and execution result of the target statement of the multi-statement transaction to which the target statement belongs sent by the first database; a recording module 1004, which is used for the database middleware to record the transaction identifier and location information of the multi-statement transaction, and send the execution result of the target statement to the client device.

[0139] Optionally, the third data management device also includes: a connection module, which is used by the database middleware to connect the second database with the client device in response to the database switching instruction; and a disconnection module, which is used by the database middleware to disconnect the second database from the client device when the transaction identifier recorded by the database middleware does not correspond to the target transaction, or the location information recorded by the database middleware does not correspond to the target resume transmission location.

[0140] Optionally, the third data management device also includes: a second sending module, used to send a resume request to the second database before the second database restores the binary log before the target resume site corresponding to the site information in the target transaction in the log cache area to the target binary log of the target transaction, wherein the resume request carries the transaction identifier and site information recorded by the database middleware; a judgment module, used for the database middleware second database to judge whether the transaction identifier recorded by the database middleware corresponds to the target transaction, and whether the site information recorded by the database middleware corresponds to the target resume site based on the resume request.

[0141] Optionally, the location information of the aforementioned target statement in the multi-statement transaction is represented by the number of undo logs after the target statement is executed.

[0142] In the embodiment of the present specification, in the process of executing a data operation statement (such as a target statement) using a main database (such as a first database), the target data can be stored in the log buffer area of ​​the standby database (such as a second database) based on the binary log generated by executing the target statement, so as to synchronize the binary log of the transaction being executed and the execution site of the transaction between the main and standby databases. When performing a master-standby switchover of the database, the target transaction that supports the continuation and the corresponding target continuation site in the transaction being executed by the main database can be determined based on the data stored in the log buffer area of ​​the standby database, and then the target binary log of the target transaction can be restored. In this way, after the master-standby database is switched, the standby database can continue to execute the target transaction from the execution site of the main database based on the target binary log, which can reduce the impact of the master-standby database switchover on the transaction execution, reduce the situation where the transaction request issued by the client device cannot receive a response due to the switchover of the master-standby database, and thus improve the data management effect.

[0143] The above is a schematic scheme of the data management device of this embodiment. It should be noted that the technical scheme of the data management device and the technical scheme of the corresponding data management method above belong to the same concept, and the details of the technical scheme of the data management device not described in detail can all be referred to the description of the technical scheme of the data management method above.

[0144] Fig.11 11 is a block diagram of a computing device provided in an embodiment of the present specification. The components of the computing device 1100 include, but are not limited to, a memory 1110 and a processor 1120. The processor 1120 is connected to the memory 1110 via a bus 1130, and the database 1150 is used to store data. The processor 1120 is used to execute a computer program / instruction, and when the computer program / instruction is executed by the processor, the steps executed by the first database, the second database or the database middleware in the above-mentioned data management method are implemented.

[0145] The computing device 1100 also includes an access device 1140 that enables the computing device 1100 to communicate via one or more networks 1160. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1140 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0146] In one embodiment of the present specification, the above components of the computing device 1100 and Fig.11 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.11 The computing device structure block diagram shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art may add or replace other components as needed. The computing device 1100 may be a mobile or stationary server or server cluster, or may be some stationary or mobile computing devices with large data storage capabilities. For example, a mobile computer or mobile computing device may include a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc., and a stationary computing device may include a desktop computer or a personal computer (PC, Personal Computer), etc.

[0147] One embodiment of the present specification also provides a computer-readable storage medium, which stores a computer program / instruction, and the steps of the above-mentioned data management method are implemented when the computer instruction is executed by the processor. The computer program / instruction includes a computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electric carrier signals and telecommunication signals.

[0148] An embodiment of the present specification also provides a computer program product, including a computer program / instruction, which, when executed in a processor, implements the steps executed by the first database, the second database or the database middleware in the above-mentioned data management method.

[0149] As for the computing device embodiment, the computer-readable storage medium embodiment and the computer program product embodiment, since they are basically similar to the data management method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the data management method embodiment.

[0150] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0151] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the described order of actions, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0152] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A data management method, comprising: After executing the target statement, the first database stores the target data in the log buffer area of ​​the second database based on the binary log generated by executing the target statement, and records the location information corresponding to the target statement in the engine layer log; wherein the target statement belongs to any data operation statement in a multi-statement transaction, and the target data includes the binary log corresponding to the target statement, the transaction identifier of the multi-statement transaction to which the target statement belongs, and the location information of the target statement in the multi-statement transaction; The second database obtains, in response to the database switching instruction, transaction information executed by the first database based on the engine layer log of the first database; The second database determines a target transaction supporting resume transmission and a corresponding target resume transmission location based on the transaction information and the data in the log buffer area; The second database restores the binary log before the target resume point corresponding to the target transaction in the log buffer area as the target binary log of the target transaction, and continues to execute the target transaction based on the target binary log.

2. The method according to claim 1, after executing the target statement, the first database stores the target data in the log buffer area of ​​the second database based on the binary log generated by executing the target statement, comprising: After executing the target statement, the first database stores the target data in a target slot in a log buffer of the second database based on a binary log generated by executing the target statement; wherein different slots in the binary log buffer are used to store binary logs of different multi-statement transactions, and the target slot corresponds to the multi-statement transaction to which the target statement belongs.

3. The method according to claim 2, after executing the target statement, the first database stores the target data in the target slot in the log buffer area of ​​the second database based on the binary log generated by executing the target statement, comprising: After executing the target statement, the first database stores the target data in the target sub-block in the target slot in the log buffer area of ​​the second database based on the binary log generated by executing the target statement; Among them, different sub-blocks in each slot in the binary log buffer area are used to store data corresponding to different statements, the header of the target sub-block is used to store the transaction identifier of the multi-statement transaction and the location information, the data part of the target sub-block is used to store the binary log corresponding to the target statement, and the tail of the target sub-block is used to store the verification information of the header and the data part.

4. The method according to claim 1, after executing the target statement, the first database stores the target data in the log buffer of the second database based on the binary log generated by executing the target statement, comprising: After executing the target statement, the first database stores the target data in a log buffer area of ​​the first database based on the binary log generated by executing the target statement; The target data stored in the log cache area of ​​the first database is copied to the log cache area of ​​the second database; wherein the data storage format in the log cache area of ​​the first database and the log cache area of ​​the second database is the same.

5. The method according to claim 1, after executing the target statement, the first database stores the target data in the log buffer of the second database based on the binary log generated by executing the target statement, comprising: After executing the target statement, the first database generates a synchronization task corresponding to the target statement based on the binary log generated by executing the target statement, and adds the synchronization task to the task queue; The synchronization task in the task queue is read by using the synchronization thread in the first database, and based on the read synchronization task, the binary log corresponding to the target statement is stored in the log buffer area of ​​the second database.

6. The method according to claim 1, after executing the target statement, the first database stores the target data in the log buffer of the second database based on the binary log generated by executing the target statement, comprising: After executing the target statement, the first database uses a remote direct memory access component to store the target data in the log buffer area of ​​the second database based on the binary log generated by executing the target statement.

7. The method according to any one of claims 1 to 6, wherein the second database determines the target transaction supporting resume transmission and the corresponding target resume transmission location based on the transaction information and the data in the log buffer area, comprising: The second database determines the transaction existing in both the transaction information and the log buffer area as a target transaction supporting resume transmission; For the target transaction, the location information in the transaction information is compared with the location information in the log buffer area to determine the target resume location corresponding to the target transaction.

8. The method according to claim 7, wherein for the target transaction, comparing the location information in the transaction information with the location information in the log buffer area to determine the target resume location corresponding to the target transaction comprises: For the target transaction, when it is determined that each location information in the transaction information is the same as each location information in the log buffer area, the last two locations of the target transaction are determined as target resume locations.

9. The method according to claim 7, wherein for the target transaction, comparing the location information in the transaction information with the location information in the log buffer area to determine the target resume location corresponding to the target transaction comprises: For the target transaction, when it is determined that the location information in the transaction information is more than the location information in the log buffer area, the last location of the target transaction in the log buffer area is determined as the target resume location; For the target transaction, when it is determined that the location information in the log buffer area is more than the location information in the transaction information, the last location of the target transaction in the transaction information is determined as the target resume location.

10. The method according to claim 9, further comprising: When the last position of the target transaction in the log buffer is determined as the target resume position, the transaction information is rolled back to a state where the statement corresponding to the target resume position is executed; When the last position of the target transaction in the transaction information is determined as the target resume position, the target data corresponding to the statement after the target resume position in the log buffer is deleted.

11. The method according to any one of claims 1 to 6 or any one of claims 8 to 10, wherein after the first database executes the target statement, based on the binary log generated by executing the target statement, stores the target data in the log buffer area of ​​the second database, further comprising: The first database sends the transaction identifier of the multi-statement transaction to which the target statement belongs, the location information, and the execution result of the target statement to the database middleware; The database middleware records the transaction identifier and the location information of the multi-statement transaction, and sends the execution result of the target statement to the client device; The second database restores the binary log before the target resume point corresponding to the target transaction in the log buffer area as the target binary log of the target transaction, including: When the transaction identifier recorded by the database middleware corresponds to the target transaction and the location information recorded by the database middleware corresponds to the target resume location, the second database restores the binary log before the target resume location corresponding to the location information in the target transaction in the log cache as the target binary log of the target transaction.

12. The method according to claim 11, wherein when the transaction identifier recorded by the database middleware corresponds to the target transaction and the location information recorded by the database middleware corresponds to the target resume location, the second database restores the binary log before the target resume location corresponding to the location information in the target transaction in the log buffer area as the target binary log of the target transaction, including: In the case where the transaction identifier recorded by the database middleware corresponds to the target transaction, the target resume location includes the last two locations of the target transaction, and the location information recorded by the database middleware corresponds to the second to last location of the target transaction, the second database deletes the binary log corresponding to the last location of the target transaction stored in the log buffer area; The second database restores the remaining binary log of the target transaction in the log buffer area as the target binary log of the target transaction.

13. The method according to claim 11, further comprising: The database middleware connects the second database with the client device in response to the database switching instruction; When the transaction identifier recorded by the database middleware does not correspond to the target transaction, or the location information recorded by the database middleware does not correspond to the target resume location, the database middleware disconnects the second database from the client device.

14. The method according to claim 11, wherein when the transaction identifier recorded by the database middleware corresponds to the target transaction and the location information recorded by the database middleware corresponds to the target resume location, the second database restores the binary log before the target resume location corresponding to the location information in the target transaction in the log buffer area to before the target binary log of the target transaction, further comprising: The database middleware sends a resume request to the second database, wherein the resume request carries the transaction identifier and location information recorded by the database middleware; The second database determines, based on the resume request, whether the transaction identifier recorded by the database middleware corresponds to the target transaction, and whether the location information recorded by the database middleware corresponds to the target resume location.

15. According to the method of any one of claims 1 to 6, any one of 8 to 10, or any one of 12 to 14, the location information of the target statement in the multi-statement transaction is represented by the number of undo logs after the execution of the target statement.

16. A data management system, comprising a first database, a second database and database middleware; The first database is connected to the second database, and the database middleware is used to control the first database or the second database to connect to the client device to execute the data operation statement sent by the client device; The data management system is used to implement the data management method described in any one of claims 1 to 15.

17. A computing device comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps executed by the first database or the second database in any one of the methods described in claims 1 to 15 are implemented, or the steps executed by the database middleware in any one of the methods described in claims 11 to 15 are implemented.

18. A computer-readable storage medium storing a computer program / instruction, which, when executed by a processor, implements the steps performed by the first database or the second database in any one of the methods described in claims 1 to 15, or implements the steps performed by the database middleware in any one of the methods described in claims 11 to 15.

19. A computer program product, comprising a computer program / instruction, which, when executed in a processor, implements the steps performed by the first database or the second database in any one of the methods described in claims 1 to 15, or implements the steps performed by the database middleware in any one of the methods described in claims 11 to 15.

Images