Log backup method, device, database system and computing device

Abstract

The embodiments of this specification provide a log backup method, apparatus, database system, and computing device. The log backup method is applied to the database system and includes: obtaining the logical log of the first data operation instruction in the target transaction generated by triggering, where the target transaction includes multiple data operation instructions, the first data operation instruction is any one of the multiple data operation instructions, and the logical log is generated by triggering the modification operation of the master database data when executing the first data operation instruction; replicating and sending the logical log to the standby database in real time, and returning to the step of obtaining the logical log of the first data operation instruction in the target transaction generated by triggering; when the standby database receives the logical logs of the data operation instructions of the target transaction, submitting the logical logs of the data operation instructions. By transmitting the logical log in real time, the risk of degradation to asynchronous replication due to timeout is reduced, and the stability, availability, and response speed of the database system are enhanced.

Description

Technical Field

[0001] The embodiments of this specification relate to the technical field of databases, and particularly to a log backup method, apparatus, database system, and computing device. Background Art

[0002] With the development of database technology, in order to cope with the continuous growth of data volume and the complexity of project requirements, it is necessary to ensure data security and recoverability through log backup. Log backup is an important disaster recovery method, which can provide a basis for data recovery in the event of system failures or unexpected events.

[0003] Currently, in order to improve the reliability and consistency of log backup, semi-synchronous replication can be considered. Semi-synchronous replication stipulates that before the logical log of a transaction is committed in the primary database, it is necessary to wait for its logical log to be transmitted to the standby database and confirm that the standby database has successfully received the logical log before the logical log can be committed in the primary database, thereby reducing the risk of data loss and ensuring data reliability.

[0004] However, the primary database generally needs to execute multiple transactions. Once the process of transmitting the logical log of a certain transaction to the standby database takes too long, the primary database will degrade to asynchronous replication and immediately commit the logical log of this transaction, which may cause data inconsistency on the primary database, reducing the integrity and consistency of data on the database system, and further reducing the availability and stability of the database system. Therefore, there is an urgent need for a log backup method that maintains the high availability and high stability of the database system. Summary of the Invention

[0005] In view of this, the embodiments of this specification provide a log backup method. One or more embodiments of this specification also relate to a log backup apparatus, a database system, a computing device, a computer-readable storage medium, and a computer program product to solve the technical defects existing in the prior art.

[0006] According to the first aspect of the embodiments of this specification, a log backup method is provided, which is applied to a database system and includes:

[0007] Obtain the logical log of the first data operation instruction in the target transaction triggered and generated, where the target transaction includes multiple data operation instructions, and the first data operation instruction is any one of the multiple data operation instructions, and the logical log is triggered and generated when the first data operation instruction completes the modification operation on the primary database data;

[0008] Replicate and send the logical log to the standby database in real time, and return to the step of obtaining the logical log of the first data operation instruction in the target transaction triggered and generated;

[0009] When the standby database receives the logical logs of the data operation instructions of the target transaction, submit the logical logs of the data operation instructions.

[0010] According to the second aspect of the embodiments of the present specification, a log backup device is provided, which is applied to a database system and includes:

[0011] An acquisition module, configured to acquire the logical log of the first data operation instruction in the target transaction triggered and generated, where the target transaction includes multiple data operation instructions, the first data operation instruction is any one of the multiple data operation instructions, and the logical log is triggered and generated when the modification operation of the primary database data is completed by executing the first data operation instruction;

[0012] A backup module, configured to copy and send the logical log to the standby database in real time, and return to the step of acquiring the logical log of the first data operation instruction in the target transaction triggered and generated;

[0013] A submission module, configured to submit the logical logs of the data operation instructions when the standby database receives the logical logs of the data operation instructions of the target transaction.

[0014] According to the third aspect of the embodiments of the present specification, a database system is provided, including a primary database and a standby database;

[0015] The primary database is used to acquire the logical log of the first data operation instruction in the target transaction triggered and generated, copy and send the logical log to the standby database in real time, and return to the step of acquiring the logical log of the first data operation instruction in the target transaction triggered and generated, where the target transaction includes multiple data operation instructions, the first data operation instruction is any one of the multiple data operation instructions, and the logical log is triggered and generated when the modification operation of the primary database data is completed by executing the first data operation instruction;

[0016] The standby database is used to receive the logical log sent by the primary database, and generate and send a reception confirmation message of the logical log of the last data operation instruction to the primary database when receiving the logical log of the last data operation instruction;

[0017] The primary database is further used to submit the logical logs of the data operation instructions of the target transaction when receiving the reception confirmation message of the logical log of the last data operation instruction feedback by the standby database.

[0018] According to the fourth aspect of the embodiments of the present specification, a computing device is provided, including:

[0019] A memory and a processor;

[0020] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the above-mentioned log backup method are implemented.

[0021] According to the fifth aspect of the embodiments of the present specification, a computer-readable storage medium is provided, which stores computer programs / instructions. When the computer programs / instructions are executed by a processor, the steps of the above-mentioned log backup method are implemented.

[0022] According to the sixth aspect of the embodiments of the present specification, a computer program product is provided, including computer programs / instructions. When the computer programs / instructions are executed by a processor, the steps of the above-mentioned log backup method are implemented.

[0023] An embodiment of the present specification provides a log backup method, which is applied to a database system and includes: obtaining the logical log of the first data operation instruction in a target transaction triggered to be generated, where the target transaction includes multiple data operation instructions, and the first data operation instruction is any one of the multiple data operation instructions, and the logical log is triggered to be generated when the first data operation instruction is executed to complete the modification operation on the master database data; replicating and sending the logical log to the standby database in real time, and returning to the step of obtaining the logical log of the first data operation instruction in the target transaction triggered to be generated; when the standby database receives the logical logs of each data operation instruction of the target transaction, submitting the logical logs of each data operation instruction.

[0024] By means of real-time transmission of logical logs, the generated logical logs are pre-transferred to the standby database during the execution of the transaction, rather than waiting until the transaction is committed for transmission. This ensures that even in the case of network latency or other anomalies between the master database and the standby database, a high level of consistency can be maintained, reducing the risk of degradation to asynchronous replication due to timeouts. The transmission of logical logs is completed during the transaction processing, greatly reducing the transmission time in the transaction commit phase, avoiding performance jitter caused by long waiting for the standby database to confirm, and thus enhancing the stability and response speed of the database system. Through the synchronous backup of log backup, the risk of data asynchrony between the master database and the standby database is reduced, ensuring the integrity and consistency of data and improving the availability of the database system. Description of the Drawings

[0025] Figure 1 is a schematic flowchart of a log backup method;

[0026] Figure 2 is a flowchart of a log backup method provided by an embodiment of the present specification;

[0027] Figure 3 is a schematic flowchart of a log backup method provided by an embodiment of the present specification;

[0028] Figure 4 It is a flowchart of the processing procedure of a log backup method applied to a MySQL database provided by an embodiment of this specification;

[0029] Figure 5 It is a schematic structural diagram of a log backup device provided by an embodiment of this specification;

[0030] Figure 6 It is a schematic structural diagram of a database system provided by an embodiment of this specification;

[0031] Figure 7 It is a structural block diagram of a computing device provided by an embodiment of this specification. Detailed implementation manners

[0032] Many specific details are set forth in the following description in order to provide a thorough understanding of this specification. However, this specification can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the connotation of this specification. Therefore, this specification is not limited by the specific implementations disclosed below.

[0033] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a", "the", and "said" used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0034] It should be understood that although the terms first, second, etc. may be used in one or more embodiments of this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0035] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or reject.

[0036] First, the noun terms involved in one or more embodiments of this specification are explained.

[0037] Binary log (abbreviated as binlog): A logical log file in the MySQL database that records data operation instructions (SQL statements) for database changes in binary numbers and is used for data recovery and master-slave replication. It is crucial for ensuring data consistency and durability.

[0038] Binlog cache: A cache area that temporarily stores uncommitted logical logs during transaction processing. When the transaction is committed, these logical logs will be written into the permanent binary log file. It is applicable to long transactions to reduce disk I / O.

[0039] Relay log: A copy of the binary log transmitted by the master library that is saved on the slave library. Through the relay log, the modification operations of the data operation instructions on the master library data can be replayed to synchronize.

[0040] Relay log cache: A cache area that temporarily stores the relay log until the relay log is written into the relay log file on the local slave library. It improves replication efficiency and reduces disk I / O.

[0041] Transaction commit: In a database management system, when all data operation instructions in a transaction are successfully executed, it is the operation to permanently save these changes to the database. Transaction commit marks the end of the transaction and ensures that all changes of the transaction become part of the database state, while meeting the ACID characteristics of the transaction: Atomicity, Consistency, Isolation, and Durability.

[0042] Semi-synchronous replication: A MySQL replication mode that requires at least one slave library to confirm receiving and recording the logical log before the master library can complete the transaction commit, thereby improving data consistency.

[0043] Semi-synchronous replication timeout: A delay is set. If the standby database does not confirm receiving the logical log within this time, the master database will fall back to asynchronous replication mode to ensure that the overall performance of the database system is not affected too much.

[0044] MySQL: An open-source relational database system that supports multiple storage engines and is widely used in web applications and other applications.

[0045] Data operation instructions: Program statements that perform operations such as insert, update, or delete in the database. They are the main way for users to interact with the database system and are used to manage and modify data in the database. Taking SQL statements as an example, they include but are not limited to:

[0046] Insert (INSERT): Add new records to a database table.

[0047] Update (UPDATE): Modify the data of existing records.

[0048] Delete (DELETE): Remove records from a database table.

[0049] Select (SELECT): Query and retrieve data from the database.

[0050] Bulk insert (INSERT... SELECT): Select data from one table and insert it into another table.

[0051] Conditional update (UPDATE with JOIN): Use the JOIN statement to update information from multiple tables.

[0052] Conditional delete (DELETE with JOIN): Delete records based on the data in other tables.

[0053] Relational Database Service (RDS for short): A managed database service provided by the cloud platform that simplifies the process of database deployment, management, and expansion, and provides features such as high availability and automatic backup.

[0054] High Availability (HA for short): Describes the ability of a system design to provide continuous and stable services, and can remain running even in the case of component failures. For a database, it means minimizing downtime and the risk of data loss.

[0055] High Availability switch (HA switch): The process of automatically or manually transferring traffic to the standby database when the master database fails, ensuring service continuity and data availability. This process is as transparent as possible to end-users and does not affect project operations.

[0056] Currently, if an interruption or failure occurs during the transmission of the logical log of a large transaction, it may cause the standby database to lose some logical logs, thereby destroying data consistency. Therefore, for the processing of large-scale transactions, it is necessary to consider optimizing the semi-synchronous replication mechanism to reduce the impact of large transactions on the overall system performance while maintaining high availability and consistency of data. Semi-synchronous replication stipulates that before the logical log of a transaction is committed in the primary database, it needs to wait until the logical log is transmitted to the standby database and it is confirmed that the standby database has successfully received the logical log before the logical log can be committed in the primary database.

[0057] However, semi-synchronous replication stipulates that before the logical log of a transaction is committed in the primary database, it needs to wait until the logical log is transmitted to the standby database and it is confirmed that the standby database has successfully received the logical log before the logical log can be committed in the primary database, so as to reduce the risk of data loss and ensure data reliability. As Figure 1 shown, Figure 1 Fig. shows a schematic flowchart of a log backup method:

[0058] In a database management system (Datebase Management System, abbreviated as DBMS), the primary database executes Transaction 1 as a large transaction, and it is necessary to transmit the logical log.000001 of this transaction from the primary database to the standby database. During the transmission process, since there is only one transmission channel between the primary and standby databases, other transactions (Transaction 2 and Transaction 3) need to wait for the logical log transmission and commit of Transaction 1 to complete before they can use this transmission channel to complete log backup. Among them, each transaction includes multiple data operation instructions. As an indivisible work unit, the transaction needs to perform semi-synchronous replication on the logical log at the transaction level. When the number of transactions is large, there are many logical logs. After waiting for the logical log transmission of Transaction 1 (this large transaction) to complete, the logical logs of other transactions can be transmitted to the standby database for backup, and the entire instance will be in a non-writable state and become blocked. Once the log backup process of Transaction 1 takes too long, in order to maintain the performance and response speed of the database system, the primary database will degrade to asynchronous replication and immediately commit the logical log of Transaction 1 without waiting for the backup to complete until the backup progress of the logical log catches up with the generation progress of the logical log again. During this period, the data reliability of the primary database instance will drop significantly. If an HA switch operation occurs, data loss may occur.

[0059] In this case, it will not only cause the submission delay of the logical logs of subsequent transactions, but may also cause the performance of the primary database to decline. In extreme cases, it may even lead to the primary database responding slowly or being unable to process new requests in a timely manner, resulting in data loss, causing data asynchronization between the primary database and the standby database, and reducing the availability and stability of the database system.

[0060] In view of the above problems, in this specification, a log backup method is provided. This specification also relates to a log backup device, a database system, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail one by one in the following embodiments.

[0061] Refer to Figure 2 , Figure 2 FIG. shows a flowchart of a log backup method provided by an embodiment of this specification. This method is applied to a database system and includes the following specific steps:

[0062] Step 202: Obtain the logical log of the first data operation instruction in the target transaction triggered to be generated, where the target transaction includes multiple data operation instructions, the first data operation instruction is any one of the multiple data operation instructions, and the logical log is triggered to be generated when the modification operation of the main database data is completed by executing the first data operation instruction.

[0063] A database system is a software system for storing and managing data in a database, generally running on one or more servers. The database system not only provides persistent storage of data, but also supports functions such as efficient access, update, query, and management of data. The database system includes a main database and a standby database, which complete functions such as storage, replication, synchronization, backup, and recovery of data in the database, ensuring high availability, consistency, and persistence of the data. For example, MySQL, or for another example, the Oracle database system, and also for another example, PostgreSQL. The database system can be applied to a variety of project scenarios, such as online retail, inventory management, social media user data storage, logistics distribution, intelligent traffic flow analysis, e-commerce, enterprise resource planning (Enterprise Resource Planning, abbreviated as ERP), energy monitoring, production control, file management, smart city, environmental monitoring and other big data scenarios.

[0064] The main database is a database instance in the database system used to modify data and generate logical logs. The standby database is a database instance in the database system used to receive logical logs from the main database and complete log backup. The main database and the standby database can run on different servers respectively to form a distributed storage, or can run on the same server to form a single-machine multi-instance deployment. Generally, the main database directly interfaces with users or application programs, while the standby database does not directly interface with users or application programs. For example, MySQL can run on a Linux server, adopting the master-slave mode (Master-Slave) or the master-master mode (Master-Master), where the main database and the standby database can be on different physical servers, or can be on the same server through different ports and service instances.

[0065] The master database data is the project data stored on the master database, including table structures, indexes, views, and other database objects, etc. Modification operations on the master database data (such as insert, update, and delete, etc.) are completed through data operation instructions, ensuring the implementation of project logic and the consistent and complete storage of data. For example, on the master database of MySQL, there is stored the order form data of an e-commerce project, including: 1. order_id (INT, primary key): the unique identifier of the order; 2. customer_id (INT): the unique identifier associated with the customer; 3. order_date (DATETIME): the timestamp when the order was created; status (VARCHAR): the order status (such as pending, shipped, completed, etc.); 4. total_amount (DECIMAL): the total amount of the order.

[0066] A transaction is a specific set of data operation instructions that are executed as a whole and either all complete or none complete. A transaction is the basic unit for modifying the master database data in a database system. The target transaction is the transaction being executed. The target transaction includes multiple data operation instructions. After the data operation instructions complete the modification operations on the master database data, it triggers the generation of a logical log of the data operation instructions. The logical log records the modification operations on the master database data, which is used to ensure data persistence and consistency and support subsequent log backup and recovery operations. Once all the data operation instructions in the target transaction are successfully executed and the logical log is committed, the logical log will be written to persistent storage. For example, when a user places an order, it triggers a target transaction that contains multiple data operation instructions, such as inserting a new order record, updating the inventory quantity, and recording payment information. These operations must be completed as an atomic unit to ensure the integrity and consistency of the order form data in the e-commerce project. If any step fails, the entire transaction will roll back, undoing all the changes made, ensuring that the status of the order form data stored in the master database remains unchanged. After each data operation instruction is executed, the system generates a corresponding logical log, recording the completed modification operations.

[0067] Data manipulation instructions are specific code language commands issued by users or applications to the main database for performing operations such as Create, Read, Update, and Delete. Data manipulation instructions are the main way for users to interact with the database system and are used to manage and modify data in the database. For example, SQL statements in MySQL: 1. INSERT: Add a new record to a database table. In MySQL, INSERT INTO orders (customer_id, order_date, status, total_amount) VALUES (12345, NOW(), 'Pending', 299.99); Insert a new order record into the orders table; 2. UPDATE: Modify the data of existing records. UPDATE orders SET status = 'Shipped' WHERE order_id = 12345; Update the order status to 'Shipped'; 3. DELETE: Remove records from a database table. DELETE FROM orders WHERE order_id = 12345 AND status = 'Cancelled'; Delete a cancelled order; 4. SELECT: Query and retrieve data from the database. SELECT * FROM customers WHERE city = 'A'; Retrieve information about all customers from location A.

[0068] The logical log of data operation instructions is a serialized form entry that records the changes made to the data in the primary database through data operation instructions. It is used to ensure data persistence and consistency and support subsequent log backup and recovery operations. The logical log of data operation instructions is generated when the modification operation on the primary database data is completed by executing the data operation instructions. Generally, the logical logs of multiple data operation instructions form a logical log file in the form of serialized entries. For example, in MySQL, the binary log (binlog) file of SQL statements, such as mysql-bin.000001, where: mysql-bin.000001 indicates that this is the binary log file numbered 000001. MySQL numbers these files in sequence. Whenever a log file reaches a certain size or meets other conditions, the system will create a new log file and continue to record. This log file contains all the operation instructions (SQL statements) for modifying the database and is stored in binary format. Each log entry records a specific database operation, including but not limited to: 1. Event type: Identifies whether this is a start, commit, rollback, or specific data change event; 2. Timestamp: Records the time when the operation occurred; 3. Transaction identifier: If transactions are enabled, it contains the unique identifier of the transaction; 4. Table information: The database name and table name involved; 5. Row-level changes: For each row change, the values before and after the modification are detailedly recorded; 6. SQL statement: In statement-based replication mode, the executed SQL statement is directly recorded; while in row-based replication mode, the specific changes to the affected rows are recorded.

[0069] The first data operation instruction is the latest executed and completed data operation instruction in the target transaction, and it is executed according to the transaction logic order update of the target transaction. The logical log of the first data operation instruction is the logical log record generated when the modification operation on the primary database data is completed by executing the first data operation instruction.

[0070] Exemplarily, the database of an e-commerce project uses the MySQL database system (database management system) of Relational Database Service (RDS) for storage and management. This MySQL database system runs on a distributed server cluster, and a primary database server and multiple standby database servers are configured for MySQL. There are 3 transactions in the transaction queue of the primary database instance: Transaction 1, Transaction 2, and Transaction 3. The currently executing target transaction is Transaction 1. Transaction 1 is a transaction triggered by a user placing an order, and this transaction contains 3 SQL statements:

[0071] SQL statement 1. Insert a new order record: "INSERT INTO orders (customer_id, order_date, status, total_amount) VALUES (12345, NOW(), 'Pending', 299.99)";

[0072] SQL statement 2. Update inventory information: "UPDATE products SET stock_quantity = stock_quantity - 1 WHERE product_id = 56789;";

[0073] SQL statement 3. Record payment information: "INSERT INTO payments (order_id, payment_method, amount_paid, payment_date) VALUES (12345, 'Credit Card', 299.99, NOW());".

[0074] Currently, SQL statement 1 has just been executed, triggering the generation of an entry binlog event sql_1 in the binary log file mysql-bin.000001 of this SQL statement.

[0075] Obtain the logical log of the first data operation instruction in the target transaction that is triggered and generated, laying a data foundation of the logical log for subsequent backup and transmission.

[0076] Step 204: Real-time copy and send the logical log to the standby database, and return to the step of executing to obtain the logical log of the first data operation instruction in the target transaction that is triggered and generated.

[0077] Real-time copy and send the logical log to the standby database. The specific method is: real-time copy the logical log to obtain a logical log copy, and send the logical log copy to the standby database.

[0078] Among them, for real-time copying of the logical log to obtain a logical log copy, an optional method is: through a caching mechanism, real-time copy the logical log to obtain a logical log copy; another optional method is: through a file system snapshot, real-time copy the logical log to obtain a logical log copy; another optional method is: adopt a network stream data format to real-time copy the logical log to obtain a logical log copy, which is not limited here.

[0079] Among them, there is an optional way to send the logical log copy to the standby database: send the logical log copy to the standby database through a reliable data transfer protocol. For example, the TCP protocol uses a three-way handshake. After the sending is completed, it is necessary to receive the receiving confirmation message feedback from the standby database to confirm the successful sending. Another optional way is: send the logical log copy to the standby database through a message queue. For example, use Kafka or RabbitMQ to asynchronously transfer the logical log copy to the standby database. This increases the flexibility and fault tolerance of the system, which is not limited here.

[0080] It should be noted that in steps 102 and 104, the processes of obtaining the logical log, replicating and sending the logical log, and the execution of the data operation instruction and triggering the generation of the logical log are executed in isolation. They can be executed by configuring isolated threads respectively.

[0081] Exemplarily, through the cache mechanism, the logical log entry binlog event sql_1 is copied to the memory buffer to obtain the logical log copy binlog event sql_1.entry. The logical log copy binlog event sql_1.entry is sent to the standby database server through the TCP protocol, and after receiving the ACK confirmation message returned by the standby database, it is confirmed that the sending is successful, and the binary log entry binlog event sql_2 of the newly triggered SQL statement 2 is returned for execution.

[0082] Replicate and send the logical log to the standby database in real time, and return to the step of obtaining the logical log of the first data operation instruction in the target transaction that is triggered and generated. By means of real-time transmission of the logical log, it is pre-transferred to the standby database during the execution of the transaction, reducing the risk of degradation to asynchronous replication due to timeout, avoiding performance jitter caused by long waiting for the standby database to confirm, thereby enhancing the stability and response speed of the database system, reducing the risk of data asynchronization between the master database and the standby database, and ensuring data integrity and consistency.

[0083] Step 206: When the standby database receives the logical logs of the respective data operation instructions of the target transaction, commit the logical logs of the respective data operation instructions.

[0084] There is an optional way for the standby database to receive the logical logs of the respective data operation instructions of the target transaction: the master database receives the receiving confirmation message of the logical logs of the respective data operation instructions feedback from the standby database. Another optional determination method is: the master database receives the receiving confirmation message of the logical log of the last data operation instruction feedback from the standby database. Another optional determination method is: the standby database stores the logical logs of the respective data operation instructions of the target transaction, which is not limited here.

[0085] It should be noted that only when the logical logs of all data operation instructions for the target transaction are committed can it be determined that the target transaction has been successfully replicated to the standby database, thus ensuring that even if the primary database fails, the standby database can take over the service without losing any committed transaction information. Optionally, the standby database may also set a checkpoint to check the received logical logs to further enhance data security and durability.

[0086] Exemplarily, when the standby database server receives the logical log entries binlogevent sql_1, binlog event sql_2, and binlog event sql_3 corresponding to SQL statements 1, 2, and 3, it parses these entries one by one and replays the corresponding SQL statements. Once all relevant logical log entries have been successfully processed, the standby database confirms these changes and sends an ACK confirmation message to the primary database, determining that the target transaction has been successfully replicated to the standby database, commits the binary log file mysql-bin.000001 of transaction 1, and continues to execute transaction 2.

[0087] In the embodiments of this specification, by means of real-time transmission of logical logs, the generated logical logs are pre-transferred to the standby database during the execution of the transaction, rather than waiting until the transaction is committed for transmission. This ensures that even in the case of network latency or other anomalies between the primary database and the standby database, a high level of consistency can be maintained, reducing the risk of degradation to asynchronous replication due to timeouts. The transmission of logical logs is completed during the transaction processing, greatly reducing the transmission time in the transaction commit phase, avoiding performance jitter caused by long waiting for standby database confirmation, and thus enhancing the stability and response speed of the database system. Through the synchronous backup of log backups, the risk of data asynchrony between the primary database and the standby database is reduced, ensuring data integrity and consistency, and improving the availability of the database system.

[0088] In an optional embodiment of this specification, after step 202, the following specific steps are further included:

[0089] Cache the logical logs in the logical log cache of the primary database;

[0090] In step 204, the real-time replication and sending of logical logs to the standby database includes the following specific steps:

[0091] When the preset sending condition is met, replicate and send the logical logs cached in the logical log cache to the standby database in real time;

[0092] Step 206 includes the following specific steps:

[0093] When the standby database receives the logical log of each data operation instruction of the target transaction, it takes out the logical log of each data operation instruction from the logical log cache and submits it.

[0094] The logical log cache is a cache area that temporarily stores uncommitted logical logs. After being committed, the logical logs will be written to persistent storage. For example, the binary log cache (binlog cache).

[0095] Preset sending conditions are pre-set trigger conditions that trigger the real-time replication of logical logs in the primary database and send them to the standby database. Preset sending conditions can trigger the backup transmission process of logical logs based on different factors such as time, data volume or system performance to ensure efficient and real-time data synchronization. Preset sending conditions may include but are not limited to preset cycles, preset data volume thresholds of logical logs, and preset performance indicator thresholds of the primary database.

[0096] In an optional embodiment of the present specification, the preset sending condition is any one of a preset period, a preset data volume threshold of the logical log, and a preset performance indicator threshold of the main database.

[0097] The preset period is a pre-set time interval that serves as a timer to trigger the transmission of logical log backup. Setting a preset period can help to regularly transmit logical logs to maintain the timeliness of log backup transmission. For example, if the preset period is once every 5 minutes, the logical log will be copied and sent to the standby database in real time every 5 minutes.

[0098] The preset data volume threshold of the logical log is the preset logical log accumulation volume. Once this volume is exceeded, the transmission of the logical log to the standby database will be triggered. By setting the data volume threshold, batch transmission can be performed when the logical log accumulates to a certain extent, optimizing the use of network resources and reducing the overhead caused by frequent small-scale transmission. For example, if the preset data volume threshold is 1MB, then when the data in the logical log cache accumulates to 1MB or more, the system will trigger the backup transmission of the logical log.

[0099] The preset performance index threshold of the main database is the expected performance index restriction of the main database, such as CPU utilization, memory usage, disk I / O delay, etc. When the performance index reaches the preset threshold, insisting on the backup transmission of the logical log may affect the normal operation and service quality of the main database. You can adjust the transmission strategy of the logical log by monitoring these performance indicators to avoid large amounts of data transmission when the main database load is too high. For example, if the CPU utilization of the main database exceeds 80%, it is necessary to postpone the transmission of non-logical logs to give priority to ensuring the response speed and service stability of the main database.

[0100] Replicate the logical logs cached in the logical log cache in real time. The specific method is as follows: Replicate the logical logs cached in the logical log cache in real time to obtain a copy of the logical logs, and send the copy of the logical logs to the standby database.

[0101] Among them, to replicate the logical logs in real time to obtain a copy of the logical logs, an optional method is: Through the caching mechanism, replicate the logical logs cached in the logical log cache in real time to obtain a copy of the logical logs. Another optional method is: Through the file system snapshot, replicate the logical logs cached in the logical log cache in real time to obtain a copy of the logical logs. Another optional method is: Adopt the network stream data format to replicate the logical logs cached in the logical log cache in real time to obtain a copy of the logical logs, which is not limited here.

[0102] Among them, the optional method of sending the copy of the logical logs to the standby database can be referred to the description in step 104 above and will not be elaborated here.

[0103] Exemplarily, cache the entry binlogevent sql_1 in the binary log file mysql-bin.000001 of the SQL statement in the logical log cache binlog cache of the master database. When the preset period (5 minutes) is reached, send the copy of the logical log entries cached in the logical log cache binlog cache to the standby database server through the TCP protocol, and confirm the successful sending after receiving the ACK confirmation message returned by the standby database. Then return to execute and obtain the binary log entry binlog event sql_2 of the newly triggered SQL statement 2. When the standby database receives the logical logs of each SQL statement of transaction 1, take out the logical logs of each SQL statement from the logical log cache binlog cache, commit the binary log file mysql-bin.000001 of transaction 1, and continue to execute transaction 2.

[0104] In the embodiments of this specification, the logical log cache and the preset sending condition are combined. The logical log cache reduces frequent disk read and write operations, and the preset sending condition ensures that the logical logs are batch-transferred to the standby database at an appropriate time point, maintaining the timeliness of data synchronization and avoiding data transmission interference during high load of the master database, thereby improving the system response speed and service quality, and enhancing the guarantee of data integrity and consistency.

[0105] In an optional embodiment of this specification, the target transaction is the transaction being executed in the transaction queue;

[0106] Caching the logical logs in the logical log cache of the master database includes the following specific steps:

[0107] Within a preset time window, cache the logical log into the logical log cache of the primary database, update the target transaction to other transactions in the transaction queue, and return to execute the step of obtaining the logical log of the first data operation instruction in the target transaction triggered and generated.

[0108] The preset time window is a pre-set execution window for caching the logical log of the target transaction. Through the preset time window, the sharding of log backup is completed, and the execution of other transactions is prevented from being completely blocked by the logical log backup transmission of the target transaction. However, this method will modify the format of the logical log and has a certain degree of invasiveness to the database system.

[0109] Exemplarily, the preset time window is 1 second. That is, within each second, the primary database will cache the binary logs of all completed first data operation instructions in the currently executing target transaction into the binary log cache (binlogcache), and attempt to replicate and send these binary logs to the standby database in real time according to the preset sending conditions. If the target transaction is not completed at the end of this 1-second time window, the system will update the target transaction to the next transaction in the transaction queue, even if the previous target transaction has not fully committed all its binary logs: when the primary database is processing transaction 1 triggered by a user placing an order, it will cache and transmit as many binary logs generated by this transaction as possible within each 1-second time window. Once 1 second has passed, even if transaction 1 has not been completed, the system will start executing the next transaction in the transaction queue: transaction 2, allowing the binary logs of transaction 2 to also start being cached and backed up for transmission.

[0110] In the embodiments of this specification, through the operations of sharded execution, caching, and backup transmission, resource occupation for a long time on one transaction is avoided, ensuring the efficiency of multi-transaction parallel processing and improving the response speed of the system.

[0111] In an optional embodiment of this specification, after the logical log is replicated and sent to the standby database in step 204, the following specific steps are further included:

[0112] Cache the logical log into the relay log cache of the standby database.

[0113] The relay log is used on the standby database to save the replica of the logical log transmitted from the primary database. Through the relay log, it can be replayed to synchronize the modification operations of the data operation instructions on the primary database data. The standby database updates the standby database data by reading and applying the events in the relay log, thereby maintaining consistency with the primary database data and ensuring that even if the primary database fails, the standby database can take over the service and provide the latest data.

[0114] The relay log cache is a cache area for temporarily storing relay logs until the relay logs are executed and persistently stored. It allows the standby database to first accumulate a certain amount of log data in memory and then write it to persistent storage in batches. To speed up the replication process, the standby database may first store the received binary log content in the relay log cache. This can reduce frequent small-scale disk write operations because writing directly to disk is usually slower than writing to memory. When the cache reaches a certain size or after a certain time interval, the log data in the cache is flushed to disk to complete persistent storage. The relay log cache effectively improves replication efficiency and reduces disk I / O.

[0115] Exemplarily, after the standby database receives it, it stores it in the relay log cache. Then, the standby database will re-execute SQL statement 1: "INSERT INTO orders (customer_id, order_date, status, total_amount) VALUES (12345, NOW(), 'pending', 299.99)" on the standby database according to the relay log entries in this relay log cache to insert a new order record and ensure that the data of the standby database is consistent with that of the master database.

[0116] In the embodiments of this specification, frequent disk read and write operations are effectively reduced. By accumulating relay logs in the cache, replication efficiency is improved and the frequency of disk writes is reduced. The standby database is allowed to process logical logs in batches, optimizing resource usage and accelerating the response speed of the standby database.

[0117] In an optional embodiment of this specification, after caching the logical log in the relay log cache of the standby database, the following specific steps are further included:

[0118] When the relay log cache caches the logical logs of each data operation instruction of the target transaction, take out the logical logs of each data operation instruction from the relay log cache, and generate a relay log file based on the logical logs of each data operation instruction.

[0119] The relay log file is the log file of the relay logs persistently stored on the standby database. It stores a copy of the logical logs received from the master database and is used to replay these log events on the standby database to synchronize the data changes of the master database. When the relay log cache caches the logical logs of each data operation instruction of the target transaction, the relay log will be flushed to the relay log file on disk. This ensures that even if the system crashes, the unfinished log application process can be restored through the relay log file. Generally, the relay log files are renamed in sequence.

[0120] Exemplarily, when the relay log cache caches the binary logs of 3 SQL statements of transaction 1, the binary logs of the 3 SQL statements are retrieved from the relay log cache, and based on the binary logs of each SQL statement, a relay log file is generated, thereby quickly completing the backup transmission.

[0121] In the embodiments of this specification, by temporarily storing the logical logs in the relay log cache and, when the relay log cache caches the logical logs of each data operation instruction of the target transaction, persistently storing the relay log file, the frequent disk read and write operations are effectively reduced, the processing efficiency and response speed of the standby database are improved, and at the same time, it is ensured that even if the system fails, the unfinished log application can be restored from the relay log file, guaranteeing the consistency and persistence of the data and enhancing the stability and availability of the database system.

[0122] In an optional embodiment of this specification, before generating the relay log file based on the logical logs of each data operation instruction, the following specific steps are further included:

[0123] Obtain the triggered logical logs of related transactions;

[0124] Replicate and send the logical logs of related transactions to the standby database in real time;

[0125] Generating the relay log file based on the logical logs of each data operation instruction includes the following specific steps:

[0126] Integrate the logical logs of related transactions and the logical logs of each data operation instruction in the target transaction to generate a relay log file.

[0127] Related transactions are other transactions associated with the target transaction, generally other transactions in the same batch. During database replication and backup, to ensure data consistency and integrity, the logical logs of one or more related transactions are usually processed together with the logical logs of each data operation instruction in the target transaction to ensure that the logical logs of these transactions are backed up to the standby database as a whole. For example, in an e-commerce project, there are two transactions that occur almost simultaneously: Transaction A: A user places an order to purchase a product. Transaction B: An administrator adjusts the inventory quantity of the product. Although the two transactions are triggered by different users, they are operations on the same product and may affect the same database records. Therefore, they are regarded as "related transactions" that need to be processed in the same batch. When performing log backup, it should be ensured that the logs of these two transactions are processed together to maintain data consistency. Related transactions and the target transaction can be distinguished based on at least one of the number of instructions in the data operation instruction, the number of logical log entries, the timestamp, the transaction identifier, the amount of master database data operated on, and the row-level changes.

[0128] The logical log of the relevant transaction is a serialized form entry that records the changes made to the data in the primary database. The logical log of the relevant transaction contains all the change information to the database during the execution of the relevant transaction, including but not limited to changes in table structures, indexes, views, and other database objects. The logical log of the relevant transaction is generated when the modification operation of the primary database data is completed during the execution of the entire relevant transaction.

[0129] The logical log of the relevant transaction is replicated and sent to the standby database in real time. The specific method is as follows: replicate the logical log of the relevant transaction in real time to obtain a copy of the logical log of the relevant transaction, and send the copy of the logical log of the relevant transaction to the standby database.

[0130] Among them, to replicate the logical log of the relevant transaction in real time to obtain a copy of the logical log of the relevant transaction, an optional method is: through a caching mechanism, replicate the logical log of the relevant transaction in real time to obtain a copy of the logical log of the relevant transaction; another optional method is: through a file system snapshot, replicate the logical log of the relevant transaction in real time to obtain a copy of the logical log of the relevant transaction; another optional method is: adopt a network stream data format to replicate the logical log of the relevant transaction in real time to obtain a copy of the logical log of the relevant transaction, which is not limited here.

[0131] Among them, to send the copy of the logical log of the relevant transaction to the standby database, an optional method is: through a reliable data transfer protocol, send the copy of the logical log of the relevant transaction to the standby database. For example, the TCP protocol uses a three-way handshake method and needs to receive an acknowledgment message from the standby database after completion of the sending to confirm the successful sending. Another optional method is: through a message queue, send the copy of the logical log of the relevant transaction to the standby database. For example, use Kafka or RabbitMQ to asynchronously transfer the copy of the logical log of the relevant transaction to the standby database. This increases the flexibility and fault tolerance of the system, which is not limited here.

[0132] It should be noted that obtaining the logical log of the relevant transaction, replicating and sending the logical log of the relevant transaction, and the execution of the data operation instruction and the generation of the logical log of the relevant transaction triggered thereby are executed in isolation, and can be executed separately through configured isolated threads.

[0133] Exemplarily, after the main database finishes executing SQL statement 1 in transaction A, an entry binlog event sql_1 in the binary log file mysql-bin.000001 of this SQL statement is triggered and generated. The binlog event sql_1 is obtained and replicated in real time to the standby database, sent to the standby database server through the TCP protocol, and it is confirmed that the sending is successful after receiving the ACK confirmation message returned by the standby database. For related transaction B, which contains 10 SQL statements for updating inventory information and the 10 SQL statements are executed, corresponding binary log files mysql-bin.000002 will also be triggered and generated. It is copied as a replica and then sent to the standby database through a reliable data transfer protocol (such as TCP). The binary log files of transaction A and transaction B are integrated to obtain a relay log file.

[0134] In the embodiments of this specification, by processing the logical logs of the target transaction and related transactions together, the frequent disk read and write operations are effectively reduced, the processing efficiency and response speed of the standby database are improved. At the same time, it is ensured that even if a system failure occurs, the unfinished log application can be restored from the relay log file, guaranteeing data consistency and persistence, and enhancing the stability and availability of the database system. In addition, this method also ensures that even in complex project scenarios, different but related transactions can be correctly reflected in the standby database, avoiding possible data inconsistency problems.

[0135] In an optional embodiment of this specification, step 206 includes the following specific steps:

[0136] In the case where the main database receives the reception confirmation message of the logical log of the last data operation instruction fed back by the standby database, the logical logs of each data operation instruction are committed.

[0137] The last data operation instruction is the last executed data operation instruction in the target transaction. It means that this final command can only be executed if all previous operations must be successful, otherwise the entire transaction will be rolled back.

[0138] The logical log of the last data operation instruction is a serialized form entry that records the changes made to the main database data through the last data operation instruction, which is used to ensure data persistence and consistency and support subsequent log backup and recovery operations. The logical log of the last data operation instruction is triggered and generated when the modification operation of the main database data is completed by executing the last data operation instruction.

[0139] The receipt confirmation message is a signal or notification sent from the standby database to the primary database, indicating that the standby database has successfully received the logical log of the last data operation instruction. This confirmation mechanism is used in a semi-synchronous replication environment to ensure that the primary database commits a transaction only after confirming that the standby database has safely received the logical log, preventing the situation where the primary database commits without the standby database receiving the logical log of the last data operation instruction, and then having to "find" the logical log of the last data operation instruction from the committed logical log file to retransmit when the standby database requests retransmission. This helps reduce the risk of data loss and improve the reliability and consistency of the system.

[0140] Exemplarily, the SQL statement "record payment information" is the last data operation instruction in a transaction. Only when the primary database server receives the ACK confirmation message of the binary log of the last data operation instruction sent by the standby database server can it be determined that Transaction 1 has been successfully processed by both database instances, thus completing the entire transaction commit process.

[0141] In the embodiments of this specification, by ensuring that the primary database commits only after receiving the receipt confirmation message for the logical log of the last data operation instruction from the standby database, the risk of data asynchrony is effectively avoided. This approach not only reduces the possibility of data loss, enhances the reliability and consistency of the system, but also prevents unnecessary retransmission operations, improves replication efficiency and system response speed, thereby enhancing the overall performance and stability of the database system.

[0142] Corresponding to the above-mentioned multiple method embodiments, Figure 3 The flowchart of a log backup method provided by an embodiment of this specification is shown, as Figure 3 shown:

[0143] On the primary database, there is a task manager for real-time backup of global logical logs and a logical log cache. The task manager is used to manage and schedule the real-time backup tasks of global logical logs. In the task manager, there is a list of real-time backups of global logical logs for recording all logical log entries to be backed up and their statuses. The logical log cache is used to cache the generated logical logs. There are user threads and backup threads running on the primary database.

[0144] On the standby database, there is a relay log cache for caching the received logical logs. There are read-write threads running on the standby database.

[0145] Execute the user thread to modify the primary database data, trigger the generation of logical logs, and after completing the transmission of all logical log backups, commit the logical log file to complete persistent storage.

[0146] Execute the backup thread and send the real-time backup entries of the logical logs of each data operation instruction in the target transaction and the real-time backup entries of the logical logs of related transactions to the standby database.

[0147] Execute the read / write thread. Cache the real-time backup entries of the logical logs of each data operation instruction in the target transaction and the real-time backup entries of the logical logs of related transactions received through the relay log cache. Directly add the real-time backup entries of the logical logs of related transactions to the relay log file. When the relay log cache caches the real-time backup entries of the logical logs of each data operation instruction in the target transaction, take out the real-time backup entries of the logical logs of each data operation instruction from the relay log cache and add them to the relay log file after renaming.

[0148] The following combines the attached Figure 4 , taking the application of the log backup method provided in this specification in the MySQL database as an example, further illustrates the log backup method. Among them, Figure 4 shows the processing process flowchart of a log backup method provided in an embodiment of this specification, which is applied to the RDS MySQL database and includes the following specific steps:

[0149] Step 402: Obtain the binary log binlog of the SQL statements in the ETL (Extract, Transform, Load) transaction triggered to be generated. Among them, the ETL transaction includes multiple SQL statements, and the binary log binlog is triggered to be generated when the SQL statements are executed to complete the modification operation of the data in the master database.

[0150] Step 404: Cache the binary log binlog into the binary log cache binlogcache of the master database.

[0151] Step 406: When the preset period is reached, replicate and send the binary log binlog cached in the binary log cache binlog in real time to the standby database.

[0152] Step 408: Cache the binary log binlog into the relay log cache relay logcache of the standby database, and return to execute step 402.

[0153] Step 410: When the binary logs (binlogs) of the SQL statements of the ETL transactions are cached in the relay log cache, take out the binary logs (binlogs) of the SQL statements from the relay log cache, generate a relay log file based on the binary logs (binlogs) of the SQL statements, and send an acknowledgment message to the master database.

[0154] Step 412: When the master database receives the acknowledgment message for the binary logs (binlogs) of the SQL statements fed back by the standby database, take out the binary logs (binlogs) of the SQL statements from the binlog cache and submit them.

[0155] In the embodiments of this specification, through real-time transmission, the binary logs (binlogs) of large transactions are pre-transferred to the standby database during the execution of the transactions. When the large transactions are committed, only further file conversion needs to be done in the standby database, which greatly reduces the time-consuming for transmitting binary logs (binlogs) in the commit stage of large transactions. Semi-synchronous replication will not degenerate, nor will it cause the instance to be unwritable, effectively avoiding the timeout rollback and performance jitter of semi-synchronous replication, and there is no need to change the format of the binary log (binlog) file, which has no intrusion on the MySQL database ecosystem.

[0156] Corresponding to the above method embodiments, this specification also provides embodiments of a log backup device. Figure 5 It shows a schematic structural diagram of a log backup device provided by an embodiment of this specification. As Figure 5 shown, this device is applied to a database system and includes:

[0157] An acquisition module 502, configured to acquire the logical log of the first data operation instruction in a target transaction triggered to be generated, where the target transaction includes multiple data operation instructions, the first data operation instruction is any one of the multiple data operation instructions, and the logical log is triggered to be generated when the first data operation instruction completes a modification operation on the master database data;

[0158] A backup module 504, configured to replicate and send the logical log to the standby database in real time, and return to the step of executing to acquire the logical log of the first data operation instruction in the target transaction triggered to be generated;

[0159] A submission module 506, configured to submit the logical logs of the data operation instructions when the standby database receives the logical logs of the data operation instructions of the target transaction.

[0160] Optionally, this device further includes: a first cache module, configured to cache the logical log into the logical log cache of the master database;

[0161] The backup module 504 is further configured to: when a preset sending condition is met, copy and send the logical logs cached in the logical log cache to the standby database in real time;

[0162] The submission module 506 is further configured to: when the standby database receives the logical logs of the data operation instructions of the target transaction, retrieve the logical logs of the data operation instructions from the logical log cache and submit them.

[0163] Optionally, the preset sending condition is any one of a preset period, a preset data volume threshold of the logical logs, and a preset performance metric threshold of the primary database.

[0164] Optionally, the target transaction is a transaction being executed in the transaction queue;

[0165] The first cache module is further configured to: within a preset time window, cache the logical logs into the logical log cache of the primary database, update the target transaction to other transactions in the transaction queue, and return to execute the step of obtaining the logical logs of the first data operation instruction in the triggered target transaction.

[0166] Optionally, the device further includes:

[0167] The second cache module is configured to cache the logical logs into the relay log cache of the standby database.

[0168] Optionally, the device further includes:

[0169] The file generation module is configured to: when the relay log cache caches the logical logs of the data operation instructions of the target transaction, retrieve the logical logs of the data operation instructions from the relay log cache, and generate a relay log file based on the logical logs of the data operation instructions.

[0170] Optionally, the device further includes:

[0171] The related backup module is configured to obtain the logical logs of the triggered related transactions; copy and send the logical logs of the related transactions to the standby database in real time;

[0172] Correspondingly, the file generation module is further configured to:

[0173] Integrate the logical logs of the related transactions and the logical logs of the data operation instructions in the target transaction to generate a relay log file.

[0174] Optionally, the submission module 506 is further configured to: when the primary database receives the reception confirmation message of the logical log of the last data operation instruction fed back by the standby database, submit the logical logs of the data operation instructions.

[0175] In the embodiments of this specification, by means of real-time transmission of logical logs, the generated logical logs are pre-transferred to the standby database during the execution of a transaction, rather than waiting until the transaction is committed for transmission. This ensures that even in the case of network latency or other anomalies between the primary database and the standby database, a high level of consistency can be maintained, reducing the risk of degradation to asynchronous replication due to timeouts. The transmission of logical logs is completed during transaction processing, greatly reducing the transmission time in the transaction commit phase, avoiding performance jitter caused by long waiting for the standby database to confirm, and thus enhancing the stability and response speed of the database system. Through synchronous backup of log backups, the risk of data asynchronization between the primary database and the standby database is reduced, ensuring data integrity and consistency, and improving the availability of the database system.

[0176] The above is a schematic solution of a log backup device according to this embodiment. It should be noted that the technical solution of this log backup device and the technical solution of the above log backup method belong to the same concept. For the details not described in detail in the technical solution of the log backup device, reference can be made to the description of the technical solution of the above log backup method.

[0177] Corresponding to the above method embodiments, this specification also provides database system embodiments. Figure 6 shows a schematic structural diagram of a database system provided by an embodiment of this specification. As Figure 6 shown, the database system 600 includes a primary database 602 and a standby database 604;

[0178] The primary database 602 is configured to obtain the logical log of the first data operation instruction in the target transaction triggered to be generated, replicate and send the logical log to the standby database 604 in real time, and return to execute the step of obtaining the logical log of the first data operation instruction in the target transaction triggered to be generated. Wherein, the target transaction includes multiple data operation instructions, the first data operation instruction is any one of the multiple data operation instructions, and the logical log is triggered to be generated when the modification operation of the data in the primary database 602 is completed by executing the first data operation instruction;

[0179] The standby database 604 is configured to receive the logical log sent by the primary database 602, and in the case of receiving the logical log of the last data operation instruction, generate and send a reception confirmation message of the logical log of the last data operation instruction to the primary database 602;

[0180] The primary database 602 is further configured to, in the case of receiving the reception confirmation message of the logical log of the last data operation instruction fed back by the standby database 604, commit the logical logs of the respective data operation instructions of the target transaction.

[0181] Optionally, the primary database 602 is further configured to cache the logical logs in the logical log cache of the primary database 602, and when a preset sending condition is met, replicate and send the logical logs cached in the logical log cache to the standby database 604 in real time. When receiving the reception confirmation message of the logical log of the last data operation instruction feedback from the standby database 604, the logical logs of each data operation instruction are retrieved from the logical log cache for submission.

[0182] Optionally, the standby database 604 is further configured to cache the logical logs in the relay log cache of the standby database 604. When receiving the reception confirmation message of the logical log of the last data operation instruction feedback from the standby database 604, the logical logs of each data operation instruction are retrieved from the relay log cache, and based on the logical logs of each data operation instruction, a relay log file is generated.

[0183] In the embodiments of this specification, in the database system, by means of real-time transmission of logical logs, the generated logical logs are pre-transferred to the standby database during the execution of the transaction, rather than waiting until the transaction is committed for transmission. This ensures that even in the case of network latency or other anomalies between the primary database and the standby database, a high level of consistency can be maintained, reducing the risk of degradation to asynchronous replication due to timeouts. The transmission of logical logs is completed during the transaction processing, greatly reducing the transmission time in the transaction commit phase, avoiding performance jitter caused by long waiting for the standby database confirmation, and thus enhancing the stability and response speed of the database system. Through the synchronous backup of log backups, the risk of data asynchrony between the primary database and the standby database is reduced, ensuring data integrity and consistency, and improving the availability of the database system.

[0184] The above is a schematic solution of a database system according to this embodiment. It should be noted that the technical solution of this database system and the technical solution of the above log backup method belong to the same concept. For the details not described in the technical solution of the database system, reference can be made to the description of the technical solution of the above log backup method.

[0185] Figure 7 The structural block diagram of a computing device provided by an embodiment of this specification is shown. The components of the computing device 700 include but are not limited to a memory 710 and a processor 720. The processor 720 is connected to the memory 710 through a bus 730, and the database 750 is used to store data.

[0186] The computing device 700 also includes an access device 740, which enables the computing device 700 to communicate via one or more networks 760. Examples of such networks include the Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of wired or wireless network interface (e.g., Network Interface Controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, Worldwide Interoperability for Microwave Access (Wi-MAX) interface, Ethernet interface, Universal Serial Bus (USB) interface, cellular network interface, Bluetooth interface, Near Field Communication (NFC).

[0187] In one embodiment of this specification, the above components of the computing device 700 and Figure 7 other components not shown may also be connected to each other, for example, via a bus. It should be understood that Figure 7 the block diagram of the computing device shown is for illustrative purposes only and is not a limitation on the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0188] The computing device 700 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 700 can also be a mobile or stationary server.

[0189] Among them, the processor 720 is used to execute the following computer program / instructions, and when the computer program / instructions are executed by the processor, the steps of the above log backup method are implemented.

[0190] The above is a schematic solution of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the above log backup method belong to the same concept. For the details not described in detail in the technical solution of the computing device, reference can be made to the description of the technical solution of the above log backup method.

[0191] An embodiment of this specification also provides a computer-readable storage medium, which stores computer programs / instructions. When the computer programs / instructions are executed by a processor, the steps of the above log backup method are implemented.

[0192] The above is a schematic solution of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the above log backup method belong to the same concept. For the details not described in detail in the technical solution of the storage medium, reference can be made to the description of the technical solution of the above log backup method.

[0193] An embodiment of this specification also provides a computer program product, including computer programs / instructions. When the computer programs / instructions are executed by a processor, the steps of the above log backup method are implemented.

[0194] The above is a schematic solution of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the above log backup method belong to the same concept. For the details not described in detail in the technical solution of the computer program product, reference can be made to the description of the technical solution of the above log backup method.

[0195] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be executed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain implementations, multitasking and parallel processing are also possible or may be advantageous.

[0196] The computer instructions include computer program code, which may be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, mobile hard disks, magnetic disks, optical disks, computer memories, read-only memories (ROM), random access memories (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0197] It should be noted that for the foregoing method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of this specification are not limited by the described action sequence, because according to the embodiments of this specification, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this specification.

[0198] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0199] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The alternative embodiments do not elaborate on all the details and do not limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can understand and utilize this specification well. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A log backup method, applied to a database system, comprising: Acquire a logic log of a first data operation instruction in a target transaction that is triggered and generated, wherein the target transaction is a transaction being executed in a transaction queue, the target transaction includes multiple data operation instructions, the first data operation instruction is any one of the multiple data operation instructions, and the logic log is triggered and generated by executing the first data operation instruction to complete a modification operation on the primary database data; The primary database copies the logical log in real time and sends it to the standby database, and returns to the step of executing the logical log of the first data operation instruction in the target transaction generated by the acquisition trigger; When the standby database receives the logical log of each data operation instruction of the target transaction, the primary database submits the logical log of each data operation instruction.

2. The method according to claim 1, after obtaining the logical log of the first data operation instruction in the target transaction, further comprising: Cache the logical log into the logical log cache of the master library; The primary database copies the logical log in real time and sends it to the standby database, including: When the preset sending condition is met, the primary database copies the logical log cached in the logical log cache in real time and sends it to the standby database; When the standby database receives the logical log of each data operation instruction of the target transaction, the primary database submits the logical log of each data operation instruction, including: When the standby database receives the logical log of each data operation instruction of the target transaction, the primary database retrieves the logical log of each data operation instruction from the logical log cache and submits it.

3. According to the method of claim 2, the preset sending condition is any one of a preset cycle, a preset data volume threshold of the logical log, and a preset performance indicator threshold of the main library.

4. The method according to claim 2, wherein the main library caches the logical log into the logical log cache of the main library, comprising: Within a preset time window, the main library caches the logical log into the logical log cache of the main library, updates the target transaction to other transactions in the transaction queue, and returns to execute the step of obtaining the logical log of the first data operation instruction in the target transaction generated by the trigger.

5. The method according to any one of claims 1 to 4, further comprising: after the primary database copies the logical log in real time and sends it to the standby database: The logical log is cached in the relay log cache of the standby database.

6. The method according to claim 5, after the step of caching the logical log in the relay log cache of the standby database, further comprises: In the case where the relay log cache has cached the logical log of each data operation instruction of the target transaction, the logical log of each data operation instruction is taken out from the relay log cache, and a relay log file is generated based on the logical log of each data operation instruction.

7. The method according to claim 6, before generating a relay log file based on the logical log of each data operation instruction, further comprises: Get the logical log of the triggered and related transactions; Copy the logical log of the relevant transaction in real time and send it to the standby database; The generating of the relay log file based on the logical log of each data operation instruction includes: The logical logs of the related transactions and the logical logs of the data operation instructions in the target transaction are integrated to generate a relay log file.

8. The method according to claim 1, wherein the primary database submits the logical log of each data operation instruction of the target transaction when the standby database receives the logical log of each data operation instruction, comprising: When the primary database receives a reception confirmation message of the logical log of the last data operation instruction fed back by the standby database, the primary database submits the logical logs of each data operation instruction.

9. A log backup device, applied to a database system, comprising: an acquisition module, configured to acquire a logic log of a first data operation instruction in a target transaction that is triggered and generated, wherein the target transaction is a transaction being executed in a transaction queue, the target transaction includes multiple data operation instructions, the first data operation instruction is any one of the multiple data operation instructions, and the logic log is triggered and generated by executing the first data operation instruction to complete a modification operation on the primary database data; The backup module is configured to copy the logical log in real time from the primary database and send it to the standby database, and return to the step of executing the logical log of the first data operation instruction in the target transaction generated by the acquisition trigger; The commit module is configured to commit the logical log of each data operation instruction of the target transaction by the primary database when the standby database receives the logical log of each data operation instruction.

10. A database system, comprising a main database and a backup database; The master database is used to obtain the logic log of the first data operation instruction in the target transaction generated by the trigger, copy the logic log in real time and send it to the standby database, and return to execute the step of obtaining the logic log of the first data operation instruction in the target transaction generated by the trigger, wherein: The target transaction includes multiple data operation instructions, the first data operation instruction is any one of the multiple data operation instructions, and the logical log is generated by triggering the execution of the first data operation instruction to complete the modification operation on the main database data; The standby database is used to receive the logical log sent by the primary database, and upon receiving the logical log of the last data operation instruction, generate and send a reception confirmation message of the logical log of the last data operation instruction to the primary database; The master database is further configured to submit the logical logs of the data operation instructions of the target transaction upon receiving a reception confirmation message of the logical log of the last data operation instruction fed back by the standby database.

11. According to the database system of claim 10, the main database is also used to cache the logical log in the logical log cache of the main database, and when the preset sending conditions are met, the logical log cached in the logical log cache is copied in real time and sent to the standby database, and when the reception confirmation message of the logical log of the last data operation instruction fed back by the standby database is received, the logical log of each data operation instruction is taken out from the logical log cache for submission.

12. According to the database system of claim 10 or 11, the standby database is further used to cache the logical log in the relay log cache of the standby database, and when receiving the reception confirmation message of the logical log of the last data operation instruction fed back by the standby database, the logical log of each data operation instruction is taken out from the relay log cache, and a relay log file is generated based on the logical log of each data operation instruction.

13. A computing device comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer program / instructions are executed by the processor, the steps of the method according to any one of claims 1 to 8 are implemented.

14. A computer-readable storage medium storing a computer program / instruction, wherein the computer program / instruction, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

15. A computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

Images