Log backup method and apparatus, database system, and computing device

Abstract

Embodiments of the present disclosure provide a log backup method and apparatus, a database system, and a computing device. The log backup method is applied to the database system, and comprises: acquiring a logical log of a first data operation instruction in a target transaction, the logical log being generated by triggering, wherein the target transaction comprises a plurality of data operation instructions, the first data operation instruction is any one of the plurality of data operation instructions, and the logical log is generated by triggering when the first data operation instruction is executed to complete a modification operation on data in a primary database; copying the logical log in real time and sending a copy of the logical log to a standby database, and returning to the step of acquiring a logical log of a first data operation instruction in a target transaction, the logical log being generated by triggering; and when the standby database receives logical logs of the data operation instructions of the target transaction, submitting the logical logs of the data operation instructions. By transmitting the logical logs in real time, the risk of degradation to asynchronous replication caused by timeout is reduced, and the stability, availability, and response speed of the database system are enhanced.

Description

Log backup methods, devices, database systems, and computing equipment Technical Field

[0001] This disclosure relates to the technical field of databases, and in particular to a log backup method, apparatus, database system, and computing device. Background Technology

[0002] With the development of database technology, in order to cope with the continuous growth of data volume and the increasing complexity of project requirements, log backup is necessary to ensure data security and recoverability. Log backup is an important disaster recovery method, providing a basis for data recovery in the event of system failure or unexpected events.

[0003] Currently, to improve the reliability and consistency of log backups, a semi-synchronous replication method can be considered. Semi-synchronous replication requires that the logical log of a transaction must wait for it to be transmitted to the standby database before it is committed on the primary database. Only after confirming that the standby database has successfully received the logical log can the logical log be committed on the primary database. This reduces the risk of data loss and ensures data reliability.

[0004] However, the primary database typically needs to execute multiple transactions. If the process of transmitting the logical log of a transaction to the standby database takes too long, the primary database will degrade to asynchronous replication and immediately commit the logical log of this transaction. This may cause data inconsistency on the primary database, reducing the integrity and consistency of data in the database system, and consequently reducing the availability and stability of the database system. Therefore, there is an urgent need for a log backup method to maintain the high availability and stability of the database system. Summary of the Invention

[0005] In view of this, embodiments of this disclosure provide a log backup method. One or more embodiments of this disclosure also relate to a log backup apparatus, a database system, a computing device, a computer-readable storage medium, and a computer program product.

[0006] According to a first aspect of the present disclosure, a log backup method is provided, applied to a database system, comprising: acquiring a logical log of a first data operation instruction in a target transaction that is triggered and generated, wherein the target transaction includes multiple data operation instructions, the first data operation instruction is any one of the multiple data operation instructions, and the logical log is triggered when the first data operation instruction completes a modification operation on the primary database data; copying the logical log in real time and sending it to a standby database, and returning to execute the step of acquiring the logical log of the first data operation instruction in the target transaction that is triggered and generated; and committing the logical log of each data operation instruction when the standby database receives the logical logs of each data operation instruction of the target transaction.

[0007] According to a second aspect of the present disclosure, a log backup apparatus is provided, applied to a database system, comprising: an acquisition module configured to acquire a logical log of a first data operation instruction in a target transaction that is triggered to be generated, wherein the target transaction includes multiple data operation instructions, the first data operation instruction is any one of the multiple data operation instructions, and the logical log is generated when the first data operation instruction completes a modification operation on the primary database data; a backup module configured to copy the logical log in real time and send it to a standby database, and return to the step of acquiring the logical log of the first data operation instruction in the target transaction that is triggered to be generated; and a submission module configured to submit the logical log of each data operation instruction when the standby database receives the logical log of each data operation instruction of the target transaction.

[0008] According to a third aspect of the present disclosure, a database system is provided, including a primary database and a secondary database. The primary database is configured to acquire a logical log of a first data operation instruction in a target transaction that is triggered and generated, replicate the logical log in real time and send it to the secondary database, and return to execute the step of acquiring the logical log of the first data operation instruction in the target transaction that is triggered and generated, wherein the target transaction includes multiple data operation instructions, the first data operation instruction is any one of the multiple data operation instructions, and the logical log is generated when the first data operation instruction completes a modification operation on the data in the primary database. The secondary database is configured to receive the logical log sent by the primary database, and, upon receiving the logical log of the last data operation instruction, generate and send a receipt confirmation message of the logical log of the last data operation instruction to the primary database. The primary database is further configured to commit the logical logs of each data operation instruction of the target transaction upon receiving the receipt confirmation message of the logical log of the last data operation instruction from the secondary database.

[0009] According to a fourth aspect of the present disclosure, a computing device is provided, including: a memory and a processor; the memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, wherein the computer programs / instructions, when executed by the processor, implement the steps of the above-described log backup method.

[0010] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the log backup method described above.

[0011] According to a sixth aspect of the present disclosure, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the log backup method described above.

[0012] One embodiment of this disclosure provides a log backup method applied to a database system, comprising: acquiring a logical log of a first data operation instruction in a target transaction that is triggered and generated, wherein the target transaction includes multiple data operation instructions, the first data operation instruction is any one of the multiple data operation instructions, and the logical log is generated when the first data operation instruction completes a modification operation on the primary database; copying the logical log in real time and sending it to a standby database; returning to the step of acquiring the logical log of the first data operation instruction in the target transaction that is triggered and generated; and committing the logical log of each data operation instruction when the standby database receives the logical log of each data operation instruction of the target transaction.

[0013] By transmitting logical logs in real time, the generated logical logs are pre-transmitted to the standby database during transaction execution, rather than waiting until transaction commit. This ensures a high level of consistency even under network latency or other anomalies between the primary and standby databases, reducing the risk of degradation to asynchronous replication due to timeouts. Log transmission is completed during transaction processing, significantly reducing transmission time during the transaction commit phase and avoiding performance fluctuations caused by long waits for standby database confirmation, thereby enhancing the stability and responsiveness of the database system. Synchronous backup of the logs reduces the risk of data inconsistency between the primary and standby databases, ensuring data integrity and consistency, and improving the availability of the database system. Attached Figure Description

[0014] Figure 1 is a flowchart illustrating a log backup method.

[0015] Figure 2 is a flowchart of a log backup method provided in an embodiment of this disclosure.

[0016] Figure 3 is a schematic flowchart of a log backup method provided in an embodiment of this disclosure.

[0017] Figure 4 is a flowchart of a log backup method for a MySQL database provided in an embodiment of this disclosure.

[0018] Figure 5 is a schematic diagram of the structure of a log backup device provided in an embodiment of this disclosure.

[0019] Figure 6 is a schematic diagram of the structure of a database system provided in an embodiment of this disclosure.

[0020] Figure 7 is a structural block diagram of a computing device provided in an embodiment of this disclosure. Detailed Implementation

[0021] Numerous specific details are set forth in the following description to provide a full understanding of this disclosure. However, this disclosure can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this disclosure. Therefore, this disclosure is not limited to the specific implementations disclosed below.

[0022] The terminology used in one or more embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this disclosure. The singular forms “a,” “the,” and “the” as used in one or more embodiments of this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this disclosure refers to and includes any or all possible combinations of one or more associated listed items.

[0023] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this disclosure, and similarly, second may also be referred to as first. Depending on the context, the word “if” as used herein may be interpreted as “when”, “in response to a determination”, or “when…”.

[0024] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this disclosure are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0025] First, the terms and concepts involved in one or more embodiments of this disclosure will be explained.

[0026] Binary log (binlog): A logical log file in MySQL databases that uses binary data to record data manipulation instructions (SQL statements) that modify the database. It is used for data recovery and master-slave replication. It is crucial for ensuring data consistency and durability.

[0027] Binary log cache: A cache area that temporarily stores uncommitted logical logs during transaction processing. When the transaction is committed, these logical logs are written to a permanent binary log file. Suitable for long transactions to reduce disk I / O.

[0028] Relay log: A copy of the binary log transmitted from the primary database to the standby database. The relay log can be replayed to synchronize data manipulation commands that modify data on the primary database.

[0029] Relay log cache: A cache area for temporary storage of relay logs until the relay logs are written to the local relay log file on the standby database. This improves replication efficiency and reduces disk I / O.

[0030] Transaction commit: In a database management system, after all data operation instructions within a transaction have been successfully executed, the changes are permanently saved to the database. Transaction commit marks the end of a transaction and ensures that all changes made by that transaction become part of the database state, while satisfying the ACID properties of transactions: Atomicity, Consistency, Isolation, and Durability.

[0031] Semi-synchronous replication: A MySQL replication mode that requires at least one standby database to confirm receipt and record the logical log before the master database can complete the transaction commit, thereby improving data consistency.

[0032] Semi-synchronous replication timeout: A delay is set. If the standby database does not confirm receipt of the logical log within this time, the master database will revert to asynchronous replication mode to ensure that the overall performance of the database system is not significantly affected.

[0033] MySQL: An open-source relational database system that supports multiple storage engines and is widely used in web applications and other applications.

[0034] Data manipulation commands: These are program statements that execute operations such as insert, update, or delete in the database. They are the primary way users interact with the database system and are used to manage and modify data in the database. Taking SQL statements as an example, they include, but are not limited to, the following.

[0035] INSERT: Adds a new record to a database table.

[0036] Update: Modifies the data in an existing record.

[0037] Delete (DELETE): Removes a record from a database table.

[0038] SELECT: Query and retrieve data from a database.

[0039] Batch insert (INSERT...SELECT): Select data from one table and insert it into another table.

[0040] Conditional update (UPDATE with JOIN): Use the JOIN statement to update information from multiple tables.

[0041] Conditional deletion (DELETE with JOIN): Deletes records based on data from other tables.

[0042] Relational Database Service (RDS): A managed database service provided by cloud platforms that simplifies the process of database deployment, management, and expansion, and provides features such as high availability and automatic backup.

[0043] High Availability (HA) describes a system design that provides continuous and stable service capabilities, maintaining operation even in the event of component failure. For databases, this means minimizing downtime and the risk of data loss.

[0044] High Availability Switching (HA switch): When the primary database fails, the process of automatically or manually transferring traffic to the standby database ensures service continuity and data availability. This process should be as transparent as possible to end users and should not affect project operations.

[0045] Currently, if the logical log transmission of large transactions is interrupted or fails, the standby database may lose some logical logs, thus compromising data consistency. Therefore, for handling large-scale transactions, it is necessary to consider optimizing the semi-synchronous replication mechanism to reduce the impact of large transactions on overall system performance while maintaining high data availability and consistency. Semi-synchronous replication requires that before a transaction's logical log is committed on the primary database, it must wait for the logical log to be transmitted to the standby database and confirm that the standby database has successfully received the logical log before the logical log can be committed on the primary database.

[0046] However, semi-synchronous replication requires that the logical log of a transaction must wait for it to be transmitted to the standby database before being committed on the primary database. Only after confirming that the standby database has successfully received the logical log can the logical log be committed on the primary database. This reduces the risk of data loss and ensures data reliability. Figure 1 illustrates a flowchart of a log backup method.

[0047] In a Database Management System (DBMS), Transaction 1, executed on the primary database, is a large transaction that requires its logical log (.000001) to be transferred from the primary to the standby database. During this transfer, because there is only one transmission channel between the primary and standby databases, other transactions (Transaction 2 and Transaction 3) must wait for Transaction 1's logical log to be transferred and committed before they can use this channel to complete their log backups. Each transaction includes multiple data operation instructions, and as an indivisible unit of work, transactions require semi-synchronous replication of their logical logs. When the number of transactions is large, the logical logs become numerous. Because other transactions' logical logs can only be transferred to the standby database for backup after the logical log of Transaction 1 has been transferred, the entire instance becomes unwritable and blocked. If the log backup process of Transaction 1 takes too long, in order to maintain the performance and responsiveness of the database system, the primary database will degrade to asynchronous replication and immediately commit the logical log of Transaction 1 before the backup is completed. This continues until the backup progress of the logical log catches up with the generation progress of the logical log. During this period, the data reliability of the primary database instance will be greatly reduced, and if an HA switchover operation occurs, data loss may occur.

[0048] In this situation, not only will the commit of the logical logs of subsequent transactions be delayed, but it may also cause a decline in the performance of the primary database. In extreme cases, it may even cause the primary database to slow down its response or be unable to process new requests in a timely manner, resulting in data loss and data inconsistency between the primary and standby databases, thus reducing the availability and stability of the database system.

[0049] To address the aforementioned issues, this disclosure provides a log backup method. This disclosure also relates to a log backup device, a database system, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0050] Referring to Figure 2, which shows a flowchart of a log backup method provided in an embodiment of the present disclosure, the method is applied to a database system and includes the following specific steps.

[0051] Step 202: Obtain the logical log of the first data operation instruction in the target transaction that triggered the generation. The target transaction includes multiple data operation instructions, and the first data operation instruction is any one of the multiple data operation instructions. The logical log is generated when the first data operation instruction completes the modification operation on the primary database data.

[0052] A database system is a software system that stores and manages data within a database, typically running on one or more servers. Database systems not only provide persistent storage for data but also support efficient access, updating, querying, and management of data. A database system includes a primary database and a secondary database, performing functions such as data storage, replication, synchronization, backup, and recovery to ensure high availability, consistency, and durability. Examples include MySQL, Oracle, and PostgreSQL. Database systems can be applied to various project scenarios, such as online retail, inventory management, social media user data storage, logistics and distribution, intelligent traffic flow analysis, e-commerce, Enterprise Resource Planning (ERP), energy monitoring, production control, document management, smart cities, and environmental monitoring—all big data-driven scenarios.

[0053] The primary database is the database instance in a database system used to modify data and generate logical logs. The secondary database is the database instance in a database system used to receive logical logs from the primary database and perform log backups. The primary and secondary databases can run on different servers, forming a distributed storage system, or they can run on the same server, forming a single-machine multi-instance deployment. Generally, the primary database directly interfaces with users or applications, while the secondary database does not directly interface with users or applications. For example, MySQL can run on a Linux server, using a master-slave or master-master mode, where the primary and secondary databases can be on different physical servers or on the same server through different ports and service instances.

[0054] The master database contains project data, including table structures, indexes, views, and other database objects. Modification operations (such as insert, update, and delete) are performed on the master database using data manipulation commands, ensuring the implementation of project logic and the consistent and complete storage of data. For example, the MySQL master database stores order table data for an e-commerce project, including: 1. `order_id` (INT, primary key): a unique identifier for the order; 2. `customer_id` (INT): a unique identifier associated with the customer; 3. `order_date` (DATETIME): the timestamp of order creation; `status` (VARCHAR): the order status (e.g., pending, shipped, completed); 4. `total_amount` (DECIMAL): the total order amount.

[0055] A transaction is a specific set of data manipulation instructions that are executed as a whole and either all complete or none complete. A transaction is the basic unit of modification operations on the primary database in a database system. The target transaction is the currently executing transaction, which includes multiple data manipulation instructions. After these instructions complete their modification operations on the primary database, a logical log of the data manipulation instructions is generated. This logical log records the modifications to the primary database, ensuring data durability and consistency, and supporting subsequent log backup and recovery operations. Once all data manipulation instructions in the target transaction are successfully executed and the logical log is committed, the logical log is written to persistent storage. For example, when a user places an order, a target transaction is triggered. This transaction contains multiple data manipulation instructions, such as inserting a new order record, updating inventory quantities, and recording payment information. These operations must be completed as an atomic unit to ensure the integrity and consistency of the order table data in the e-commerce project. If any step fails, the entire transaction is rolled back, undoing all changes and ensuring that the state of the order table data for the e-commerce project stored in the primary database remains unchanged. After each data manipulation instruction is executed, the system generates a corresponding logical log, recording the completed modifications.

[0056] Data manipulation commands are specific code-language commands issued by users or applications to the master database to perform operations such as Create, Read, Update, and Delete. Data manipulation commands are the primary way users interact with the database system, used to manage and modify data in the database. For example, SQL statements in MySQL include: 1. INSERT: Adds a new record to a database table. In MySQL, `INSERT INTO orders(customer_id, order_date, status, total_amount) VALUES(12345, NOW(), 'Pending', 299.99)` inserts a new order record into the `orders` table; 2. UPDATE: Modifies the data in an existing record. `UPDATE orders SET status = 'Shipped' WHERE order_id = 12345` updates the order status to "Shipped"; 3. DELETE: Removes a record from a database table. DELETE FROM orders WHERE order_id=12345 AND status='Cancelled'; Deletes a cancelled order. 4. SELECT: Queries and retrieves data from the database. SELECT * FROM customers WHERE city='A'; Retrieves information on all customers from city A.

[0057] The logical log of data manipulation instructions records serialized entries that modify the primary database data using these instructions. This ensures data durability and consistency and supports subsequent log backup and recovery operations. The logical log of data manipulation instructions is generated when a modification operation on the primary database is completed using a data manipulation instruction. Generally, logical logs of multiple data manipulation instructions are arranged into a logical log file as serialized entries. For example, in MySQL, the binary log (binlog) file for SQL statements is named mysql-bin.000001, where mysql-bin.000001 indicates that this is the binary log file numbered 000001. MySQL numbers these files sequentially, and whenever a log file reaches a certain size or meets other conditions, the system creates a new log file and continues recording. This log file contains all the operation instructions (SQL statements) that modify the database, stored in binary format. Each log entry records a specific database operation, including but not limited to: 1. Event type: Identifies whether it is a start, commit, rollback, or specific data change event; 2. Timestamp: Records the time the operation occurred; 3. Transaction identifier: If transactions are enabled, it contains the unique identifier of the transaction; 4. Table information: The names of the databases and tables involved; 5. Row-level changes: For each row change, the values ​​before and after the modification are recorded in detail; 6. SQL statement: In statement-based replication mode, the executed SQL statement is recorded directly; while in row-based replication mode, the specific changes to the affected rows are recorded.

[0058] The first data operation instruction is the most recently completed data operation instruction in the target transaction, and it is executed according to the transaction logic order of the target transaction. The logical log of the first data operation instruction is the logical log record generated when the execution of the first data operation instruction completes the modification operation on the primary database.

[0059] For example, the database of an e-commerce project uses the MySQL database system (database management system) of Relational Database Service (RDS) for storage and management. The MySQL database system runs on a distributed server cluster, and one master database server and multiple slave database servers are configured for MySQL. The primary database instance's transaction queue contains three transactions: Transaction 1, Transaction 2, and Transaction 3. The currently executing target transaction is Transaction 1, which was triggered by a user placing an order. This transaction contains three SQL statements: SQL statement 1: Insert a new order record: "INSERT INTO orders(customer_id, order_date, status, total_amount) VALUES(12345, NOW(), 'Pending', 299.99)"; SQL statement 2: Update inventory information: "UPDATE products SET stock_quantity=stock_quantity-1WHERE product_id=56789;"; SQL statement 3: Record payment information: "INSERT INTO payments(order_id, payment_method, amount_paid, payment_date) VALUES(12345, 'Credit Card', 299.99, NOW());".

[0060] The SQL statement 1 has just been executed, triggering the generation of the entry binlog event sql_1 in the binary log file mysql-bin.000001 for that SQL statement.

[0061] The logical log of the first data operation instruction in the target transaction that was triggered and generated is obtained, which lays the data foundation for subsequent backup and transmission.

[0062] Step 204: Replicate the logical log in real time and send it to the standby database, then return to the step of retrieving the logical log that triggered the generation of the first data operation instruction in the target transaction.

[0063] The logical logs are replicated and sent to the standby database in real time. Specifically, the logical logs are replicated in real time to obtain a copy of the logical logs, and then the copy of the logical logs is sent to the standby database.

[0064] One possible way to obtain a copy of the logical log in real time is through a caching mechanism. Another possible way is through a file system snapshot. Yet another possible way is to use a network streaming data format to obtain a copy of the logical log in real time. No specific method is specified here.

[0065] One possible method for sending logical log copies to the backup database is through a reliable data transmission protocol, such as the TCP protocol which uses a three-way handshake. After sending, a confirmation message from the backup database is required to confirm successful transmission. Another possible method is to send the logical log copies to the backup database asynchronously using a message queue, such as Kafka or RabbitMQ. This increases the system's flexibility and fault tolerance, and is not limited here.

[0066] It should be noted that in steps 202 and 204, the acquisition of logical logs, the copying and sending of logical logs, and the execution of data operation instructions and the triggering of logical log generation are executed in isolation. These two processes can be executed separately by configuring isolated threads.

[0067] For example, through a caching mechanism, the logical log entry binlog event sql_1 is copied to the memory cache to obtain a logical log copy binlog event sql_1.entry. The logical log copy binlog event sql_1.entry is sent to the standby server via the TCP protocol, and after receiving an ACK confirmation message from the standby server, the sending is confirmed to be successful, and the process returns to retrieve the binary log entry binlog event sql_2 for the newly triggered SQL statement 2.

[0068] The logical log is replicated and sent to the standby database in real time. The process of retrieving the logical log that triggered the generation of the first data operation instruction in the target transaction is then executed. By transmitting the logical log in real time, it is pre-transmitted to the standby database during the transaction execution process. This reduces the risk of degradation to asynchronous replication due to timeout and avoids performance fluctuations caused by waiting for the standby database to confirm for a long time. As a result, the stability and response speed of the database system are enhanced, the risk of data asynchrony between the primary and standby databases is reduced, and the integrity and consistency of the data are guaranteed.

[0069] Step 206: When the standby database receives the logical logs of each data operation instruction of the target transaction, commit the logical logs of each data operation instruction.

[0070] The standby database receives the logical logs of each data operation instruction of the target transaction. One possible method is that the primary database receives the receipt confirmation message of the logical logs of each data operation instruction from the standby database. Another possible method is that the primary database receives the receipt confirmation message of the logical logs of the last data operation instruction from the standby database. Yet another possible method is that the standby database stores the logical logs of each data operation instruction that completes the target transaction. This is not limited here.

[0071] It should be noted that only after the logical logs of all data operation instructions for the target transaction have been committed can it be considered that the target transaction has been successfully replicated to the standby database. This ensures that even if the primary database fails, the standby database can take over the service without losing any committed transaction information. Optionally, the standby database may also set checkpoints to check the received logical logs to further enhance data security and durability.

[0072] For example, after the standby server receives the logical log entries binlog event sql_1, binlog event sql_2, and binlog event sql_3 corresponding to SQL statements 1, 2, and 3, it parses these entries one by one and replays the corresponding SQL statements. Once all relevant logical log entries have been successfully processed, the standby server confirms these changes and sends an ACK confirmation message to the primary server, acknowledging that the target transaction has been successfully replicated to the standby server. It then commits the binary log file mysql-bin.000001 for transaction 1 and continues executing transaction 2.

[0073] In this embodiment, by transmitting logical logs in real time, the generated logical logs are pre-transmitted to the standby database during transaction execution, rather than waiting until transaction commit. This ensures a high level of consistency even under network latency or other anomalies between the primary and standby databases, reducing the risk of degradation to asynchronous replication due to timeouts. The transmission of logical logs during transaction processing significantly reduces transmission time during the transaction commit phase, avoiding performance fluctuations caused by long waits for standby database confirmation, thereby enhancing the stability and responsiveness of the database system. Synchronous backup of the logs reduces the risk of data asynchrony between the primary and standby databases, ensuring data integrity and consistency, and improving the availability of the database system.

[0074] In one optional embodiment of this disclosure, after step 202, the following specific steps are further included: caching the logical log in the logical log cache of the primary database; in step 204, the logical log is copied and sent to the standby database in real time, including the following specific steps: when the preset sending conditions are met, the logical log cached in the logical log cache is copied and sent to the standby database in real time; step 206 includes the following specific steps: when the standby database receives the logical logs of each data operation instruction of the target transaction, the logical logs of each data operation instruction are retrieved from the logical log cache and committed.

[0075] A logical log cache is a temporary storage area for uncommitted logical logs. Once committed, the logical logs are written to persistent storage. An example is a binary log cache (binlog cache).

[0076] Preset sending conditions are pre-defined triggers that initiate the real-time replication and transmission of logical logs from the primary database to the standby database. These pre-defined sending conditions can be based on various factors such as time, data volume, or system performance to trigger the logical log backup and transmission process, ensuring efficient and real-time data synchronization. Preset sending conditions may include, but are not limited to, preset periods, preset data volume thresholds for the logical logs, and preset performance metric thresholds for the primary database.

[0077] In one optional embodiment of this disclosure, the preset sending condition is any one of a preset period, a preset data volume threshold for the logical log, and a preset performance index threshold for the master database.

[0078] The preset period is a pre-defined time interval that serves as a timer to trigger logical log backup transmission. Setting a preset period helps to transmit logical logs regularly, ensuring timely log backup transmission. For example, a preset period of once every 5 minutes means that the logical logs will be copied and sent to the backup database in real time every 5 minutes.

[0079] The preset data volume threshold for logical logs is a pre-defined cumulative amount of logical logs. Once this amount is exceeded, the logical logs will be transferred to the backup database. By setting a data volume threshold, batch transfers can be performed when logical logs accumulate to a certain level, optimizing network resource usage and reducing the overhead of frequent small-scale transfers. For example, if the preset data volume threshold is 1MB, then when the data in the logical log cache accumulates to 1MB or more, the system will trigger a backup transfer of the logical logs.

[0080] The primary database's preset performance thresholds are pre-defined limits on its performance metrics, such as CPU utilization, memory usage, and disk I / O latency. Insisting on logical log backup and transmission when these performance metrics reach these thresholds may impact the primary database's normal operation and service quality. The logical log transmission strategy can be adjusted by monitoring these performance metrics to avoid large-scale data transfers when the primary database is under high load. For example, if the primary database's CPU utilization exceeds 80%, the transmission of non-logical logs should be postponed to prioritize the primary database's response speed and service stability.

[0081] The logical logs cached in the logical log cache are replicated in real time. Specifically, the logical logs cached in the logical log cache are replicated in real time to obtain a copy of the logical logs, and then the copy of the logical logs is sent to the standby database.

[0082] One possible way to obtain a copy of the logical log in real time is through a caching mechanism, which copies the logical logs cached in the logical log cache in real time. Another possible way is through a file system snapshot, which copies the logical logs cached in the logical log cache in real time. Yet another possible way is to use a network stream data format to copy the logical logs cached in the logical log cache in real time. No further restrictions are imposed here.

[0083] The optional method for sending a copy of the logical log to the standby database is described in step 204 above, and will not be repeated here.

[0084] For example, the entry binlog event sql_1 in the binary log file mysql-bin.000001 of the SQL statement is cached in the logical log cache binlog cache of the primary database. When the preset period (5 minutes) is reached, the copy of the cached logical log entry in the logical log cache binlog cache is sent to the standby database server via the TCP protocol. After receiving the ACK confirmation message returned by the standby database, the sending is confirmed to be successful. Then, the execution returns to retrieve the binary log entry binlog event sql_2 of the newly triggered SQL statement 2. When the standby database receives the logical logs of each SQL statement of transaction 1, it retrieves the logical logs of each SQL statement from the logical log cache binlog cache, commits the binary log file mysql-bin.000001 of transaction 1, and continues to execute transaction 2.

[0085] In this embodiment of the disclosure, logical log caching and preset sending conditions are used in combination. Logical log caching reduces frequent disk read and write operations, while preset sending conditions ensure that logical logs are transmitted to the backup database in batches at appropriate times. This maintains the timeliness of data synchronization and avoids data transmission interference when the primary database is under high load, thereby improving the system's response speed and service quality, and enhancing data integrity and consistency.

[0086] In one optional embodiment of this disclosure, the target transaction is a transaction being executed in the transaction queue; caching the logical log in the logical log cache of the master database includes the following specific steps: within a preset time window, caching the logical log in the logical log cache of the master database, updating the target transaction to other transactions in the transaction queue, and returning to the step of retrieving the logical log of the first data operation instruction in the target transaction that was triggered and generated.

[0087] The preset time window is a pre-defined execution window used for the logical log cache of the target transaction. By using this preset time window, log backup fragmentation is achieved, preventing the logical log backup transmission of the target transaction from completely blocking the execution of other transactions. However, this method modifies the format of the logical log, which is somewhat intrusive to the database system.

[0088] For example, the preset time window is 1 second. Within each second, the primary database caches the binary logs of all completed first data operation instructions in the currently executing target transaction into the binary log cache and attempts to replicate and send these binary logs to the backup database in real time according to preset sending conditions. If the target transaction is not completed by the end of this 1-second time window, the system will update the target transaction to the next transaction in the transaction queue, even if the previous target transaction has not fully committed all its binary logs: when the primary database is processing transaction 1 triggered by a user order, it will cache and transmit as much of the binary logs generated by that transaction as possible within each 1-second time window. Once 1 second has passed, even if transaction 1 is not yet completed, the system will begin executing the next transaction in the transaction queue: transaction 2, allowing the binary logs of transaction 2 to also begin to be cached and backed up for transmission.

[0089] In this embodiment of the disclosure, by performing operations in segments, caching, and backup transmission, the system avoids occupying resources for a long time on a single transaction, ensuring the high efficiency of parallel processing of multiple transactions and improving the system's response speed.

[0090] In one optional embodiment of this disclosure, after the logical log is copied and sent to the standby database in real time in step 204, the following specific step is further included: caching the logical log in the relay log cache of the standby database.

[0091] The relay log is a copy of the logical log transmitted from the primary database, stored on the standby database. Relay logs can be replayed to synchronize data manipulation commands that modify the primary database. The standby database updates its data by reading and applying events from the relay log, thus maintaining consistency with the primary database and ensuring that even if the primary database fails, the standby database can take over the service and provide the latest data.

[0092] The relay log cache is a temporary storage area for relay logs until they are executed and persisted. It allows the standby database to accumulate a certain amount of log data in memory before writing it to persistent storage in batches. To speed up the replication process, the standby database may first store the received binary log content in the relay log cache. This reduces frequent small-scale disk writes, as writing directly to disk is typically slower than writing to memory. When the cache reaches a certain size or after a certain time interval, the log data in the cache is flushed to disk, completing the persistent storage. The relay log cache effectively improves replication efficiency and reduces disk I / O.

[0093] For example, after receiving the data, the standby database stores it in the relay log cache. Then, based on the relay log entry in the relay log cache, the standby database re-executes SQL statement 1: "INSERT INTO orders(customer_id, order_date, status, total_amount) VALUES(12345, NOW(), 'Pending Processing', 299.99)", inserting a new order record to ensure that the standby database data is consistent with the primary database data.

[0094] In this embodiment of the disclosure, frequent disk read and write operations are effectively reduced. By accumulating relay logs in the cache, replication efficiency is improved and the frequency of disk writes is reduced. This allows the standby database to process logical logs in batches, optimizes resource utilization, and speeds up the response of the standby database.

[0095] In one optional embodiment of this disclosure, after caching the logical logs in the relay log cache of the standby database, the following specific steps are further included: if the relay log cache contains the logical logs of each data operation instruction of the target transaction, the logical logs of each data operation instruction are retrieved from the relay log cache, and a relay log file is generated based on the logical logs of each data operation instruction.

[0096] The relay log file is a log file for the relay log that is persistently stored on the standby database. It stores a copy of the logical logs received from the primary database and is used to replay these log events on the standby database to synchronize data changes on the primary database. When the logical logs of each data operation instruction for the target transaction are cached in the relay log cache, the relay logs are flushed to the relay log file on disk. This ensures that even if the system crashes, the incomplete log application process can be recovered using the relay log file. Generally, relay log files are renamed sequentially.

[0097] For example, if the relay log cache contains the binary logs of the three SQL statements of transaction 1, the binary logs of the three SQL statements are retrieved from the relay log cache. Based on the binary logs of each SQL statement, a relay log file is generated, thereby quickly completing the backup transmission.

[0098] In this embodiment of the disclosure, by temporarily storing the logical log in the relay log cache, and caching the logical log of each data operation instruction of the target transaction in the relay log cache, the relay log file is persistently stored, which effectively reduces frequent disk read and write operations, improves the processing efficiency and response speed of the backup database, and ensures that even if the system fails, the incomplete log application can be recovered from the relay log file, thus ensuring data consistency and durability and enhancing the stability and availability of the database system.

[0099] In one optional embodiment of this disclosure, before generating a relay log file based on the logical logs of each data operation instruction, the following specific steps are further included: obtaining the logical logs of the related transactions that triggered the generation; copying and sending the logical logs of the related transactions to the backup database in real time; generating a relay log file based on the logical logs of each data operation instruction, including the following specific steps: integrating the logical logs of the related transactions and the logical logs of each data operation instruction in the target transaction to generate a relay log file.

[0100] Related transactions are other transactions that are related to the target transaction, generally other transactions within the same batch. During database replication and backup, to ensure data consistency and integrity, the logical logs of one or more related transactions are usually processed together with the logical logs of each data operation instruction in the target transaction. This ensures that the logical logs of these transactions are backed up to the standby database as a whole. For example, in an e-commerce project, there are two transactions that occur almost simultaneously: Transaction A: A user places an order for a product. Transaction B: An administrator adjusts the inventory quantity of that product. Although the two transactions are triggered by different users, they are operations on the same product and may affect the same database records. Therefore, they are considered "related transactions" and need to be processed in the same batch. When performing log backup, it should be ensured that the logs of these two transactions are processed together to maintain data consistency. Related transactions and the target transaction can be distinguished based on at least one of the following: the number of data operation instructions, the number of logical log entries, timestamps, transaction identifiers, the amount of data operated on in the primary database, and row-level changes.

[0101] The logical log of a related transaction consists of serialized entries recording changes made to the primary database. It contains information on all changes made to the database during the execution of the related transaction, including but not limited to changes to table structures, indexes, views, and other database objects. The logical log of a related transaction is generated when the entire related transaction has completed its modification operations on the primary database.

[0102] The logical logs of relevant transactions are replicated and sent to the standby database in real time. Specifically, the logical logs of relevant transactions are replicated in real time to obtain a copy of the logical logs of relevant transactions, and then the copy of the logical logs of relevant transactions is sent to the standby database.

[0103] One possible method for obtaining a copy of the logical log of a relevant transaction in real time is through a caching mechanism. Another possible method is through a file system snapshot. Yet another possible method is to use a network streaming data format to obtain a copy of the logical log of a relevant transaction in real time. No further restrictions are imposed here.

[0104] One possible method for sending a copy of the logical log of a relevant transaction to the standby database is through a reliable data transmission protocol. For example, the TCP protocol uses a three-way handshake, requiring a confirmation message from the standby database after successful transmission. Another possible method is to use a message queue, such as Kafka or RabbitMQ, to asynchronously send the copy of the logical log to the standby database. This increases the system's flexibility and fault tolerance, and is not limited here.

[0105] It should be noted that the processes of obtaining the logical logs of relevant transactions, copying and sending the logical logs of relevant transactions, and executing data operation instructions and triggering the generation of logical logs of relevant transactions are executed in isolation. They can be executed separately by configuring isolated threads.

[0106] For example, after the primary database completes the execution of SQL statement 1 in transaction A, it triggers the generation of the entry `binlog event sql_1` in the binary log file `mysql-bin.000001` for that SQL statement. `binlog event sql_1` is retrieved and replicated to the standby database in real time, sent to the standby server via TCP, and its successful transmission is confirmed upon receiving an ACK confirmation message from the standby database. For related transaction B, which contains 10 SQL statements updating inventory information, the execution of these 10 SQL statements also triggers the generation of the corresponding binary log file `mysql-bin.000002`. This file is replicated and then sent to the standby database via a reliable data transmission protocol (e.g., TCP). The binary log files of transactions A and B are integrated to obtain a relay log file.

[0107] In this embodiment, by processing the logical logs of the target transaction and related transactions together, frequent disk read / write operations are effectively reduced, improving the processing efficiency and response speed of the standby database. Simultaneously, it ensures that even in the event of a system failure, incomplete log applications can be recovered from the relay log file, guaranteeing data consistency and durability, and enhancing the stability and availability of the database system. Furthermore, this method ensures that even in complex project scenarios, different but related transactions are correctly reflected in the standby database, avoiding potential data inconsistency issues.

[0108] In one optional embodiment of this disclosure, step 206 includes the following specific steps: when the primary database receives a confirmation message from the backup database regarding the logical log of the last data operation instruction, the logical log of each data operation instruction is submitted.

[0109] The last data operation instruction is the last data operation instruction executed in the target transaction. This final command can only be executed if all previous operations have succeeded; otherwise, the entire transaction will be rolled back.

[0110] The logical log of the last data operation instruction records serialized entries of changes made to the primary database data by that last data operation instruction. This ensures data durability and consistency and supports subsequent log backup and recovery operations. The logical log of the last data operation instruction is generated when the primary database data is modified by executing that last data operation instruction.

[0111] A confirmation message is a signal or notification sent from the standby database to the primary database, indicating that the standby database has successfully received the logical log of the last data operation instruction. This confirmation mechanism is used in semi-synchronous replication environments to ensure that the primary database only commits transactions after confirming that the standby database has safely received the logical log. It prevents situations where the primary database commits before the standby database has received the logical log of the last data operation instruction, and the primary database then has to search for the logical log of the last data operation instruction in the already committed logical log file to retransmit it. This helps reduce the risk of data loss and improves system reliability and consistency.

[0112] For example, the SQL statement "record payment information" is the last data operation instruction in the transaction. Only when the primary database server receives the ACK confirmation message from the binary log of the last data operation instruction sent by the standby database server can it be determined that transaction 1 has been successfully processed by both database instances, thus completing the entire transaction commit process.

[0113] In this embodiment, by ensuring that the primary database only commits data after receiving a confirmation message from the backup database's logical log for the last data operation instruction, the risk of data inconsistency is effectively avoided. This approach not only reduces the possibility of data loss and enhances system reliability and consistency, but also prevents unnecessary retransmission operations, improves replication efficiency and system response speed, thereby enhancing the overall performance and stability of the database system.

[0114] Corresponding to the above-described method embodiments, Figure 3 shows a flowchart of a log backup method provided in one embodiment of this disclosure. As shown in Figure 3: On the master database, a task manager and a logical log cache for real-time backup of global logical logs are set up. The task manager is used to manage and schedule real-time backup tasks of global logical logs. A list of real-time backups of global logical logs is set up in the task manager to record all logical log entries that need to be backed up and their status. The logical log cache is used to cache the logical logs generated during the backup process. User threads and backup threads run on the master database.

[0115] A relay log cache is configured on the standby database to cache received logical logs. Read / write threads run on the standby database.

[0116] The user thread is executed to modify the master database data, triggering the generation of logical logs. After all logical log backups and transmissions are completed, the logical log files are committed to achieve persistent storage.

[0117] The backup thread is executed to send the real-time backup entries of the logical logs of each data operation instruction in the target transaction and the real-time backup entries of the logical logs of related transactions to the standby database.

[0118] The execution read / write thread caches the real-time backup entries of the logical logs of each data operation instruction in the target transaction and the real-time backup entries of the logical logs of related transactions through the relay log cache. The real-time backup entries of the logical logs of related transactions are directly added to the relay log file. If the real-time backup entries of the logical logs of each data operation instruction in the target transaction are cached in the relay log cache, the real-time backup entries of the logical logs of each data operation instruction are retrieved from the relay log cache, renamed, and added to the relay log file.

[0119] The log backup method provided in this disclosure will be further described below with reference to Figure 4, taking the application of the log backup method in a MySQL database as an example. Figure 4 shows a flowchart of the processing procedure of a log backup method applied to a MySQL database according to an embodiment of this disclosure. This method is applied to an RDS MySQL database and includes the following specific steps.

[0120] Step 402: Obtain the binary log (binlog) of the SQL statements in the ETL (Extract, Transform, Load) transaction that triggered the generation. The ETL transaction includes multiple SQL statements, and the binary log (binlog) is generated when the SQL statements are executed and modify the data in the master database.

[0121] Step 404: Add the binary log cache (binlog cache) to the primary database's binary log cache (binlog cache).

[0122] Step 406: When the preset period is reached, the binary log binlog cached in the binary log cache is copied in real time and sent to the backup database.

[0123] Step 408: Move the binary log cache (binlog cache) to the relay log cache of the standby database, and return to step 402.

[0124] Step 410: If the relay log cache contains the binary logs (binlogs) of each SQL statement for ETL transactions, retrieve the binary logs (binlogs) of each SQL statement from the relay log cache, generate a relay log file based on the binary logs (binlogs) of each SQL statement, and send a receipt confirmation message to the master database.

[0125] Step 412: When the primary database receives the confirmation message for receiving the binary logs (binlog) of each SQL statement from the standby database, it retrieves the binary logs (binlog) of each SQL statement from the binary log cache and commits them.

[0126] In this embodiment, the binary log (binlog) of large transactions is pre-transmitted to the standby database during the transaction execution process through real-time transmission. When the large transaction is committed, only further file conversion is required on the standby database, which greatly reduces the time spent transmitting the binary log (binlog) during the commit phase of large transactions. Semi-synchronous replication does not degrade and does not cause the instance to become unwritable, effectively avoiding timeout rollback and performance jitter of semi-synchronous replication. Moreover, it does not require changing the format of the binary log (binlog) file and does not intrude on the MySQL database ecosystem.

[0127] Corresponding to the above method embodiments, this disclosure also provides a log backup device embodiment. Figure 5 shows a schematic diagram of the structure of a log backup device provided in one embodiment of this disclosure. As shown in Figure 5, the device is applied to a database system and includes: an acquisition module 502, configured to acquire a logical log of a first data operation instruction in a target transaction that is triggered and generated, wherein the target transaction includes multiple data operation instructions, the first data operation instruction is any one of the multiple data operation instructions, and the logical log is triggered when the first data operation instruction completes a modification operation on the primary database data; a backup module 504, configured to copy the logical log in real time and send it to a standby database, and return to the step of acquiring the logical log of the first data operation instruction in the target transaction that is triggered and generated; and a submission module 506, configured to submit the logical log of each data operation instruction when the standby database receives the logical log of each data operation instruction of the target transaction.

[0128] Optionally, the device further includes: a first caching module configured to cache logical logs in the logical log cache of the primary database; a backup module 504 further configured to: copy and send the logical logs cached in the logical log cache to the standby database in real time when a preset sending condition is met; and a commit module 506 further configured to: retrieve the logical logs of each data operation instruction from the logical log cache and commit them when the standby database receives the logical logs of each data operation instruction of the target transaction.

[0129] Optionally, the preset sending condition is any one of the following: preset period, preset data volume threshold of logical log, and preset performance index threshold of master database.

[0130] Optionally, the target transaction is a transaction currently being executed in the transaction queue; the first cache module is further configured to: cache the logical log in the master database's logical log cache within a preset time window, update the target transaction to other transactions in the transaction queue, and return to the step of retrieving the logical log of the first data operation instruction in the target transaction that was triggered and generated.

[0131] Optionally, the device further includes a second caching module configured to cache logical logs in a relay log cache of the standby database.

[0132] Optionally, the device further includes a file generation module, configured to, when the logical logs of each data operation instruction of the target transaction are cached in the relay log cache, retrieve the logical logs of each data operation instruction from the relay log cache, and generate a relay log file based on the logical logs of each data operation instruction.

[0133] Optionally, the device further includes: a related backup module, configured to acquire the logical logs of the related transactions that were triggered and generated; to copy the logical logs of the related transactions in real time and send them to the backup database; correspondingly, a file generation module is further configured to: integrate the logical logs of the related transactions and the logical logs of each data operation instruction in the target transaction to generate a relay log file.

[0134] Optionally, the submission module 506 is further configured to submit the logical logs of each data operation instruction when the primary database receives a confirmation message from the standby database regarding the logical log of the last data operation instruction.

[0135] In this embodiment, by transmitting logical logs in real time, the generated logical logs are pre-transmitted to the standby database during transaction execution, rather than waiting until transaction commit. This ensures a high level of consistency even under network latency or other anomalies between the primary and standby databases, reducing the risk of degradation to asynchronous replication due to timeouts. The transmission of logical logs during transaction processing significantly reduces transmission time during the transaction commit phase, avoiding performance fluctuations caused by long waits for standby database confirmation, thereby enhancing the stability and responsiveness of the database system. Synchronous backup of the logs reduces the risk of data asynchrony between the primary and standby databases, ensuring data integrity and consistency, and improving the availability of the database system.

[0136] The above is an illustrative scheme of a log backup device according to this embodiment. It should be noted that the technical solution of this log backup device and the technical solution of the log backup method described above belong to the same concept. For details not described in detail in the technical solution of the log backup device, please refer to the description of the technical solution of the log backup method described above.

[0137] Corresponding to the above method embodiments, this disclosure also provides a database system embodiment. Figure 6 shows a schematic diagram of the structure of a database system provided in one embodiment of this disclosure. As shown in Figure 6, the database system 600 includes a primary database 602 and a standby database 604. The primary database 602 is used to obtain the logical log of the first data operation instruction in the target transaction that was triggered and generated, replicate the logical log in real time and send it to the standby database 604, and return to execute the step of obtaining the logical log of the first data operation instruction in the target transaction that was triggered and generated. The target transaction includes multiple data operation instructions, and the first data operation instruction is any one of the multiple data operation instructions. The logical log is generated when the first data operation instruction completes the modification operation on the data in the primary database 602. The standby database 604 is used to receive the logical log sent by the primary database 602. When the logical log of the last data operation instruction is received, it generates and sends a receipt confirmation message of the logical log of the last data operation instruction to the primary database 602. The primary database 602 is also used to commit the logical logs of each data operation instruction of the target transaction when it receives the receipt confirmation message of the logical log of the last data operation instruction from the standby database 604.

[0138] Optionally, the primary database 602 is also used to cache logical logs in the logical log cache of the primary database 602. When the preset sending conditions are met, the logical logs cached in the logical log cache are copied in real time and sent to the standby database 604. When the standby database 604 receives the reception confirmation message of the logical log of the last data operation instruction, the logical logs of each data operation instruction are retrieved from the logical log cache and committed.

[0139] Optionally, the standby database 604 is also used to cache logical logs in the relay log cache of the standby database 604. Upon receiving the confirmation message of the logical log of the last data operation instruction from the standby database 604, the standby database 604 retrieves the logical logs of each data operation instruction from the relay log cache and generates a relay log file based on the logical logs of each data operation instruction.

[0140] In this embodiment of the disclosure, the database system utilizes real-time transmission of logical logs. The generated logical logs are pre-transmitted to the standby database during transaction execution, rather than waiting until transaction commit. This ensures a high level of consistency even under network latency or other anomalies between the primary and standby databases, reducing the risk of degradation to asynchronous replication due to timeouts. The transmission of logical logs during transaction processing significantly reduces transmission time during the transaction commit phase, avoiding performance fluctuations caused by long waits for standby database confirmation, thereby enhancing the stability and responsiveness of the database system. Synchronous backup of the logs reduces the risk of data inconsistency between the primary and standby databases, ensuring data integrity and consistency, and improving the availability of the database system.

[0141] The above is an illustrative scheme of a database system according to this embodiment. It should be noted that the technical solution of this database system and the technical solution of the log backup method described above belong to the same concept. For details not described in detail in the technical solution of the database system, please refer to the description of the technical solution of the log backup method described above.

[0142] Figure 7 shows a structural block diagram of a computing device according to an embodiment of the present disclosure. The components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and a database 750 is used to store data.

[0143] The computing device 700 also includes an access device 740, which enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 740 may include one or more of any type of wired or wireless network interface (e.g., Network Interface Controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or Near Field Communication (NFC).

[0144] In one embodiment of this disclosure, the aforementioned components of the computing device 700, as well as other components not shown in FIG. 7, may be interconnected, for example, via a bus. It should be understood that the computing device block diagram shown in FIG. 7 is merely for illustrative purposes and is not intended to limit the scope of this disclosure. Those skilled in the art can add or replace other components as needed.

[0145] The computing device 700 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 700 can also be a mobile or stationary server.

[0146] The processor 720 is used to execute the following computer program / instruction, which, when executed by the processor, implements the steps of the above-described log backup method.

[0147] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the log backup method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the log backup method described above.

[0148] An embodiment of this disclosure also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described log backup method.

[0149] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the log backup method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the log backup method described above.

[0150] An embodiment of this disclosure also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described log backup method.

[0151] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the log backup method described above belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the log backup method described above.

[0152] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0153] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0154] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this disclosure are not limited to the described order of actions, because according to the embodiments of this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this disclosure.

[0155] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0156] The preferred embodiments disclosed above are merely illustrative of this disclosure. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments of this disclosure. These embodiments are selected and specifically described in this disclosure to better explain the principles and practical applications of the embodiments of this disclosure, thereby enabling those skilled in the art to better understand and utilize this disclosure. This disclosure is limited only by the claims and their full scope and equivalents.

Claims

1. A log backup method, applied to a database system, comprising: Obtain the logical log of the first data operation instruction in the target transaction that was triggered and generated, wherein the target transaction includes multiple data operation instructions, the first data operation instruction is any one of the multiple data operation instructions, and the logical log is generated when the first data operation instruction is executed to modify the data in the master database. The logical log is copied and sent to the backup database in real time, and the process returns to the step of obtaining the logical log of the first data operation instruction in the target transaction that was triggered and generated. When the backup database receives the logical logs of each data operation instruction for the target transaction, it commits the logical logs of each data operation instruction.

2. The method according to claim 1, after obtaining the logical log of the first data operation instruction in the target transaction, further includes: The logical logs are cached in the logical log cache of the main database; The step of replicating and sending the logical log to the backup database in real time includes: When the preset sending conditions are met, the logical logs cached in the logical log cache are copied in real time and sent to the backup database. When the standby database receives the logical logs of each data operation instruction for the target transaction, the logical logs of each data operation instruction are committed, including: When the backup database receives the logical logs of each data operation instruction for the target transaction, it retrieves the logical logs of each data operation instruction from the logical log cache and commits them.

3. The method according to claim 2, wherein the preset sending condition is any one of a preset period, a preset data volume threshold for the logical log, and a preset performance index threshold for the master database.

4. The method according to claim 2, wherein the target transaction is a transaction being executed in the transaction queue; The step of caching the logical logs in the logical log cache of the main database includes: Within a preset time window, the logical log is cached in the logical log cache of the master database, the target transaction is updated to other transactions in the transaction queue, and the process returns to the step of obtaining the logical log of the first data operation instruction in the target transaction that was triggered and generated.

5. The method according to any one of claims 1-4, further comprising, after the real-time replication and transmission of the logical log to the backup database: The logical logs are cached in the relay log cache of the backup database.

6. The method according to claim 5, further comprising, after the step of caching the logical log in the relay log cache of the backup database: If the relay log cache contains logical logs of each data operation instruction of the target transaction, the logical logs of each data operation instruction are retrieved from the relay log cache, and a relay log file is generated based on the logical logs of each data operation instruction.

7. The method according to claim 6, further comprising, before generating the relay log file based on the logical log of each data operation instruction: Retrieve the logical logs of the related transactions that triggered the generation; The logical logs of the relevant transactions are copied and sent to the backup database in real time. The generation of relay log files based on the logical logs of the various data operation instructions includes: Integrate the logical logs of the relevant transactions and the logical logs of the data operation instructions in the target transaction to generate a relay log file.

8. The method according to claim 1, wherein when the standby database receives the logical logs of each data operation instruction of the target transaction, the logical logs of each data operation instruction are committed, comprising: Upon receiving a confirmation message from the backup database regarding the logical log of the last data operation instruction, the primary database submits the logical logs of each data operation instruction.

9. A log backup device, applied to a database system, comprising: The acquisition module is configured to acquire the logical log of the first data operation instruction in the target transaction that is triggered and generated. The target transaction includes multiple data operation instructions, and the first data operation instruction is any one of the multiple data operation instructions. The logical log is generated when the first data operation instruction is executed to modify the data in the master database. The backup module is configured to copy the logical log in real time and send it to the backup database, and return to the step of obtaining the logical log of the first data operation instruction in the target transaction that was triggered and generated; The commit module is configured to commit the logical logs of each data operation instruction when the standby database receives the logical logs of each data operation instruction for the target transaction.

10. A database system comprising a primary database and a standby database; The primary database is used to obtain the logical log of the first data operation instruction in the target transaction that triggered the generation, replicate the logical log in real time and send it to the backup database, and return to execute the step of obtaining the logical log of the first data operation instruction in the target transaction that triggered the generation, wherein... The target transaction includes multiple data operation instructions, the first data operation instruction is any one of the multiple data operation instructions, and the logical log is generated when the first data operation instruction is executed to modify the data in the master database. The backup database is used to receive the logical logs sent by the primary database, and upon receiving the logical log of the last data operation instruction, generates and sends a receipt confirmation message of the logical log of the last data operation instruction to the primary database. The primary database is also used to commit the logical logs of each data operation instruction of the target transaction upon receiving a confirmation message from the backup database regarding the logical log of the last data operation instruction.

11. The database system according to claim 10, wherein the primary database is further configured to cache the logical logs in the logical log cache of the primary database, and, when a preset sending condition is met, to copy and send the cached logical logs in the logical log cache to the backup database in real time, and, upon receiving a confirmation message from the backup database regarding the logical logs of the last data operation instruction, to retrieve the logical logs of each data operation instruction from the logical log cache and submit them.

12. The database system according to claim 10 or 11, wherein the standby database is further configured to cache the logical log in the relay log cache of the standby database, and upon receiving a confirmation message from the standby database regarding the logical log of the last data operation instruction, retrieve the logical log of each data operation instruction from the relay log cache, and generate a relay log file based on the logical log of each data operation instruction.

13. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which perform the following operations when executed by the processor: Obtain the logical log of the first data operation instruction in the target transaction that was triggered and generated, wherein the target transaction includes multiple data operation instructions, the first data operation instruction is any one of the multiple data operation instructions, and the logical log is generated when the first data operation instruction is executed to modify the data in the master database. The logical log is copied and sent to the backup database in real time, and the process returns to the step of obtaining the logical log of the first data operation instruction in the target transaction that was triggered and generated. When the backup database receives the logical logs of each data operation instruction for the target transaction, it commits the logical logs of each data operation instruction.

14. The computing device of claim 13, further comprising: The logical logs are cached in the logical log cache of the main database; The step of replicating and sending the logical log to the backup database in real time includes: When the preset sending conditions are met, the logical logs cached in the logical log cache are copied in real time and sent to the backup database. When the standby database receives the logical logs of each data operation instruction for the target transaction, the logical logs of each data operation instruction are committed, including: When the backup database receives the logical logs of each data operation instruction for the target transaction, it retrieves the logical logs of each data operation instruction from the logical log cache and commits them.

15. The computing device according to claim 14, wherein the preset sending condition is any one of a preset period, a preset data volume threshold for the logical log, and a preset performance index threshold for the main database.

16. The computing device of claim 14, wherein the target transaction is a transaction being executed in a transaction queue; The step of caching the logical logs in the logical log cache of the main database includes: Within a preset time window, the logical log is cached in the logical log cache of the master database, the target transaction is updated to other transactions in the transaction queue, and the process returns to the step of obtaining the logical log of the first data operation instruction in the target transaction that was triggered and generated.

17. The computing device according to any one of claims 13-16, wherein the operation further comprises: The logical logs are cached in the relay log cache of the backup database.

18. The computing device of claim 17, further comprising: If the relay log cache contains logical logs of each data operation instruction of the target transaction, the logical logs of each data operation instruction are retrieved from the relay log cache, and a relay log file is generated based on the logical logs of each data operation instruction.

19. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.

20. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.

Images