A database log replay method and related products

Abstract

This invention provides a database log replay method and related products. The database log replay method includes: monitoring the write-ahead log data generation status of each node in the master database cluster; determining that the target node has entered a log idle state in response to the detection that no new write-ahead log data has been generated by the target node within a preset monitoring period; triggering the target node to generate auxiliary write-ahead log data based on the log idle state; and synchronizing the auxiliary write-ahead log data to the slave database via the target node's streaming replication link. This invention, through the master database node's proactive awareness of its own write-ahead log data generation status, generates auxiliary write-ahead log data and sends it to the slave database, ensuring that the slave database's log buffer structure is continuously filled. This fundamentally solves the technical problem of the slave database being unable to complete the selection of write-ahead log data to be replayed and the replay process stalling due to a single node being idle for a long time.

Description

Technical Field

[0001] This invention relates to database technology, and in particular to a database log playback method and related products. Background Technology

[0002] In a master-slave database cluster, multiple nodes in the master database cluster process transactions in parallel and generate write-ahead log data independently, which is then sent to the slave databases via streaming replication. The slave databases replay the received write-ahead log data in a globally unified order. To ensure the replay order, the slave databases allocate a corresponding log buffer structure for each active master database node to store the write-ahead log data to be replayed by that node. If any master database node fails to generate new write-ahead log data for an extended period, the log replay process in the slave databases stalls, potentially blocking transaction processing on the nodes of the master database cluster. Summary of the Invention

[0003] One objective of this invention is to improve the continuity of streaming replication in database master-slave clusters when nodes are idle.

[0004] A further objective of this invention is to eliminate the defect that the slave database log replay process stalls due to the master database node being idle, and to avoid the master database node transaction processing being blocked.

[0005] Specifically, the present invention provides a database log replay method, comprising: Monitor the write-ahead log data generation status of each node in the main database cluster; In response to the detection that the target node among the nodes has not generated new write-ahead log data within the preset monitoring period, it is determined that the target node enters the log idle state. Based on the log idle state, the target node is triggered to generate auxiliary write-ahead log data; The auxiliary write-ahead log data is synchronized to the slave database via the stream replication link of the target node, for the slave database to perform log replay.

[0006] Optionally, the step of determining that the target node has entered the log idle state includes: If it is detected that the target node does not generate new write-ahead log data during the preparation period in the monitoring cycle, the target node is determined to enter a temporary log idle state. In the temporary log idle state, continue to monitor whether the target node generates new write-ahead log data during the idle confirmation period after the preparation period in the monitoring cycle, wherein the sum of the preparation period and the idle confirmation period is equal to the preset monitoring cycle; If the target node fails to generate new write-ahead log data during the idle confirmation period, the target node is determined to transition from the temporary log idle state to the log idle state. In response to the target node generating new write-ahead log data during the idle confirmation period, the temporary log idle state is exited and the write-ahead log data active state is restored.

[0007] Optionally, after determining that the target node has entered the log idle state, the method further includes: Initiate heartbeat detection to the target node; If a valid confirmation response based on the heartbeat detection is received from the target node within the preset response time, the determination of the log idle state is revoked. If not, maintain the determination of the log's idle state.

[0008] Optionally, the step of triggering the target node to generate auxiliary write-ahead log data includes: Send a generation instruction to the target node, wherein the generation instruction carries a global sequence identifier for sorting, and the global sequence identifier is greater than the sequence number of any committed write-ahead log in the main database cluster; Based on the generated instructions, a request response is received from the target node; After verifying the validity of the request response, the target node is controlled to generate auxiliary write-ahead log data according to the generation instruction.

[0009] Optionally, the step of synchronizing the auxiliary write-ahead log data to the slave database via the streaming replication link of the target node includes: Write the auxiliary write-ahead log data into the write-ahead log buffer corresponding to the target node; In response to the completion of the writing of the auxiliary write-ahead log data, the streaming replication and sending process of the target node is awakened; The streaming replication sending process reads the auxiliary write-ahead log data from the write-ahead log buffer and pushes it to the slave database in the form of a continuous data stream through the established streaming replication link; The auxiliary write-ahead log data is used to enable the database to perform the following operations: The auxiliary write-ahead log data is written into the leaf node corresponding to the target node in the paired heap data structure. The paired heap data structure is used to maintain the replay order of write-ahead log data from multiple master database nodes. The root node of the paired heap data structure stores the write-ahead log data with the smallest global sequence identifier among all leaf nodes as the next write-ahead log data to be replayed. In response to the auxiliary write-ahead log data being written to the leaf node, the paired heap data structure is triggered to perform a reordering operation to update the root node.

[0010] In particular, the present invention provides another method for replaying database logs, including: Monitor the status of data received from the database's write-ahead logs; In response to receiving auxiliary write-ahead log data, the auxiliary write-ahead log data is inserted into the replay log sequence maintained from the database; Based on the updated log sequence to be replayed, continue to perform the log replay operation.

[0011] Optionally, the step of inserting the auxiliary write-ahead log data into the sequence of logs to be replayed maintained from the database includes: Identify the global sequence identifier carried by the auxiliary write-ahead log data; The global sequence identifier is compared with the start sort identifier and end sort identifier of the log sequence to be replayed; If the global sequence identifier is greater than the end sorting identifier, the auxiliary write-ahead log data is appended to the end of the log sequence to be replayed; If the global sequence identifier is less than the starting sorting identifier, it is determined that the auxiliary write-ahead log data has a header out-of-bounds exception, and a retransmission request is sent to the target node. If the global sequence identifier is between the start sort identifier and the end sort identifier, the auxiliary write-ahead log data is inserted at the insertion position that matches the global sequence identifier in the log sequence to be replayed.

[0012] According to another aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the database log replay method described above.

[0013] According to another aspect of the present invention, a computer program product is also provided, comprising a computer program that, when executed by a processor, implements the steps of the database log playback method described above.

[0014] According to another aspect of the present invention, a computer device is also provided, which includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any database log playback method.

[0015] The database log replay method of this invention, through the proactive perception of the master database node's own write-ahead log data generation status, generates auxiliary write-ahead log data during the node's idle log state and sends it to the slave database. This ensures that the slave database's log buffer structure is continuously filled, fundamentally solving the technical problem of the slave database being unable to complete the selection of write-ahead log data to be replayed and the replay process stalling due to a single node being idle for a long time. By maintaining the continuity of the slave database replay process, the impact of the slave database being stuck on the master database transaction commit is eliminated, and all nodes of the master database are prevented from being blocked while waiting for replay confirmation from the slave database, significantly improving the availability and fault recovery capability of the master-slave database cluster.

[0016] Furthermore, the database log replay method of the present invention introduces a temporary log idle state as a buffer judgment area, dividing the preset monitoring period into a preparation period and an idle confirmation period. When the target node is only temporarily idle due to transient reasons such as transaction gaps or load fluctuations, it only enters the temporary log idle state instead of immediately generating auxiliary pre-written log data; only when no pre-written log data is generated in two consecutive periods is the target node confirmed to have entered the log idle state and the auxiliary pre-written log data generation operation is triggered, thereby eliminating the log storm problem caused by microsecond-level or millisecond-level business idle periods.

[0017] Furthermore, the log replay method of the present invention continuously fills the log buffer structure corresponding to the target node of the main database by receiving and inserting auxiliary write-ahead log data from the database. The log sequence to be replayed from the slave database remains complete at all times, and all active main database nodes have corresponding write-ahead log data to be replayed for sorting, thereby completely eliminating the problem of permanent freeze in the replay process caused by missing logs from a single node.

[0018] The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0019] The following sections will describe some specific embodiments of the invention in detail by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or portions. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings: Figure 1 This is a schematic flowchart of a database log playback method according to an embodiment of the present invention; Figure 2 This is a schematic flowchart illustrating the process of a target node entering a log idle state in a database log replay method according to an embodiment of the present invention. Figure 3This is a schematic flowchart illustrating the determination of log idle status in a database log replay method according to an embodiment of the present invention; Figure 4 This is a schematic flowchart illustrating the generation of auxiliary write-ahead log data by a target node in a database log replay method according to an embodiment of the present invention. Figure 5 This is a schematic flowchart illustrating the synchronization of write-ahead log data to the slave database in a database log replay method according to an embodiment of the present invention. Figure 6 This is a schematic flowchart of a database log playback method according to another embodiment of the present invention; Figure 7 This is a schematic flowchart illustrating the insertion of pre-written log data into the log sequence to be replayed in a database log replay method according to an embodiment of the present invention. Figure 8 This is a schematic diagram of a computer program product according to an embodiment of the present invention; Figure 9 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention; Figure 10 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0020] In a master-slave database cluster, the master database cluster is deployed using a shared storage architecture, with multiple independent nodes processing business transactions in parallel. Each node independently generates write-ahead log data, recording every modification operation it makes to the database, and sends the generated write-ahead log data to the slave database in real time through its own established streaming replication link.

[0021] The slave database continuously receives write-ahead log (WAP) data streams from multiple master database nodes. Because each node operates independently, WAP data arriving at the slave database from different nodes is out of order. To ensure data consistency, the slave database must replay these WAP data streams sequentially according to the global sequence identifiers recorded in the WAP data. The smaller the global sequence identifier, the higher the priority of WAP data replay; when at least two equal global sequence identifiers exist, the smaller the master database node number, the higher the priority of WAP data replay.

[0022] To ensure globally ordered replay of write-ahead log data, a dedicated log buffer structure is allocated to each active node in the current cluster by the slave database to store the write-ahead log data currently awaiting replay. The slave database continuously compares the global sequence identifiers of the write-ahead log data in all log buffer structures and selects the smallest one as the next replay target. When write-ahead log data is extracted and replayed, a gap appears in the corresponding log buffer structure. The slave database attempts to retrieve the next write-ahead log data from the master database node to fill the log buffer structure and re-executes the global sequence identifier sorting. This process is repeated continuously, forming a continuous replay pipeline.

[0023] However, when a node in the main database cluster is idle for an extended period of time due to low business hours, uneven load, or other reasons and does not generate new business write-ahead log data, the log buffer structure corresponding to that node in the slave database will be unable to obtain new write-ahead log data to fill it for a long time, causing the replay of write-ahead log data in the slave database to stagnate.

[0024] In high-availability synchronization modes such as remote_apply, the transaction commit of the nodes in the master database cluster needs to wait for a notification from the slave database confirming that the write-ahead log data generated by the transaction has been replayed. When the replay of the write-ahead log data in the slave database stalls, the transaction processing of the nodes in the master database cluster is blocked, causing the entire master-slave database cluster to be unable to process any new transaction requests.

[0025] The following description, with reference to the accompanying drawings, illustrates a database log playback method and related products according to embodiments of the present invention.

[0026] Figure 1 This is a schematic flowchart of a database log replay method according to an embodiment of the present invention, such as... Figure 1 As shown, the log replay method for this database can generally include: Step S101: Monitor the write-ahead log data generation status of each node in the main database cluster; Step S102: In response to the detection that the target node in each node has not generated new write-ahead log data within the preset monitoring period, it is determined that the target node has entered the log idle state. Step S103: Based on the log idle status, trigger the target node to generate auxiliary write-ahead log data; Step S104: The auxiliary write-ahead log data is synchronized to the slave database via the stream replication link of the target node, so that the slave database can perform log replay.

[0027] In this embodiment, a lightweight log generation status tracker is deployed. This tracker is decoupled from the log writing threads on each node of the main database cluster. It periodically polls or receives log write completion notifications from the log writing threads to collect the number of newly added pre-written log data records and the starting sequence number of the newly added pre-written log data within the current time window for each node. The log generation status tracker maintains the collected output statistics in an in-memory status table, node by node, along with the timestamp of the last update, providing raw data support for subsequent idle state determination.

[0028] In this embodiment, the log generation status tracker scans the memory status table at a fixed frequency and calculates the difference between the current time and the last write-ahead log data generation time of each node in the main database. When it is found that the duration of a target node's idle state has continuously covered a complete preset monitoring period (the threshold of this period can be dynamically adjusted according to the characteristics of business load), and no new write-ahead log data records or write-ahead log data completion notifications are captured within the preset monitoring period, the target node is determined to have entered the log idle state.

[0029] In this embodiment, after receiving the log idle status event from the target node, the decision engine of the main database cluster verifies the validity of the event. Upon successful verification, the decision engine issues an authorization for generating auxiliary write-ahead log data to the target node. This authorization includes at least: the target node identifier, the authorization sequence number, the authorization validity period, and the decision engine's digital signature or authentication token. Upon receiving the authorization, the target node verifies its validity and validity. After successful verification, the target node assigns a global sequence identifier to the auxiliary write-ahead log data to be generated. Then, the target node constructs an auxiliary write-ahead log data entry in its local memory. This auxiliary write-ahead log data follows the standard format of write-ahead logs and does not carry any transaction operation records or data page modification images.

[0030] In this embodiment, the target node submits the persisted auxiliary write-ahead log data to the write-ahead log buffer used for performing streaming replication operations, and loads the auxiliary write-ahead log data into the streaming replication sending process according to the standard streaming replication protocol, synchronizing it to the slave database through the streaming replication link. After receiving the auxiliary write-ahead log data, the slave database inserts it into the log sequence to be replayed according to the global sequence identifier carried by the auxiliary write-ahead log data, thereby maintaining the continuity of the replay process.

[0031] Based on the above steps, by enabling the master database node to proactively perceive its own write-ahead log data generation status, auxiliary write-ahead log data is generated and sent to the slave database during the node's idle log state. This ensures that the slave database's log buffer structure is continuously filled, fundamentally solving the technical problem of the slave database being unable to complete the selection of write-ahead log data to be replayed and the replay process stalling due to a single node being idle for a long time. By maintaining the continuity of the slave database's replay process, the impact of the slave database being stuck on the master database's transaction commit is eliminated, and all nodes in the master database are prevented from being blocked while waiting for replay confirmation from the slave database, significantly improving the availability and fault recovery capabilities of the master-slave database cluster.

[0032] Figure 2 This is a schematic flowchart illustrating the process of a target node entering a log idle state in a database log replay method according to an embodiment of the present invention, as shown below. Figure 2 As shown, the steps to determine if a target node has entered a log idle state include: Step S201: If it is detected that the target node has not generated new write-ahead log data during the preparation period of the monitoring cycle, it is determined that the target node has entered the temporary log idle state. Step S202: In the temporary log idle state, continue to monitor whether the target node generates new write-ahead log data during the idle confirmation period after the preparation period in the monitoring cycle, where the sum of the preparation period and the idle confirmation period is equal to the preset monitoring cycle. Step S203: In response to the fact that the target node has not generated new write-ahead log data during the idle confirmation period, determine that the target node has migrated from the temporary log idle state to the log idle state. Step S204: In response to the target node generating new write-ahead log data during the idle confirmation period, exit the temporary log idle state and restore the write-ahead log data active state.

[0033] In this embodiment, the preset monitoring period is divided into a first monitoring sub-period and a second monitoring sub-period, which are continuous in time and do not overlap. The first monitoring sub-period serves as a preparatory period for idle determination. If the target node does not generate any new write-ahead log data within the first monitoring sub-period, it is determined that the target node has entered a temporary log idle state, and the timer for the second monitoring sub-period is started. The second monitoring sub-period serves as an idle confirmation period. While the target node is in the temporary log idle state, it continues to monitor the target node's write-ahead log data generation behavior. If the target node is still in the temporary log idle state, and the second monitoring sub-period has been fully exhausted without interruption, and the target node has not generated any new write-ahead log data within the entire second monitoring sub-period, a state transition action is triggered, and the target node is officially changed from the temporary log idle state to the log idle state; otherwise, it exits the temporary log idle state and returns to the write-ahead log data active state.

[0034] Based on the above steps, by introducing a temporary log idle state as a buffer judgment area, the preset monitoring period is divided into a preparation period and an idle confirmation period. When the target node is only temporarily idle due to transient reasons such as transaction gaps or load fluctuations, it only enters the temporary log idle state instead of immediately generating auxiliary write-ahead log data; only when no write-ahead log data is generated within two consecutive periods is the target node confirmed to have entered the log idle state and the auxiliary write-ahead log data generation operation is triggered, thereby eliminating the log storm problem caused by microsecond-level or millisecond-level business idle periods.

[0035] Figure 3 This is a schematic flowchart illustrating the determination of log idle status in a database log replay method according to an embodiment of the present invention, as shown below. Figure 3 As shown, after determining that the target node has entered the log idle state, the following steps are also included: Step S301: Initiate heartbeat detection to the target node; Step S302: If a valid confirmation response based on heartbeat detection is received from the target node within the preset response time, the determination of the log idle state is cancelled. Step S303: If not, maintain the determination of the log idle state.

[0036] Based on the above steps, after the initial assessment, a heartbeat detection is proactively initiated to the target node. Through direct, real-time, and bidirectional activity detection, the system directly verifies whether the target node is truly reachable, whether the system kernel is responding normally, and whether the log writing thread is running healthily. If a valid confirmation response is received, the idle status assessment is immediately revoked to avoid misjudging normal nodes as idle and triggering unnecessary auxiliary write-ahead log data generation, thus preventing false triggers at the source.

[0037] Figure 4 This is a schematic flowchart illustrating the generation of auxiliary write-ahead log data by the target node in a database log replay method according to an embodiment of the present invention. Figure 4 As shown, the steps to trigger the target node to generate auxiliary write-ahead log data include: Step S401: Send a generation instruction to the target node, wherein the generation instruction carries a global sequence identifier for sorting, and the global sequence identifier is greater than the sequence number of any committed write-ahead log in the main database cluster. Step S402: Based on the generation instruction, receive the request response from the target node; Step S403: After verifying the validity of the request response, the target node generates auxiliary write-ahead log data according to the generation instruction.

[0038] Based on the above steps, the generation instruction carries a global sequence identifier that is greater than the sequence number of any committed write-ahead log in the main database cluster. This global sequence identifier serves as the sorting basis for the auxiliary write-ahead log data, avoiding sequence number conflicts between the auxiliary write-ahead log data and the write-ahead log data. This also prevents the auxiliary write-ahead log data from interfering with the replay order of the write-ahead log data when sorting from the database.

[0039] Figure 5 This is a schematic flowchart illustrating the synchronization of write-ahead log data to the slave database in a database log replay method according to an embodiment of the present invention, as shown below. Figure 5 As shown, the steps for synchronizing auxiliary write-ahead log data to the slave database via the target node's streaming replication link include: Step S501: Write the auxiliary write-ahead log data into the write-ahead log buffer corresponding to the target node; Step S502: In response to the completion of writing auxiliary write-ahead log data, wake up the streaming replication sending process of the target node; In step S503, the streaming replication sending process reads auxiliary write-ahead log data from the write-ahead log buffer and pushes it to the slave database as a continuous data stream through the established streaming replication link.

[0040] It should be noted that the auxiliary write-ahead log data is used to enable the slave database to perform the following operations: The auxiliary write-ahead log data is written into the leaf node corresponding to the target node in the paired heap data structure. The paired heap data structure is used to maintain the replay order of write-ahead log data from multiple master database nodes. The root node of the paired heap data structure stores the write-ahead log data with the smallest global sequence identifier among all leaf nodes as the next write-ahead log data to be replayed. In response to the writing of auxiliary write-ahead log data to the leaf node, a reordering operation is triggered on the paired heap data structure to update the root node.

[0041] Based on the above steps, through the complete closed loop of "write → wake up → read → push → insert → reorder", the auxiliary write-ahead log data is accurately written into the leaf node corresponding to the target node in the paired heap. After the writing is completed, the leaf node obtains a valid write-ahead log data entry to be replayed, so that the paired heap data structure can regain a complete set of sorting candidates, the reordering operation can be executed smoothly, the root node can be updated, and the replay thread can obtain the next write-ahead log data to be replayed, thereby solving the problem of being stuck when replaying from the database.

[0042] Figure 6 This is a schematic flowchart of a database log replay method according to another embodiment of the present invention, such as... Figure 6 As shown, the log replay method for this database can generally include: Step S601: Monitor the status of receiving write-ahead log data from the database; Step S602: In response to receiving auxiliary write-ahead log data, insert the auxiliary write-ahead log data into the replay log sequence maintained from the database; Step S603: Based on the updated log sequence to be replayed, continue to perform the log replay operation.

[0043] In this embodiment, the slave database maintains a long connection with the master database cluster via a streaming replication protocol and continuously receives write-ahead log data packets pushed by each node of the master database cluster. The log receiving thread of the slave database performs header parsing on each arriving write-ahead log data to identify its log type field. When the log type field is detected to be marked as the predefined "auxiliary write-ahead log", it is determined that an auxiliary write-ahead log data has been successfully received.

[0044] In this embodiment, the global sequence identifier encapsulated in the header of the auxiliary write-ahead log data is extracted from the database, and the auxiliary write-ahead log data is written into the locally maintained log sequence to be replayed based on the global sequence identifier. The write operation follows the order constraint of the sequence to ensure that the auxiliary write-ahead log data is placed at the logical position corresponding to its global sequence identifier, without destroying the original global monotonically increasing order of the sequence to be replayed.

[0045] In this embodiment, the database is scanned in a cyclic manner to retrieve the log sequence to be replayed, and ready-to-write log data records are retrieved sequentially from the head of the sequence for replay. After the auxiliary write-ahead log data is successfully inserted, the length of the log sequence to be replayed increases, the sequence head pointer continues to advance, and the log replay operation is performed.

[0046] Based on the above steps, by receiving and inserting auxiliary write-ahead log data from the database, the log buffer structure corresponding to the target node in the main database can be continuously populated. The sequence of logs to be replayed in the slave database always remains complete, and all active main database nodes have corresponding write-ahead log data to be replayed for sorting, thereby completely eliminating the problem of the replay process being permanently stuck due to missing logs from a single node.

[0047] Figure 7 This is a schematic flowchart illustrating the insertion of write-ahead log data into the log sequence to be replayed in a database log replay method according to an embodiment of the present invention, as shown below. Figure 7 As shown, the steps for inserting auxiliary write-ahead log data into a sequence of logs to be replayed maintained from the database include: Step S701: Identify the global sequence identifier carried by the auxiliary write-ahead log data; Step S702: Compare the global sequence identifier with the start sort identifier and end sort identifier of the log sequence to be replayed; Step S703: If the global sequence identifier is greater than the end sorting identifier, the auxiliary write-ahead log data is appended to the end of the log sequence to be replayed. Step S704: If the global sequence identifier is less than the starting sorting identifier, it is determined that the auxiliary write-ahead log data has a header out-of-bounds exception, and a retransmission request is sent to the target node. Step S705: If the global sequence identifier is between the start sort identifier and the end sort identifier, the auxiliary write-ahead log data is inserted into the insertion position at the location of the log sequence to be replayed that matches the global sequence identifier.

[0048] Based on the above steps, the auxiliary write-ahead log data is strictly inserted into the exact position of the log sequence to be replayed, according to the global sequence identifier carried by it. If the sequence is greater than the end, it is appended; if it is in between, it is inserted in the middle; and if it is less than the beginning, exception handling is triggered. This insertion logic completely follows the natural ordering rules of the global sequence identifier and is entirely consistent with the ordering criteria of the business write-ahead log data. The database does not need to open a separate sorting channel for the auxiliary write-ahead log data, fundamentally eliminating the risk of order disorder and ensuring strong consistency of the database master-slave cluster in any scenario.

[0049] The flowchart provided in this embodiment is not intended to indicate that the operations of the method will be performed in any particular order, or that all operations of the method are included in every case. Furthermore, the method may include additional operations. Within the scope of the technical concept provided by the method in this embodiment, additional variations can be made to the above method.

[0050] It should be understood that in some embodiments, the components may be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods may be implemented using software or firmware stored in memory and executed by a suitable instruction execution system.

[0051] This embodiment also provides a computer program product 10, a computer-readable storage medium 20, and a computer device 30. Figure 8 This is a schematic diagram of a computer program product 10 according to an embodiment of the present invention. Figure 9 This is a schematic diagram of a computer-readable storage medium 20 according to an embodiment of the present invention. Figure 10This is a schematic diagram of a computer device 30 according to an embodiment of the present invention. The computer program product 10 includes a computer program 11, which, when executed by the processor 32, implements the steps of any of the above-described methods. A computer-readable storage medium 20 stores the computer program 11 thereon, which, when executed by the processor 32, implements the steps of the database log playback method of any of the above embodiments. The computer device 30 may include a memory 31, a processor 32, and the computer program 11 stored on the memory 31 and running on the processor 32.

[0052] The computer program 11 used to perform the operations of this invention may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages ​​and procedural programming languages. The computer program 11 may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer. In some embodiments, to perform aspects of this invention, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer-readable program instructions to personalize the electronic circuits by utilizing status information of the computer-readable program instructions.

[0053] For the purposes of this embodiment, computer program product 10 is a related product that includes computer program 11.

[0054] For the purposes of this embodiment, computer-readable storage medium 20 is a tangible device capable of holding and storing a computer program 11. It can be any device capable of containing, storing, communicating, propagating, or transmitting the program 11 for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable storage medium 20 include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical encoding device, and any suitable combination thereof.

[0055] Computer device 30 can be, for example, a server, desktop computer, laptop computer, tablet computer, or smartphone. In some examples, computer device 30 can be a cloud computing node. Computer device 30 can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. Computer device 30 can be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can reside on local or remote computing system storage media, including storage devices.

[0056] Computer device 30 may include a processor 32 adapted to execute stored instructions and a memory 31 that provides temporary storage space for the operation of said instructions during operation. The processor 32 may be a single-core processor, a multi-core processor, a computing cluster, or any other configuration. The memory 31 may include random access memory (RAM), read-only memory, flash memory, or any other suitable storage system.

[0057] Computer device 30 may also include a network adapter / interface and an input / output (I / O) interface. The I / O interface allows external devices that can be connected to the computer device to input and output data. The network adapter / interface provides communication between the computer device and a network, typically represented as a communication network.

[0058] Therefore, those skilled in the art should recognize that although numerous exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications conforming to the principles of the present invention can be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Thus, the scope of the present invention should be understood and construed as covering all such other variations or modifications.

Claims

1. A method for replaying database logs, characterized in that, include: Monitor the write-ahead log data generation status of each node in the main database cluster; In response to the detection that the target node among the nodes has not generated new write-ahead log data within the preset monitoring period, it is determined that the target node enters the log idle state. Based on the log idle state, the target node is triggered to generate auxiliary write-ahead log data; The auxiliary write-ahead log data is synchronized to the slave database via the stream replication link of the target node, for the slave database to perform log replay.

2. The database log replay method according to claim 1, characterized in that, The steps to determine that the target node has entered the log idle state include: If it is detected that the target node does not generate new write-ahead log data during the preparation period in the monitoring cycle, the target node is determined to enter a temporary log idle state. In the temporary log idle state, continue to monitor whether the target node generates new write-ahead log data during the idle confirmation period after the preparation period in the monitoring cycle, wherein the sum of the preparation period and the idle confirmation period is equal to the preset monitoring cycle; If the target node fails to generate new write-ahead log data during the idle confirmation period, the target node is determined to transition from the temporary log idle state to the log idle state. In response to the target node generating new write-ahead log data during the idle confirmation period, the temporary log idle state is exited and the write-ahead log data active state is restored.

3. The database log replay method according to claim 1, characterized in that, After determining that the target node has entered the log idle state, the process also includes: Initiate heartbeat detection to the target node; If a valid confirmation response based on the heartbeat detection is received from the target node within the preset response time, the determination of the log idle state is revoked. If not, maintain the determination of the log's idle state.

4. The database log replay method according to claim 1, characterized in that, The steps to trigger the target node to generate auxiliary write-ahead log data include: Send a generation instruction to the target node, wherein the generation instruction carries a global sequence identifier for sorting, and the global sequence identifier is greater than the sequence number of any committed write-ahead log in the main database cluster; Based on the generated instructions, a request response is received from the target node; After verifying the validity of the request response, the target node generates auxiliary write-ahead log data according to the generation instruction.

5. The database log replay method according to claim 1, characterized in that, The steps of synchronizing the auxiliary write-ahead log data to the slave database via the streaming replication link of the target node include: Write the auxiliary write-ahead log data into the write-ahead log buffer corresponding to the target node; In response to the completion of the writing of the auxiliary write-ahead log data, the streaming replication and sending process of the target node is awakened; The streaming replication sending process reads the auxiliary write-ahead log data from the write-ahead log buffer and pushes it to the slave database in the form of a continuous data stream through the established streaming replication link; The auxiliary write-ahead log data is used to enable the database to perform the following operations: The auxiliary write-ahead log data is written into the leaf node corresponding to the target node in the paired heap data structure. The paired heap data structure is used to maintain the replay order of write-ahead log data from multiple master database nodes. The root node of the paired heap data structure stores the write-ahead log data with the smallest global sequence identifier among all leaf nodes as the next write-ahead log data to be replayed. In response to the auxiliary write-ahead log data being written to the leaf node, the paired heap data structure is triggered to perform a reordering operation to update the root node.

6. A method for replaying database logs, characterized in that, include: Monitor the status of data received from the database's write-ahead logs; In response to receiving auxiliary write-ahead log data, the auxiliary write-ahead log data is inserted into the replay log sequence maintained from the database; Based on the updated log sequence to be replayed, continue to perform the log replay operation.

7. The database log replay method according to claim 6, characterized in that, The step of inserting the auxiliary write-ahead log data into the replay log sequence maintained from the database includes: Identify the global sequence identifier carried by the auxiliary write-ahead log data; The global sequence identifier is compared with the start sort identifier and end sort identifier of the log sequence to be replayed; If the global sequence identifier is greater than the end sorting identifier, the auxiliary write-ahead log data is appended to the end of the log sequence to be replayed; If the global sequence identifier is less than the starting sorting identifier, it is determined that the auxiliary write-ahead log data has a header out-of-bounds exception, and a retransmission request is sent to the target node. If the global sequence identifier is between the start sort identifier and the end sort identifier, the auxiliary write-ahead log data is inserted into the insertion position that matches the global sequence identifier in the log sequence to be replayed.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the database log playback method according to any one of claims 1 to 7.

9. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the database log playback method according to any one of claims 1 to 7.

10. A computer device, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the log playback method for the database according to any one of claims 1 to 7.

Images