A shared storage-based database cluster stream replication method and related product

Abstract

This invention provides a database cluster streaming replication method and related products based on shared storage. The method includes: monitoring the continuous idle time of the master database node; in response to the continuous idle time exceeding a first threshold, the master database node generates auxiliary write-ahead log data at a preset period and sends it to the slave database; in response to the continuous idle time exceeding a second threshold, the master database node stops generating auxiliary write-ahead log data and sends a state transition log to the slave database. The solution of this invention, by periodically generating auxiliary write-ahead log data and filling the leaf nodes of the paired heap within the first threshold, ensures that the leaf nodes of the slave database are always active, avoiding the problem of slave database replay blocking due to a node having no business logs for a long time. By stopping the generation of auxiliary write-ahead log data after the continuous idle time exceeds the second threshold and removing the corresponding leaf node from the paired heap, the waste of storage space and I / O resources is eliminated.

Description

Technical Field

[0001] This invention relates to database technology, and in particular to a database cluster streaming replication method and related products based on shared storage. Background Technology

[0002] In a master-slave database cluster, the slave database sorts and replays data change records from multiple master database nodes using a priority data structure. When the business data change frequency of a certain node is too low, the slave database will be blocked from replaying due to waiting for records from that node. Currently, the blocking problem is solved by having idle nodes actively generate auxiliary data change records. However, excessively high auxiliary record generation frequency leads to inefficient use of storage space and I / O resources. Therefore, how to balance the auxiliary record generation frequency with resource consumption has become an urgent technical problem to be solved. Summary of the Invention

[0003] One objective of this invention is to improve the playback continuity of slave databases in a master-slave database cluster and avoid playback blockage caused by idle master database nodes.

[0004] A further objective of this invention is to reduce the resource consumption caused by the generation of auxiliary data change records, thereby reducing the ineffective use of storage space and I / O resources.

[0005] Specifically, this invention provides a database cluster streaming replication method based on shared storage, comprising: Monitor the continuous idle time of the main database node; In response to the continuous idle time exceeding a first threshold, the master database node generates auxiliary write-ahead log data at a preset period and sends it to the slave database. The auxiliary write-ahead log data is used to fill the leaf nodes corresponding to the master database node in the pair heap maintained by the slave database. In response to the continuous idle time exceeding a second threshold, the master database node stops generating the auxiliary write-ahead log data and sends a state transition log to the slave database, the state transition log being used to instruct the slave database to remove the leaf node from the pairing heap, wherein the second threshold is greater than the first threshold.

[0006] Optionally, the step of the master database node generating auxiliary write-ahead log data at a preset period and sending it to the slave database includes: A timer is started when the continuous idle time exceeds a first threshold. In response to the timer reaching the preset period, the main database node is triggered to generate an auxiliary write-ahead log data containing a preset fill flag, wherein the preset fill flag is used to indicate that the auxiliary write-ahead log data is an empty log that does not contain business data. The generated auxiliary write-ahead log data is sent to the slave database, the active state of the leaf nodes in the paired heap is maintained based on the preset fill flag, and the timer is reset to start counting again.

[0007] Optionally, the database cluster streaming replication method based on shared storage further includes: If the master database node receives a write-ahead log data request during the timer's countdown period, it is determined that the master database node has resumed its active state. In response to the primary database node regaining an active state, the timer is stopped, and the generation operation of the auxiliary write-ahead log data is terminated. The write-ahead log data corresponding to the write request is sent to the secondary database. The secondary database updates the log sequence number corresponding to the leaf node in the pairing heap based on the write-ahead log data, and adjusts the replay priority of the leaf node in the pairing heap according to the updated log sequence number. The replay priority is negatively correlated with the value of the log sequence number.

[0008] Optionally, the step of stopping the primary database node from generating auxiliary write-ahead log data and sending state transition logs to the secondary database includes: When the timer is turned off, the main database node stops generating auxiliary write-ahead log data; A state transition log carrying a resource release identifier is generated based on the master database node; The state transition log is sent to the slave database, whereby the slave database removes the leaf node from the pairing heap based on the resource release identifier.

[0009] Optionally, the database cluster streaming replication method based on shared storage further includes: If the master database node receives a write-ahead log data write request during the continuous idle time exceeding the first threshold but not exceeding the second threshold, it is determined that the master database node has resumed its active state. In response to the primary database node becoming active again, the primary database node stops generating auxiliary write-ahead log data and sends the write-ahead log data corresponding to the write request to the slave database.

[0010] In particular, this invention provides another method for streaming replication of a database cluster based on shared storage, comprising: Maintain a pairing heap, which includes at least one leaf node, and each leaf node corresponds to a master database node; Receive auxiliary write-ahead log data generated by the main database node at a preset period, wherein the auxiliary write-ahead log data is generated when the continuous idle time of the main database node exceeds a first threshold; Based on the auxiliary write-ahead log data, fill the leaf nodes corresponding to the primary database nodes in the paired heap; The system receives a state transition log sent by the master database node when the continuous idle time exceeds a second threshold. Based on the state transition log, the system removes the leaf node from the pairing heap, wherein the second threshold is greater than the first threshold.

[0011] Optionally, the database cluster streaming replication method based on shared storage further includes: The system receives write-ahead log data sent by the master database node after it recovers to an active state. The write-ahead log data is generated when the master database node receives a write-ahead log data request during the timer's countdown period, or when the master database node receives a write-ahead log data request during the period when the continuous idle time exceeds a first threshold but does not exceed a second threshold. Based on the pre-written log data, update the log sequence number corresponding to the leaf node in the paired heap; Adjust the replay priority of the leaf node in the pairing heap based on the updated log sequence number.

[0012] According to another aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of any of the above-described shared storage-based database cluster streaming replication methods.

[0013] According to another aspect of the present invention, a computer program product is also provided, comprising a computer program that, when executed by a processor, implements the steps of any of the above-described shared-memory-based database cluster streaming replication methods.

[0014] According to another aspect of the present invention, a computer device is also provided, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any shared-memory-based database cluster streaming replication method.

[0015] The shared storage-based database cluster streaming replication method of this invention ensures that the leaf nodes corresponding to each master database node in the paired heap are always active by periodically generating auxiliary write-ahead log data and filling the leaf nodes in the paired heap within a first threshold. This avoids the problem of database replay blocking caused by a node having no business logs for a long time, thus guaranteeing the continuity and stability of data synchronization. By stopping the generation of auxiliary write-ahead log data and removing the corresponding leaf node from the paired heap after a continuous idle time exceeds a second threshold, the waste of storage space and I / O resources caused by long-term idle nodes continuously generating empty logs without business data is prevented.

[0016] Furthermore, in the shared storage-based database cluster streaming replication method of the present invention, when the master database node recovers from an idle state and generates business write-ahead log data, the slave database updates the log sequence number stored in the corresponding leaf node in the paired heap based on this log data. Since the log sequence number monotonically increases, the updated sequence number accurately reflects the time order of the node's latest transactions, ensuring the correctness of the global replay order and data consistency. Based on the updated log sequence number, the slave database dynamically adjusts the replay priority of the leaf node in the paired heap, giving higher priority to leaf nodes with smaller log sequence numbers and correspondingly lower priority to leaf nodes with larger log sequence numbers. This allows the paired heap to respond in real-time to changes in the master database node's business status, always maintaining the optimal replay order.

[0017] The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0018] The following sections will describe some specific embodiments of the invention in detail by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or portions. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings: Figure 1 This is a schematic flowchart of a database cluster streaming replication method based on shared storage according to an embodiment of the present invention; Figure 2 This is a schematic flowchart illustrating auxiliary write-ahead log data communication in a shared storage-based database cluster streaming replication method according to an embodiment of the present invention. Figure 3 This is a schematic flowchart illustrating the recovery of the primary database node to an active state in a shared storage-based database cluster streaming replication method according to an embodiment of the present invention. Figure 4 This is a schematic flowchart illustrating the process of stopping the generation of auxiliary write-ahead log data by the primary database node in a shared storage-based database cluster streaming replication method according to an embodiment of the present invention. Figure 5 This is a schematic flowchart illustrating a scenario in a shared storage-based database cluster streaming replication method according to an embodiment of the present invention, where the continuous idle time exceeds a first threshold but does not exceed a second threshold. Figure 6 This is a schematic flowchart of a database cluster streaming replication method based on shared storage according to another embodiment of the present invention; Figure 7This is a schematic flowchart illustrating the process of receiving write-ahead log data sent by the master database node after it has recovered to an active state in a database cluster streaming replication method based on shared storage according to an embodiment of the present invention. Figure 8 This is a schematic diagram of a computer program product according to an embodiment of the present invention; Figure 9 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention; Figure 10 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0019] In a distributed database master-slave cluster architecture, slave database nodes replay transactional data change records (such as write-ahead log data) from multiple master database nodes in an ordered manner to ensure eventual consistency between master and slave data. To balance replay efficiency and data priority among multiple master database nodes, slave databases generally use priority data structures (such as paired heaps, red-black trees, etc.) to sort and schedule transactional data change records from different master database nodes. That is, they prioritize replaying records from nodes with high-frequency changes, while reserving replay queue positions for nodes with low-frequency changes.

[0020] However, when a primary database node experiences a low frequency of business data changes due to business downturns or cold data access, the priority data structure of the secondary database will become blocked as it continuously waits for valid change records from that node. For example, the queue position corresponding to this node remains in a "pending fill" state for an extended period, blocking the batch replay of records from subsequent high-priority nodes and causing a decrease in the overall replay throughput. Furthermore, the scheduling thread of the secondary database will repeatedly poll the record status of this idle node, consuming additional CPU and memory resources and further reducing the processing efficiency of the secondary database.

[0021] To address this congestion issue, existing solutions propose having idle primary database nodes proactively generate auxiliary data change records (e.g., auxiliary write-ahead log data). By filling the queue position corresponding to that node in the slave database's priority data structure, the replay congestion is broken, and the normal scheduling logic of the slave database is restored. However, this solution introduces new resource consumption problems: if the generation frequency of auxiliary records is too high, a large number of logs with no business value will be continuously written to the slave database storage medium, which will not only quickly occupy limited storage space but also trigger frequent disk I / O operations, thereby crowding out the read and write bandwidth of normal business logs and increasing the response latency of the slave database. At the same time, the process of primary database nodes generating auxiliary records also consumes their own CPU and network bandwidth resources, affecting their ability to respond to sudden business requests.

[0022] Therefore, how to dynamically balance the generation frequency of auxiliary records with the consumption of system resources has become a core technical problem that urgently needs to be solved in the optimization of distributed database master-slave clusters.

[0023] This embodiment provides a solution for a database cluster streaming replication method based on shared storage. While ensuring high availability and data consistency of the database cluster, it achieves a dynamic balance between playback continuity and resource consumption.

[0024] Figure 1 This is a schematic flowchart of a database cluster streaming replication method based on shared storage according to an embodiment of the present invention, as shown below. Figure 1 As shown, this shared storage-based database cluster streaming replication method generally includes: Step S101: Monitor the continuous idle time of the master database node; In step S102, in response to the continuous idle time exceeding the first threshold, the master database node generates auxiliary write-ahead log data at a preset period and sends it to the slave database. The auxiliary write-ahead log data is used to fill the leaf nodes corresponding to the master database node in the pair heap maintained by the slave database. In step S103, in response to the continuous idle time exceeding the second threshold, the master database node stops generating auxiliary write-ahead log data and sends a state transition log to the slave database. The state transition log is used to indicate that the slave database removes the leaf node from the pairing heap, where the second threshold is greater than the first threshold.

[0025] In this embodiment, an idle monitoring thread is used to track the real-time running status of the main database node. This idle monitoring thread uses a transaction-triggered idle time accumulation mechanism to obtain idle periods during the main database node's database transaction processing. The idle monitoring thread is designed as a lightweight background daemon thread, created when the main database system starts and destroyed when the main database system shuts down. The idle monitoring thread has a medium-to-low priority to avoid competing for CPU resources with high-priority transaction processing threads; it is periodically scheduled in fixed time slices (e.g., 1 millisecond); it can optionally be bound to a specific CPU core to reduce cache misses and scheduling latency; and it has a reasonable stack size (e.g., 64KB) and memory usage limit to avoid excessive resource consumption.

[0026] In this embodiment, the idle monitoring thread operates in a periodic wake-up mode. After the idle monitoring thread starts, it reads the last active timestamp stored in shared memory through atomic operations. This last active timestamp records the moment when the last transaction of the main database node was completed (i.e., when write-ahead log data was generated). The idle monitoring thread obtains the current time by calling the time interface. The idle monitoring thread calculates the difference between the current time and the last active time to obtain the continuous idle time of the main database node.

[0027] In this embodiment, if the continuous idle time of the primary database node exceeds a preset first threshold (this first threshold is determined comprehensively based on factors such as the performance requirements of the primary database, data consistency requirements, and network environment, for example, set to 3 seconds), the idle monitoring thread will send a trigger signal to the primary database node. After receiving the trigger signal, the primary database node starts generating auxiliary write-ahead log data. The primary database node performs a comprehensive scan of its own memory state, collecting information including dirty data in the data cache (i.e., data that has been modified but not yet written to disk), transaction processing status information (such as the list of ongoing transactions, transaction lock status, etc.), and system configuration parameters; according to a preset period (e.g., every 500 milliseconds) and in accordance with the standard format of database logs, the above information is encoded into auxiliary write-ahead log data. During the generation process, to ensure data integrity and recoverability, checksums, sequence numbers, and other technical means are used to process the auxiliary write-ahead log data.

[0028] In this embodiment, after generating auxiliary write-ahead log data, the master database node sends the auxiliary write-ahead log data to the slave database via the network. During data transmission, a reliable communication protocol (such as TCP) is used, and mechanisms such as data retransmission and congestion control are enabled to ensure that the auxiliary write-ahead log data arrives at the slave database accurately and completely.

[0029] In this embodiment, after receiving the auxiliary write-ahead log data, the slave database verifies the data's integrity and accuracy. Based on the identification information in the auxiliary write-ahead log data, it locates the leaf node corresponding to the master database node in a pre-maintained paired heap data structure. It should be noted that the paired heap offers fast insertion and lookup operations. The slave database fills the parsed auxiliary write-ahead log data into this leaf node and updates its status information, enabling the slave database to synchronize the master database node's status in real time. This provides accurate data support for subsequent data recovery and failover.

[0030] In this embodiment, if the continuous idle time of the master database node exceeds a pre-set second threshold (the second threshold is greater than the first threshold; the second threshold is set to distinguish different idle states of the master database node and avoid frequent log generation and state transition operations from affecting system performance), the idle monitoring thread will send a trigger signal to the master database node again. Upon receiving this trigger signal, the master database node immediately stops generating auxiliary write-ahead log data. At this time, the master database node generates a state transition log containing clear indication information to inform the slave database that the current state of the master database node has changed and corresponding operations are required. After generating the state transition log, the master database node immediately sends it to the slave database. Upon receiving the state transition log, the slave database parses and verifies it. Based on the indication information in the log, the slave database finds the leaf node corresponding to the master database node in the paired heap data structure and performs a deletion operation, removing the leaf node from the paired heap. This operation helps release the storage resources of the slave database, reduces unnecessary computational overhead, and ensures that the data structure of the slave database remains consistent with the actual state of the master database node.

[0031] Based on the above steps, by periodically generating auxiliary write-ahead log data and filling the leaf nodes in the paired heap within the first threshold, it is ensured that the leaf nodes corresponding to each master database node in the paired heap of the slave database are always in an active state. This avoids the problem of slave database replay being blocked due to a node having no business logs for a long time, thus ensuring the continuity and stability of data synchronization. By stopping the generation of auxiliary write-ahead log data after the continuous idle time exceeds the second threshold and removing the corresponding leaf node from the paired heap, the waste of storage space and I / O resources caused by long-term idle nodes continuously generating empty logs without business data is prevented.

[0032] Figure 2 This is a schematic flowchart illustrating auxiliary write-ahead log data communication in a shared-storage-based database cluster streaming replication method according to an embodiment of the present invention, as shown below. Figure 2 As shown, the steps for the master database node to generate auxiliary write-ahead log data and send it to the slave database at a preset period include: Step S201: Start the timer when the continuous idle time exceeds the first threshold; Step S202: In response to the timer's countdown value reaching the preset period, the main database node is triggered to generate an auxiliary write-ahead log data containing a preset fill flag, wherein the preset fill flag is used to indicate that the auxiliary write-ahead log data is an empty log that does not contain business data. Step S203: Send the generated auxiliary write-ahead log data to the slave database, maintain the active state of the leaf nodes in the paired heap based on the preset fill flag, and reset the timer to start timing again.

[0033] In this embodiment, if the idle monitoring thread detects that the continuous idle time exceeds a first threshold, and confirms that the synchronization link of the database master-slave cluster is unobstructed and the master database node is in normal operating condition (fault-free, maintenance-free mode), the timer built into the master database is started. This timer is bound to the thread that generates auxiliary write-ahead log data. Upon startup, the timer value is initialized to 0, and a preset period is read from the master database configuration file. This preset period is used as the trigger threshold for the timer to ensure that the timer period matches the preset requirements. After startup, the timer enters a continuous timing state.

[0034] In this embodiment, after the timer starts, it continuously accumulates a count. When the count reaches a preset period, a trigger signal is sent to the auxiliary write-ahead log data generation thread to trigger the generation of auxiliary write-ahead log data. Upon receiving the trigger signal, the auxiliary write-ahead log data generation thread generates a piece of auxiliary write-ahead log data containing a preset padding identifier. This preset padding identifier is typically a fixed byte string used to indicate to the database that this auxiliary write-ahead log data is an empty log with no actual business value, used only to maintain the active state of the paired heap leaf nodes, and does not require business data parsing or playback operations. After the auxiliary write-ahead log data is generated, it is immediately written to the main database's local log buffer, awaiting a command.

[0035] In this embodiment, after the auxiliary write-ahead log data is written to the log buffer, the master database's synchronization sending process reads the auxiliary write-ahead log data and sends it to the slave database via the synchronization link. After receiving the auxiliary write-ahead log data, the slave database's log parsing thread extracts the preset filling flag, confirming that this auxiliary write-ahead log data is an auxiliary empty log. No business replay is required; only the last update timestamp of the corresponding leaf node in the paired heap is updated to maintain the "active state" of the leaf node, preventing the slave database from falling into replay blocking due to long-term lack of data updates.

[0036] Based on the above steps, the logic of "timer + periodic generation of auxiliary write-ahead log data" ensures that the master database can continuously send data to the slave database when idle, avoiding the slave database being blocked from replaying due to waiting for valid business logs from the master database. The slave database identifies the auxiliary write-ahead log data through a preset filling flag, without performing business parsing and replay operations, only needing to update the active status of the leaf nodes in the paired heap, avoiding invalid business replay overhead, ensuring the normal scheduling and replay of business logs from other master databases by the slave database, and improving the overall throughput of the synchronization link.

[0037] Figure 3 This is a schematic flowchart illustrating the process of restoring the primary database node to an active state in a shared storage-based database cluster streaming replication method according to an embodiment of the present invention. Figure 3As shown, this database cluster streaming replication method based on shared storage also includes: Step S301: If the master database node receives a write-ahead log data request during the timer's timing period, it is determined that the master database node has recovered to an active state. In step S302, in response to the master database node regaining its active state, the timer is stopped and the auxiliary write-ahead log data generation operation is terminated. The write-ahead log data corresponding to the write request is sent to the slave database. The slave database updates the log sequence number corresponding to the leaf node in the pairing heap based on the write-ahead log data, and adjusts the replay priority of the leaf node in the pairing heap according to the updated log sequence number. The replay priority is negatively correlated with the value of the log sequence number.

[0038] Based on the above steps, by monitoring write requests for write-ahead log data, the recovery status of the primary database can be determined in real time without waiting for the timer cycle to end, enabling rapid response to business recovery. Upon the primary database node's recovery status, the timer is immediately stopped, terminating the generation of auxiliary write-ahead log data. This prevents the parallel generation of auxiliary write-ahead log data and business-related write-ahead log data, and avoids auxiliary write-ahead log data preempting the primary database's CPU, memory, and network resources. The slave database updates the log sequence number of the corresponding leaf node based on this write-ahead log data, achieving synchronization and alignment of the primary and slave log sequences. This avoids log sequence number confusion caused by the accumulation of auxiliary write-ahead log data, preventing data misalignment or omissions during slave database replay, and further ensuring primary-slave data consistency.

[0039] Figure 4 This is a schematic flowchart illustrating the process of stopping the primary database node from generating auxiliary write-ahead log data in a shared-storage-based database cluster streaming replication method according to an embodiment of the present invention. Figure 4 As shown, the steps to stop the primary database node from generating secondary write-ahead log data and to send state transition logs to the secondary databases include: Step S401: Turn off the timer, and the primary database node stops generating auxiliary write-ahead log data; Step S402: Generate a state transition log carrying a resource release identifier based on the master database node; Step S403: The state transition log is sent to the slave database, where the slave database removes the leaf node from the pairing heap based on the resource release flag.

[0040] Based on the above steps, when the continuous idle time exceeds the second threshold, the timer is preferentially shut down to stop the generation of auxiliary write-ahead log data. This prevents the primary database from continuously consuming CPU, memory, and local storage resources when it does not need to keep the leaf nodes of the secondary database active. After stopping the generation of auxiliary write-ahead log data, the primary database can allocate all the released resources to business processing, ensuring the response rate of business requests, achieving reasonable allocation of primary database resources, and avoiding the occupation of invalid resources.

[0041] Figure 5 This is a schematic flowchart illustrating a scenario in a shared-storage-based database cluster streaming replication method according to an embodiment of the present invention, where the continuous idle time exceeds a first threshold but does not exceed a second threshold. Figure 5 As shown, this database cluster streaming replication method based on shared storage also includes: Step S501: If the master database node receives a write-ahead log data write request during the continuous idle time exceeding the first threshold but not exceeding the second threshold, it is determined that the master database node has recovered to an active state. In step S502, in response to the primary database node regaining its active state, the primary database node stops generating auxiliary write-ahead log data and sends the write-ahead log data corresponding to the write request to the secondary database.

[0042] Based on the above steps, if the main database is identified as being in a short-term idle scenario where the continuous idle time exceeds the first threshold but does not exceed the second threshold, when the main database receives a write request for write-ahead log data, it immediately determines to resume activity and stops generating auxiliary write-ahead log data. There is no need to wait for the continuous idle time to reach the second threshold, nor is there a need to perform leaf node removal operations. This avoids the resource consumption of "continuously generating auxiliary write-ahead log data after the short-term idle period is restored" and saves the overhead of node removal and subsequent reconstruction, thereby achieving precise resource management.

[0043] Figure 6 This is a schematic flowchart of a database cluster streaming replication method based on shared storage according to another embodiment of the present invention, as shown below. Figure 6 As shown, this shared storage-based database cluster streaming replication method generally includes: Step S601: Maintain a pairing heap, which includes at least one leaf node, and each leaf node corresponds to a master database node; Step S602: Receive auxiliary write-ahead log data generated by the master database node at a preset period. The auxiliary write-ahead log data is generated when the continuous idle time of the master database node exceeds a first threshold. Step S603: Based on the auxiliary write-ahead log data, populate the leaf nodes corresponding to the primary database nodes in the paired heap; Step S604: Receive the state transition log sent by the master database node when the continuous idle time exceeds the second threshold. Based on the state transition log, remove the leaf node from the pairing heap, where the second threshold is greater than the first threshold.

[0044] In this embodiment, the database maintains a pairing heap data structure in memory to manage the replay order of write-ahead log data from multiple master database nodes. The pairing heap is an implementation of a priority queue, with a time complexity of O(1) for insertion and merging operations and O(log n) for deleting the smallest element. Therefore, it is suitable for dynamically changing node sets. Since the pairing heap does not require maintaining a safe binary tree structure, its memory usage is more flexible.

[0045] In this embodiment, since the auxiliary write-ahead log data is a non-blocking incremental log used for asynchronous synchronization during periods of light load, it does not affect the normal business writes to the main database. The auxiliary write-ahead log data is received and cached from the database. Based on the main database node identifier carried in the auxiliary write-ahead log data, the database locates the leaf node corresponding to the main database node in the paired heap, and fills the leaf node with information such as the log segment, offset, timestamp, and idle duration of the auxiliary write-ahead log data, thus completing the update of the status and log information of the main database node.

[0046] In this embodiment, when the continuous idle time of the primary database node further increases and exceeds a larger second threshold (the second threshold is greater than the first threshold), the primary database node enters a switchable / offline state, generating and sending a state transition log. The primary database node identifier in the state transition log is parsed from the database, and the corresponding leaf node is precisely located and atomically deleted in the paired heap, thus removing the primary database node.

[0047] Based on the above steps, by maintaining a pairing heap and assigning leaf nodes to each master database node, a one-to-one correspondence between master database and slave database pairing heap nodes is established, providing a basic framework for the orderly scheduling of log replay of multiple master database nodes by slave databases. At the same time, the unified pairing heap maintenance logic can avoid log chaos among multiple master database nodes, ensuring that slave databases can accurately associate with the pre-written log data of each master database node, thereby improving the standardization and efficiency of scheduling.

[0048] Figure 7 This is a schematic flowchart illustrating the process of receiving write-ahead log data sent by the master database node after it recovers to an active state in a shared storage-based database cluster streaming replication method according to an embodiment of the present invention. Figure 7 As shown, this database cluster streaming replication method based on shared storage also includes: Step S701: Receive write-ahead log data sent by the master database node after it recovers to an active state. The write-ahead log data is generated when the master database node receives a write-ahead log data write request during the timer's countdown period, or when the master database node receives a write-ahead log data write request during a continuous idle time exceeding a first threshold but not exceeding a second threshold. Step S702: Based on the write-ahead log data, update the log sequence number corresponding to the leaf node in the paired heap; Step S703: Adjust the replay priority of the leaf node in the pairing heap according to the updated log sequence number.

[0049] Based on the above steps, when the primary database node recovers from an idle state and generates business write-ahead log data, the secondary database updates the log sequence number stored in the corresponding leaf node in the paired heap based on this log data. Since the log sequence number monotonically increases, the updated sequence number accurately reflects the time order of the node's latest transactions, ensuring the correctness of the global replay order and data consistency. Based on the updated log sequence number, the secondary database dynamically adjusts the replay priority of the leaf node in the paired heap, giving higher priority to leaf nodes with smaller log sequence numbers and correspondingly lower priority to leaf nodes with larger log sequence numbers. This allows the paired heap to respond in real-time to changes in the primary database node's business status, always maintaining the optimal replay order.

[0050] The flowchart provided in this embodiment is not intended to indicate that the operations of the method will be performed in any particular order, or that all operations of the method are included in every case. Furthermore, the method may include additional operations. Within the scope of the technical concept provided by the method in this embodiment, additional variations can be made to the above method.

[0051] It should be understood that in some embodiments, the components may be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods may be implemented using software or firmware stored in memory and executed by a suitable instruction execution system.

[0052] This embodiment also provides a computer program product 10, a computer-readable storage medium 20, and a computer device 30. Figure 8 This is a schematic diagram of a computer program product 10 according to an embodiment of the present invention. Figure 9 This is a schematic diagram of a computer-readable storage medium 20 according to an embodiment of the present invention. Figure 10This is a schematic diagram of a computer device 30 according to an embodiment of the present invention. The computer program product 10 includes a computer program 11, which, when executed by the processor 32, implements the steps of any of the above-described methods. A computer-readable storage medium 20 stores the computer program 11 thereon, which, when executed by the processor 32, implements the steps of the shared-memory-based database cluster streaming replication method of any of the above embodiments. The computer device 30 may include a memory 31, a processor 32, and the computer program 11 stored on the memory 31 and running on the processor 32.

[0053] The computer program 11 used to perform the operations of this invention may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages ​​and procedural programming languages. The computer program 11 may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer. In some embodiments, to perform aspects of this invention, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer-readable program instructions to personalize the electronic circuits by utilizing status information of the computer-readable program instructions.

[0054] For the purposes of this embodiment, computer program product 10 is a related product that includes computer program 11.

[0055] For the purposes of this embodiment, computer-readable storage medium 20 is a tangible device capable of holding and storing a computer program 11. It can be any device capable of containing, storing, communicating, propagating, or transmitting the program 11 for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable storage medium 20 include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical encoding device, and any suitable combination thereof.

[0056] Computer device 30 can be, for example, a server, desktop computer, laptop computer, tablet computer, or smartphone. In some examples, computer device 30 can be a cloud computing node. Computer device 30 can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. Computer device 30 can be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can reside on local or remote computing system storage media, including storage devices.

[0057] Computer device 30 may include a processor 32 adapted to execute stored instructions and a memory 31 that provides temporary storage space for the operation of said instructions during operation. The processor 32 may be a single-core processor, a multi-core processor, a computing cluster, or any other configuration. The memory 31 may include random access memory (RAM), read-only memory, flash memory, or any other suitable storage system.

[0058] Computer device 30 may also include a network adapter / interface and an input / output (I / O) interface. The I / O interface allows external devices that can be connected to the computer device to input and output data. The network adapter / interface provides communication between the computer device and a network, typically represented as a communication network.

[0059] Therefore, those skilled in the art should recognize that although numerous exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications conforming to the principles of the present invention can be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Thus, the scope of the present invention should be understood and construed as covering all such other variations or modifications.

Claims

1. A database cluster streaming replication method based on shared storage, characterized in that, include: Monitor the continuous idle time of the main database node; In response to the continuous idle time exceeding a first threshold, the master database node generates auxiliary write-ahead log data at a preset period and sends it to the slave database. The auxiliary write-ahead log data is used to fill the leaf nodes corresponding to the master database node in the pair heap maintained by the slave database. In response to the continuous idle time exceeding a second threshold, the master database node stops generating the auxiliary write-ahead log data and sends a state transition log to the slave database, the state transition log being used to instruct the slave database to remove the leaf node from the pairing heap, wherein the second threshold is greater than the first threshold.

2. The database cluster streaming replication method based on shared storage according to claim 1, characterized in that, The steps of the master database node generating auxiliary write-ahead log data at a preset period and sending it to the slave database include: A timer is started when the continuous idle time exceeds a first threshold. In response to the timer reaching the preset period, the main database node is triggered to generate an auxiliary write-ahead log data containing a preset fill flag, wherein the preset fill flag is used to indicate that the auxiliary write-ahead log data is an empty log that does not contain business data. The generated auxiliary write-ahead log data is sent to the slave database, the active state of the leaf nodes in the paired heap is maintained based on the preset fill flag, and the timer is reset to start counting again.

3. The database cluster streaming replication method based on shared storage according to claim 2, characterized in that, Also includes: If the master database node receives a write-ahead log data request during the timer's countdown period, it is determined that the master database node has resumed its active state. In response to the primary database node regaining an active state, the timer is stopped, and the generation operation of the auxiliary write-ahead log data is terminated. The write-ahead log data corresponding to the write request is sent to the secondary database. The secondary database updates the log sequence number corresponding to the leaf node in the pairing heap based on the write-ahead log data, and adjusts the replay priority of the leaf node in the pairing heap according to the updated log sequence number. The replay priority is negatively correlated with the value of the log sequence number.

4. The database cluster streaming replication method based on shared storage according to claim 1, characterized in that, The steps of stopping the primary database node from generating secondary write-ahead log data and sending state transition logs to the secondary database include: When the timer is turned off, the main database node stops generating auxiliary write-ahead log data; A state transition log carrying a resource release identifier is generated based on the master database node; The state transition log is sent to the slave database, whereby the slave database removes the leaf node from the pairing heap based on the resource release identifier.

5. The database cluster streaming replication method based on shared storage according to claim 1, characterized in that, Also includes: If the master database node receives a write-ahead log data write request during the continuous idle time exceeding the first threshold but not exceeding the second threshold, it is determined that the master database node has resumed its active state. In response to the primary database node becoming active again, the primary database node stops generating auxiliary write-ahead log data and sends the write-ahead log data corresponding to the write request to the slave database.

6. A database cluster streaming replication method based on shared storage, characterized in that, include: Maintain a pairing heap, which includes at least one leaf node, and each leaf node corresponds to a master database node; Receive auxiliary write-ahead log data generated by the main database node at a preset period, wherein the auxiliary write-ahead log data is generated when the continuous idle time of the main database node exceeds a first threshold; Based on the auxiliary write-ahead log data, fill the leaf nodes corresponding to the primary database nodes in the paired heap; The system receives a state transition log sent by the master database node when the continuous idle time exceeds a second threshold. Based on the state transition log, the system removes the leaf node from the pairing heap, wherein the second threshold is greater than the first threshold.

7. The database cluster streaming replication method based on shared storage according to claim 6, characterized in that, Also includes: The system receives write-ahead log data sent by the master database node after it recovers to an active state. The write-ahead log data is generated when the master database node receives a write-ahead log data request during the timer's countdown period, or when the master database node receives a write-ahead log data request during the period when the continuous idle time exceeds a first threshold but does not exceed a second threshold. Based on the pre-written log data, update the log sequence number corresponding to the leaf node in the paired heap; Adjust the replay priority of the leaf node in the pairing heap based on the updated log sequence number.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the database cluster streaming replication method based on shared storage as described in any one of claims 1 to 7.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the shared storage-based database cluster streaming replication method as described in any one of claims 1 to 7.

10. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the steps of the shared storage-based database cluster streaming replication method according to any one of claims 1 to 7.

Images