A database high availability implementation method based on asynchronous replication

Abstract

The application discloses a kind of database high availability implementation methods based on asynchronous replication, which comprises: the main node and backup node for data storage are deployed in service end, to form the main backup collaborative state;When the client initiates data operation request to main node, client continuously monitors whether main node is available, to provide trigger mechanism for subsequent main backup node switching;In the case where main node is running normally, main node will be executed data operation according to data operation request Persistent record, and synchronously give backup node, to ensure the data consistency of backup node and main node;In the case where main node is unavailable, client triggers main backup node automatic switching, so that backup node takes over the data operation request of client, guarantee business continuity;After the fault repair of original main node, the synchronization management module of service end obtains the WAL log difference set between original main node and new main node, and synchronizes WAL log difference set, to restore data consistency.

Description

Technical Field

[0001] This invention relates to the field of computer database technology, and in particular to a method for achieving high availability of a database based on asynchronous replication. Background Technology

[0002] With the deepening application of information technology in the Industrial Internet of Things (IIoT) field, customers are placing higher demands on the availability and reliability of data storage systems. Some users, due to deployment environment limitations, can only configure two servers, yet still need to ensure both business continuity and data integrity. In general enterprise scenarios, traditional disaster recovery models also face prominent problems such as low resource utilization, slow fault recovery, and difficulty in guaranteeing data consistency. Summary of the Invention

[0003] This invention provides a method for achieving high availability of databases based on asynchronous replication, aiming to achieve high availability and data reliability with limited hardware resources, while also being adaptable to general disaster recovery scenarios.

[0004] This invention provides a method for achieving high availability of a database based on asynchronous replication, comprising:

[0005] Deploy primary and backup nodes for data storage on the server side to form a primary-backup collaborative state;

[0006] When the client initiates a data operation request to the master node, the client continuously monitors whether the master node is available, so as to provide a triggering mechanism for subsequent master-slave node switching;

[0007] When the master node is running normally, the master node will persistently record the data operations executed according to the data operation request and synchronize them to the backup node to ensure the data consistency between the backup node and the master node.

[0008] If the primary node becomes unavailable, the client triggers an automatic switchover between the primary and backup nodes, allowing the backup node to take over the client's data operation requests and ensure business continuity.

[0009] After the original master node is repaired, the server-side synchronization management module obtains the WAL log difference set between the original master node and the new master node, and synchronizes the WAL log difference set to restore data consistency.

[0010] Preferably, the client determines that the master node is unavailable when it identifies the master node as abnormal through node heartbeat detection or when it initiates a data operation request to the master node and detects the master node as abnormal.

[0011] Preferably, if the client fails to receive a heartbeat response from the master node three times consecutively during the process of sending a heartbeat packet to the master node, or if the heartbeat response carries an identifier indicating that the service is unavailable, or if the heartbeat response carries abnormal information about the master node components, then the client determines that the master node is abnormal.

[0012] Preferably, if the number of consecutive failed requests to the master node reaches a pre-configured maximum threshold or the connection times out during the process of the client initiating a data operation request to the master node, the master node is determined to be abnormal.

[0013] Preferably, the master node persistently records the data operations performed according to the data operation request and synchronizes them with the backup node to ensure data consistency between the backup node and the master node, including:

[0014] The master node uses the Write-Ahead Log (WAL) to persistently record the data operations, obtain new WAL logs, and report them to the synchronization management module.

[0015] After receiving the newly added WAL log synchronized by the master node through the synchronization management module, the backup node updates its local data through log replay to ensure consistency with the master node's data.

[0016] Preferably, the newly added WAL log has a data source tag for preventing circular synchronization, the data source tag including a synchronization data tag and a native data tag.

[0017] Preferably, after updating local data, the backup node filters WAL logs with synchronization data tags to avoid data cyclic synchronization.

[0018] Preferably, in the event that the primary node is unavailable, the client triggers an automatic switchover between the primary and backup nodes, allowing the backup node to take over the client's data operation requests, thus ensuring business continuity, including:

[0019] The client marks the master node as unavailable and disconnects from it, then establishes a connection with the backup node according to the pre-configured backup node address.

[0020] After establishing a connection with the backup node, the client marks the backup node as the new master node;

[0021] The client routes the data operation request to the new master node, which then processes the data operation request to ensure business continuity.

[0022] Preferably, the method further includes:

[0023] After the fault is repaired, the original master node is re-registered to the synchronization management module and the master-slave collaborative state is restored.

[0024] The client identifies the original master node as a backup node after it has been repaired by continuously sending heartbeat packets to the original master node.

[0025] Preferably, the server-side synchronization management module obtains the WAL log difference set between the original master node and the new master node, and synchronizes the WAL log difference set to restore data consistency, including:

[0026] The synchronization management module obtains the first WAL log difference set that the new master node was not synchronized to the original master node during the period when the original master node was unavailable, and the second WAL log difference set that the original master node was not synchronized to the new master node.

[0027] The original master node obtains the first WAL log difference set through the synchronization management module, and updates the local data of the original master node by replaying the logs in the first WAL log difference set.

[0028] The new master node obtains the second WAL log difference set through the synchronization management module, and updates the local data of the new master node by replaying the logs in the second WAL log difference set.

[0029] The technical effects of this invention are as follows:

[0030] 1. This invention solves the high availability problem of database services in resource-constrained scenarios (such as deployment of two servers), and enables automatic switching to the backup node when the master node fails, ensuring seamless business continuity.

[0031] 2. This invention achieves dual data storage through an efficient data replication mechanism, which improves data security and avoids the resource waste of traditional disaster recovery models;

[0032] 3. This invention breaks through the limitation of a single log stream being bound to a fixed machine, optimizes the log synchronization logic, and improves the overall system performance and fault tolerance.

[0033] 4. This invention solves the consistency problem in the data synchronization process, avoids cyclic synchronization, and can ensure stable system operation under abnormal scenarios.

[0034] 5. This invention is suitable for special fields such as the Industrial Internet of Things, and meets the requirements of business continuity and data security under limited hardware conditions. Attached Figure Description

[0035] Figure 1 This is a flowchart of the database high availability implementation method based on asynchronous replication of the present invention;

[0036] Figure 2 This is a schematic diagram of the database active-active system architecture of the present invention;

[0037] Figure 3 yes Figure 2 The diagram shows the structural composition of the system architecture. Detailed Implementation

[0038] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described below are only for illustration and explanation of the present invention and are not intended to limit the present invention.

[0039] See Figure 1 The method for achieving high availability of a database based on asynchronous replication according to the present invention includes:

[0040] Step S101: Deploy the primary and backup nodes for data storage on the server side to form a primary-backup collaborative state.

[0041] Step S102: When the client initiates a data operation request to the master node, the client continuously monitors whether the master node is available, so as to provide a triggering mechanism for subsequent master-slave node switching.

[0042] The client determines that the master node is unavailable when it identifies an anomaly in the master node through node heartbeat detection or when it initiates a data operation request to the master node and detects an anomaly in the master node.

[0043] If the client fails to receive a heartbeat response from the master node three times consecutively during the process of sending a heartbeat packet to the master node, or if the heartbeat response carries an identifier indicating that the service is unavailable, or if the heartbeat response carries abnormal information about the master node components, then the master node is determined to be abnormal.

[0044] The master node collects the operating metrics of the core components in real time, including the component process survival status, CPU or memory load value, and data synchronization delay time. When a component process exits, the load value exceeds the load threshold, or the data synchronization delay time exceeds the delay setting value, the master node generates the component abnormal information and carries it in the heartbeat response.

[0045] If, during the process of the client initiating a data operation request to the master node, the number of consecutive failed requests reaches a pre-configured maximum threshold or the connection times out, the master node is determined to be abnormal.

[0046] As a supplement, the client's configuration module can determine the fault type of the master node through cross-validation of information.

[0047] Step S103: If the master node is running normally, the master node will persist the data operation performed according to the data operation request and synchronize it to the backup node to ensure the data consistency between the backup node and the master node.

[0048] When the master node is running normally, it uses Write-Ahead Logs (WAL) to persistently record the data operations, obtain new WAL logs, and report them to the synchronization management module. After receiving the new WAL logs synchronized by the master node through the synchronization management module, the backup node updates its local data through log replay to ensure consistency with the master node's data.

[0049] As a supplement, the newly added WAL logs may include data source tags for preventing circular synchronization, including synchronization data tags and native data tags. After updating local data, the backup node can filter WAL logs with synchronization data tags to avoid circular data synchronization.

[0050] Step S104: If the primary node is unavailable, the client triggers an automatic switchover between the primary and backup nodes, allowing the backup node to take over the client's data operation requests and ensure business continuity.

[0051] If the primary node is unavailable, the client marks the primary node as unavailable and disconnects from it. Then, it establishes a connection with the backup node based on the pre-configured backup node address. After the client establishes a connection with the backup node, it marks the backup node as the new primary node and routes the data operation request to the new primary node. The new primary node then processes the data operation request to ensure business continuity.

[0052] Step S105: After the original master node is repaired, the synchronization management module of the server obtains the WAL log difference set between the original master node and the new master node, and synchronizes the WAL log difference set to restore data consistency.

[0053] After the original master node is repaired, it re-registers with the synchronization management module to restore the master-slave collaborative state. The client identifies the original master node as a backup node by continuously sending heartbeat packets to it.

[0054] The synchronization management module obtains a first WAL log difference set that the new master node did not synchronize with the original master node during the period when the original master node was unavailable, and a second WAL log difference set that the original master node did not synchronize with the new master node. The original master node obtains the first WAL log difference set through the synchronization management module and updates its local data by replaying the logs in the first WAL log difference set. The new master node obtains the second WAL log difference set through the synchronization management module and updates its local data by replaying the logs in the second WAL log difference set.

[0055] The following section provides a detailed explanation of the overall system architecture and process mechanism, system operation scenarios and processing logic.

[0056] I. Overall System Architecture

[0057] This invention constructs a dual-active database system, see [link / reference]. Figure 2 It consists of four core components: master node, standby node, client, and synchronization management module. Each component is modularly designed to achieve collaborative functions such as master-standby node data synchronization and automatic failover. The main functions of each component are as follows:

[0058] 1. See Figure 3 The core modules of the primary / backup nodes include a data storage module, a log service module, a network communication module, and a system monitoring module.

[0059] (1) Data storage module: The distributed storage logic is used to realize the persistence of business data and logs, support data redundancy configuration, ensure data storage reliability, and avoid data loss caused by single point of failure.

[0060] (2) Log service module: Based on the write-ahead log (WAL) mechanism, persistent recording of data operations is realized. The logs generated by the master node are synchronized to the standby node in real time. The standby node completes the data update through log replay to ensure the consistency of master and standby data. At the same time, data source markers are added to the logs to avoid loops during data synchronization.

[0061] (3) Network communication module: Supports common network protocols such as WebSocket, and is responsible for data transmission between nodes and clients, and between nodes and synchronization management modules, ensuring the stability and efficiency of data transmission and adapting to network environments in different scenarios.

[0062] (4) System monitoring module: Real-time monitoring of key indicators such as node heartbeat, network connection status, component status, and data synchronization progress. When anomalies such as node crash or synchronization delay exceeding the threshold are detected, an alarm mechanism is triggered and a preset fault recovery strategy is executed to ensure that system anomalies can be detected and handled in a timely manner.

[0063] 2. The client uses a failover mechanism, see [link / reference] Figure 3 The client includes a client driver module and a client configuration module, which are used to establish connections with the primary / backup nodes and execute business requests. It has a built-in retry mechanism to automatically switch nodes in case of failure, configure parameters such as primary / backup addresses and retry counts, use a heartbeat check mechanism to achieve fault recovery and service discovery, perform dynamic node marking and automatic switching between primary and backup nodes, and achieve load balancing.

[0064] (1) The client-side driver module has built-in retry and automatic failover logic. When a request to the primary node fails, a retry is automatically triggered. Once the maximum retry threshold is reached, the primary node is deemed unavailable, and the client switches to the backup node to initiate a request to the backup node. After a successful switchover, the client automatically marks the backup node as the new primary node, and subsequent business requests are directly routed to the new primary node. The entire process requires no manual intervention, achieving seamless business failover.

[0065] (2) The client configuration module supports dynamically setting the primary and backup node addresses, retry parameters, and other information through configuration files. These settings take effect without restarting the client, reducing operational complexity. The retry parameters include the initial number of retries, the retry interval, and the maximum retry threshold.

[0066] (3) The client registers and discovers nodes through the heartbeat mechanism, thereby enabling nodes to regain awareness and switch states.

[0067] When a node starts up, it automatically completes registration and reports information such as node address and service status.

[0068] When the primary node fails, the node status is updated in real time, marking the primary node as "unavailable" and the backup node as "available", providing a status basis for client switching;

[0069] After a node recovers, it automatically re-registers, synchronizes with the current system status, and quickly integrates into the active-active architecture, avoiding recovery delays caused by manual configuration.

[0070] (4) The load balancing module performs load balancing and resource optimization.

[0071] During normal operation, the master node carries the full business load, while the backup node only synchronizes data and does not participate in business processing, thus avoiding resource waste.

[0072] When the primary node is overloaded, the client dynamically adjusts the routing of some read requests to the backup node to relieve the pressure on the primary node.

[0073] After a node recovers, requests are gradually switched back to the master node to avoid system instability caused by sudden load surges and ensure optimal resource utilization.

[0074] 3. See Figure 3The synchronization management module is configured for active-active operation. It primarily monitors and manages the status of active-active nodes based on a configuration scheme (distributed coordination technology), automatically detects differences between the primary and backup nodes, and automatically merges these differences based on a distributed log merging strategy to ensure data consistency. When the primary node recovers, it updates the node status in real time and synchronizes incremental data.

[0075] II. System Process Mechanism

[0076] The active-active database system achieves collaborative operation and business continuity assurance of primary and backup nodes through a closed-loop process of "data synchronization - fault detection - automatic switchover - state recovery". The four stages are closely linked and progressively advance, as detailed below:

[0077] 1. Data Synchronization: Building the foundation for master-slave data consistency

[0078] Data synchronization is the core underlying support of a dual-active system. Through real-time and efficient log and data replication, it ensures that the data of the standby node and the primary node are consistent, providing a "usable data copy" for failover. The specific process and mechanism are as follows:

[0079] (1) Master Node Data Persistence: After receiving read and write requests from clients, the master node first persists data operation records to a local log file through the Write-Ahead Log (WAL) mechanism. Simultaneously, during data synchronization, the synchronization component adds a unique marker to the WAL log when writing data. When reading the log for synchronization, it automatically filters data with this unique marker to prevent circular synchronization between the master and backup nodes, thus implementing a log-marking anti-circular synchronization mechanism. For example, adding a data source marker to the log (such as marking synchronized data as SOURCE_TAOSX and raw data as SOURCE_NULL) provides a basis for subsequent anti-circular synchronization.

[0080] (2) Data distribution of the synchronization management module: The synchronization component (TDengine's taosX tool) is used to monitor changes in the local WAL log in real time, and the newly added log data is transmitted to the backup node through the network. During the transmission process, streaming transmission optimization is adopted to reduce the impact of network latency on synchronization efficiency.

[0081] (3) Backup node log replay: After receiving the log synchronized by the master node, the backup node applies the data operation to its own data storage module through the log replay mechanism to complete the data update; at the same time, the subscription interface of the backup node will automatically filter logs with "synchronization source mark" to avoid sending the synchronized data back to the master node and forming a circular synchronization.

[0082] (4) Data difference synchronization: After the master node recovers from a crash, the synchronization management module automatically detects the data difference between the master and backup nodes (i.e., the data that was not synchronized to the backup node during the master node's crash, and the data added by the backup node during the master node's crash), and prioritizes synchronizing the difference data until the master and backup data are completely consistent.

[0083] (5) Metadata Synchronization Guarantee and Optimization: For metadata operations such as database creation, table creation, and table structure modification, real-time synchronization is achieved through a dedicated synchronization channel. A consistency protocol is adopted to ensure the consistency of metadata between primary and standby nodes, avoiding business anomalies caused by metadata non-synchronization. This protocol can ensure the consistency of metadata between primary and standby nodes through WAL log synchronization, avoiding business anomalies such as business write failures caused by metadata non-synchronization.

[0084] 2. Fault Detection: Accurately detects abnormal system states.

[0085] The client proactively detects anomalies in the primary node, backup node, and components, providing a triggering mechanism for subsequent failover. The core implementation is as follows:

[0086] (1) Node heartbeat detection: The client and the primary and backup nodes send heartbeat packets at fixed intervals (e.g., 30 seconds), which contain key information such as node running status, data synchronization progress, and component health. If the client does not receive a heartbeat response from the other end for three consecutive times, or if the heartbeat packet carries a "service unavailable" flag, the node is determined to be abnormal.

[0087] Among them, the primary and backup nodes monitor the status of components. Specifically, the system monitoring module of the primary and backup nodes collects the operating indicators of the core components of the primary and backup nodes (such as taosd, taosAdapter, and synchronization tools) in real time, including process survival status, CPU / memory load, network connection status, data synchronization latency, etc. When a component process exits, the load exceeds the threshold (such as CPU utilization > 80%), or the synchronization latency exceeds the set value (such as latency > 10 seconds), an abnormal alarm is triggered. As a supplement, component abnormal information can be generated and written into the heartbeat response and sent to the client so that the client can determine the node abnormality.

[0088] (2) Client request feedback: The client has a built-in retry mechanism and timeout detection logic. When the client driver module sends a request to the master node, if the number of consecutive failed requests reaches the configured threshold (e.g., 3 times) or the connection times out (e.g., no response for 500ms), it determines that the master node service is unavailable and reports the fault information to the client configuration module.

[0089] (4) Client configuration module summary verification: After receiving the abnormal signals from the node heartbeat, component monitoring and driver module, the configuration module performs cross-verification (such as receiving both the loss of the master node heartbeat and the failure of the client request at the same time), eliminates false alarm scenarios such as network jitter, and finally confirms the fault type (such as master node crash, component abnormality, synchronization interruption).

[0090] 3. Automatic switching: Enables seamless business takeover.

[0091] When the client confirms that the primary node is unavailable, the failover process is automatically triggered, seamlessly routing business requests to the backup node to minimize the impact of business interruption. The specific steps are as follows:

[0092] (1) Fault status marking and notification: The client configuration module marks the master node as "unavailable" and broadcasts a notification to the client driver module that "the master node is faulty and the switch to the backup node is required".

[0093] (2) Client routing switch: After receiving the status notification, the client driver module interrupts the connection with the original master node and initiates a new connection according to the configured backup node address. After the connection is successful, the temporary master node is marked as the "new master node". All subsequent business requests (read, write, query) are routed to the new master node. The whole process does not require manual intervention and the business is unaware of it.

[0094] Automatic failover limitation: Because some client requests depend on the context, not all requests can be seamlessly handled. In such cases, the request will return an error, requiring the client application to retry from the initial request in the context (including STMT initialization, query entry requests, etc.).

[0095] 4. State Recovery: Rebuilding the stability of the active-active architecture

[0096] Once the original faulty node (original master node) is repaired, the synchronization management module re-integrates it into the dual-active architecture through an automated process, restoring the "master-slave collaboration" state and ensuring long-term system stability. The specific process is as follows:

[0097] (1) Restart and recovery of faulty nodes: After the original master node is repaired, the core components (taosd, synchronization tools, etc.) are automatically started, and the node is re-registered with the synchronization management module to report its own status (such as data integrity and component health) and the unsynchronized logs during the fault period (data difference set(AB)).

[0098] (2) Data difference set synchronization: The synchronization management module first synchronizes the data added during the downtime from the new master node (difference set (B1-B)) to the original master node, and then synchronizes the logs that were not synchronized on the original master node (difference set (AB)) to the new master node.

[0099] Role adaptation and architecture reconstruction: After data synchronization is complete, the master node needs to be switched manually (switching can be completed by restarting the application).

[0100] III. System Operation Scenarios and Processing Logic

[0101] 1. Normal operating scenario

[0102] Business request routing: All read and write requests from the client are sent to the master node. The master node processes the requests in real time and writes the data to local storage. At the same time, the data and logs are synchronized to the backup node through the synchronization tool.

[0103] Backup node status: The backup node does not directly process business requests, but only receives data and logs synchronized from the master node, and updates local data through log replay to ensure real-time consistency with the master node's data;

[0104] System monitoring: The system monitoring module of the primary and backup nodes continuously collects indicators such as CPU utilization, memory usage, and synchronization latency of the primary and backup nodes. There are no alarms when the indicators are normal, and the synchronization progress is updated in real time.

[0105] 2. Master node failure scenario

[0106] Fault detection: When a client fails to send a request to the master node, a retry mechanism is triggered. After the retry reaches the maximum threshold, the client driver module obtains the node status through the client configuration module and confirms that the master node is unavailable.

[0107] Automatic failover: The client driver module immediately switches to the standby node to initiate the request. After a successful failover, the standby node is marked as the new primary node, and subsequent business requests are directly routed to the new primary node.

[0108] Data consistency guarantee: If there is data that has not been synchronized to the standby node when the master node fails, the system will automatically synchronize the data difference set after the master node recovers to ensure that the master and standby data are eventually consistent.

[0109] 3. Master node recovery scenario

[0110] Node registration and status synchronization: After the master node recovers, it automatically re-registers with the synchronization management module to obtain the current system status (i.e., the original backup node has become the new master node).

[0111] Data difference synchronization: The synchronization management module uses the synchronization tools of the primary and backup nodes to detect the data difference between the primary and backup nodes and the new primary node (including data that was not synchronized during the primary node's downtime and data added by the new primary node during the primary node's downtime), and prioritizes synchronizing the difference data;

[0112] Role positioning: After data synchronization is completed, the original master node automatically switches to the backup node, synchronizes the data of the new master node in real time, and restores the normal operation architecture of the dual-active system.

[0113] 4. Abnormal Synchronization Delay Scenarios

[0114] Anomaly Detection: The system monitoring module detects that the data synchronization delay between the primary and backup nodes exceeds a preset threshold (e.g., a delay of more than 10 seconds), triggers an alarm, and notifies the operation and maintenance personnel.

[0115] Temporary handling: If the delay duration is short (not exceeding the switchover interval threshold), the client automatically restricts unnecessary read requests to the backup node, prioritizes data synchronization bandwidth, and accelerates the synchronization progress;

[0116] Switchover process: If the delay duration exceeds the switchover interval threshold, the client will automatically switch the backup node to the primary node, and the original primary node will become the backup node and focus on data synchronization. After the synchronization is completed, it will resume its normal role to avoid affecting business due to synchronization delay.

[0117] The present invention has the following technical effects:

[0118] 1. In the traditional single-data center disaster recovery model, the long-term idleness of the standby machine leads to serious waste of resources. Moreover, the standby center is slow to start and prone to unknown failures when a disaster occurs. In contrast, the present invention adopts the Active-Standby mode, where the standby node synchronizes data in real time instead of being idle for a long time. Combined with load balancing to dynamically adjust request routing, the resource utilization rate is significantly improved compared with the traditional disaster recovery model. In addition, when the master node fails, the client automatically completes the switch in seconds, and the standby node seamlessly takes over the business. This avoids the business interruption caused by slow startup and easy failure in the traditional disaster recovery model, ensuring business continuity and greatly improving system availability.

[0119] 2. Distributed state machine replication technology often restricts log streams to fixed machines, making system performance constrained by single-machine hardware bottlenecks and resulting in weak fault tolerance. However, this invention breaks through the limitation of single log streams being bound to fixed machines, optimizes log synchronization logic, and improves the overall system performance and fault tolerance.

[0120] 3. Multi-replica architecture and traditional database master-slave replication solutions have shortcomings in data synchronization consistency and read-write separation effectiveness in abnormal scenarios. They are difficult to balance throughput and high availability and cannot adapt to the core needs of resource-constrained scenarios. This invention is a lightweight active-active solution for time-series database scenarios. By simplifying the node architecture, client drivers and WAL synchronization, it achieves high availability in a resource-constrained environment of two servers.

[0121] 4. In terms of architectural flexibility and scenario adaptability, this invention is suitable for resource-constrained scenarios where only two servers can be deployed (such as industrial control and shipboard monitoring), and can also be applied to general database cluster disaster recovery scenarios without requiring significant architectural adjustments for different scenarios.

[0122] 5. Regarding data security and consistency, this invention ensures that data is not lost or duplicated through mechanisms such as dual data storage, log marking to prevent circularity, and data difference synchronization, thus guaranteeing data consistency and reliability.

[0123] 6. In terms of operational complexity, this invention supports functions such as automatic node registration and discovery, dynamic configuration activation, and automatic alarm and recovery for anomalies, reducing manual intervention and lowering operational costs and difficulty.

[0124] Although the present invention has been described in detail above, it is not limited thereto, and those skilled in the art can make various modifications based on the principles of the present invention. Therefore, all modifications made in accordance with the principles of the present invention should be understood to fall within the protection scope of the present invention.

Claims

1. A method for achieving high availability of a database based on asynchronous replication, characterized in that, include: Deploy primary and backup nodes on the server side for data storage to form a primary-backup collaborative state; When the client initiates a data operation request to the master node, the client continuously monitors whether the master node is available, so as to provide a triggering mechanism for subsequent master-slave node switching; When the master node is running normally, the master node will persistently record the data operations executed according to the data operation request and synchronize them to the backup node to ensure the data consistency between the backup node and the master node. If the primary node becomes unavailable, the client triggers an automatic switchover between the primary and backup nodes, allowing the backup node to take over the client's data operation requests and ensure business continuity. After the original master node is repaired, the server-side synchronization management module obtains the WAL log difference set between the original master node and the new master node, and synchronizes the WAL log difference set to restore data consistency.

2. The method according to claim 1, characterized in that, The client determines that the master node is unavailable when it identifies an anomaly in the master node through node heartbeat detection or when it initiates a data operation request to the master node and detects an anomaly in the master node.

3. The method according to claim 2, characterized in that, If the client fails to receive a heartbeat response from the master node three times consecutively during the process of sending a heartbeat packet to the master node, or if the heartbeat response carries an identifier indicating that the service is unavailable, or if the heartbeat response carries abnormal information about the master node components, then the client determines that the master node is abnormal.

4. The method according to claim 2, characterized in that, If the client initiates a data operation request to the master node and the number of consecutive failed requests reaches a pre-configured maximum threshold or the connection times out, the master node is determined to be abnormal.

5. The method according to claim 1, characterized in that, The master node will persistently record the data operations performed according to the data operation request and synchronize them with the backup node to ensure data consistency between the backup node and the master node, including: The master node uses the Write-Ahead Log (WAL) to persistently record the data operations, obtain new WAL logs, and report them to the synchronization management module. After receiving the newly added WAL log synchronized by the master node through the synchronization management module, the backup node updates its local data through log replay to ensure consistency with the master node's data.

6. The method according to claim 5, characterized in that, The newly added WAL logs are marked with data source tags to prevent circular synchronization. These data source tags include synchronization data tags and native data tags.

7. The method according to claim 6, characterized in that, After updating local data, the backup node filters WAL logs marked with synchronization data to avoid data cyclic synchronization.

8. The method according to claim 1, characterized in that, In the event that the primary node is unavailable, the client triggers an automatic switchover between the primary and backup nodes, allowing the backup node to take over the client's data operation requests and ensuring business continuity, including: The client marks the master node as unavailable and disconnects from it, then establishes a connection with the backup node according to the pre-configured backup node address. After establishing a connection with the backup node, the client marks the backup node as the new master node; The client routes the data operation request to the new master node, which then processes the data operation request to ensure business continuity.

9. The method according to claim 1, characterized in that, The method further includes: After the fault is repaired, the original master node is re-registered to the synchronization management module and the master-slave collaborative state is restored. The client identifies the original master node as a backup node after it has been repaired by continuously sending heartbeat packets to the original master node.

10. The method according to claim 9, characterized in that, The server-side synchronization management module obtains the WAL log difference set between the original master node and the new master node, and synchronizes the WAL log difference set to restore data consistency, including: The synchronization management module obtains the first WAL log difference set that the new master node was not synchronized to the original master node during the period when the original master node was unavailable, and the second WAL log difference set that the original master node was not synchronized to the new master node. The original master node obtains the first WAL log difference set through the synchronization management module, and updates the local data of the original master node by replaying the logs in the first WAL log difference set. The new master node obtains the second WAL log difference set through the synchronization management module, and updates the local data of the new master node by replaying the logs in the second WAL log difference set.

Images