A method and device for automatic disaster recovery drill and intelligent rollback of a primary-standby cluster database supporting breakpoint resuming

Abstract

This application discloses a method and apparatus for automated disaster recovery drills and intelligent rollback of primary and standby database clusters that supports breakpoint resumption. This method achieves fully automated control of the disaster recovery drill process through five core mechanisms: atomic step management, two-stage data barriers, hierarchical intelligent rollback, global distributed locks, and progress visualization. This invention atomically decomposes the disaster recovery drill process and persistently stores progress snapshots, supporting breakpoint resumption after interruption; it ensures strong consistency between primary and standby data by inserting global transaction barrier markers and unifying the standby cluster log replay points; it divides risk intervals according to the drill progress and performs hierarchical differentiated rollbacks to avoid topology errors; it uses distributed locks to achieve global concurrency mutual exclusion, preventing split-brain and synchronization disorders; and it quantitatively displays the drill progress in real time, improving observability. This invention effectively solves the problems of poor process fault tolerance, weak data consistency, coarse rollback strategies, low concurrency security, and insufficient observability in existing technologies.

Description

Technical Field

[0001] This application relates to the field of database disaster recovery drill technology, specifically to a method, apparatus, computer-readable storage medium, and electronic device for automated disaster recovery drills and intelligent rollback of master-slave cluster databases that supports breakpoint resume operation. Background Technology

[0002] As the core infrastructure of business systems, the disaster recovery capability of databases directly determines business continuity and data security. Enterprise-level databases commonly adopt a primary-standby cluster architecture, where the primary cluster handles online read and write operations, while the standby cluster maintains data redundancy through real-time log synchronization to quickly take over in the event of a primary cluster failure. To ensure the effectiveness of this disaster recovery mechanism, industry standards require regular failover drills.

[0003] However, existing disaster recovery drill solutions mostly rely on manual scripts or simple one-click switching, which have the following technical shortcomings in practical applications:

[0004] (1) Insufficient process fault tolerance: Existing drills mostly use fixed serial execution logic. Any abnormality in any link will cause the entire process to fail. Furthermore, the previously completed operation state cannot be retained and must be re-executed from the initial state. The degree of automation is low and the efficiency of drills is limited.

[0005] (2) Passive data consistency guarantee: Existing solutions mainly rely on the database’s native log synchronization mechanism, which lacks the ability to actively lock transaction boundaries. During the switching process, problems such as incomplete log playback, half-transaction residues, or data gaps are likely to occur, resulting in uncontrollable recovery point objectives (RPO).

[0006] (3) The rollback strategy is crude: the existing rollback mechanism usually only supports two modes: full rollback or no rollback. It cannot perform graded and refined rollback operations based on the actual execution progress of the exercise and the real-time status of the cluster, which can easily cause cluster topology disorder or additional performance loss.

[0007] (4) Lack of concurrency control and progress awareness: The exercise operation lacks an effective concurrency control mechanism, and there is a risk of split brain or data synchronization disorder when triggered in parallel; at the same time, the operation and maintenance personnel cannot obtain the global execution status and progress information of the exercise in real time, resulting in insufficient observability.

[0008] In summary, existing primary and backup cluster database disaster recovery drill technologies have significant shortcomings in terms of process fault tolerance, data consistency assurance, rollback strategy refinement, concurrency security, and observability. There is an urgent need for an automated disaster recovery drill solution that supports breakpoint resumption, strong consistency barrier locking, hierarchical intelligent rollback, and global mutual exclusion control. Summary of the Invention

[0009] To overcome the aforementioned shortcomings of existing technologies, this application proposes a novel method for automated disaster recovery drills and intelligent rollback of master-slave cluster databases that supports breakpoint resumption. This invention aims to achieve breakpoint resumption during drills, strong data consistency, and full-process controllability through a closed-loop mechanism that includes atomic step management, two-stage data barriers, hierarchical intelligent rollback, global mutual exclusion, and progress visualization.

[0010] To achieve the above objectives, this invention constructs a four-layer closed-loop control system consisting of a "data status acquisition layer, consistency decision evaluation layer, intelligent security scheduling layer, and exercise execution and rollback layer." The system achieves full-process control from cluster status perception, data consistency determination, intelligent decision-making, automatic exercise execution to lossless rollback through four-layer collaboration and five core mechanisms (atomic step management, dual-stage data barrier, hierarchical intelligent rollback, global distributed lock, and progress visualization).

[0011] Specifically, this application provides the following technical solutions:

[0012] The first aspect of this application provides a method for automated disaster recovery drills and intelligent rollback of master-slave cluster databases that supports breakpoint resumption, such as... Figure 1 As shown, the method includes the following steps:

[0013] S1. Data Status Acquisition: The disaster recovery drill process of the primary and backup cluster databases is divided into multiple atomic steps, and a unique step identifier is assigned to each atomic step; during the execution of the atomic steps, the running status information is collected and persistently stored to form a drill progress snapshot; when the drill process is interrupted and restarted, the execution is resumed from the breakpoint based on the drill progress snapshot.

[0014] S2. Consistency Decision Assessment: Before the drill switchover, freeze the incremental log synchronization of the standby cluster and insert a globally unified transaction barrier marker at the end position of the log replay; control each node of the standby cluster to force the replay of the existing logs until the barrier marker position is reached, lock the replay status of each node to unify the data view inside the standby cluster; perform a global consistency check on the primary and standby clusters, and enter the switchover process after the preset consistency conditions are met.

[0015] S3. Intelligent Safety Scheduling: Set rollback thresholds based on the exercise execution progress, dividing the exercise process into a pre-low-risk interval and a post-high-risk interval; when an anomaly occurs in the low-risk interval, perform a full reverse rollback; when an anomaly occurs in the high-risk interval, make a comprehensive judgment based on cluster status indicators, and trigger a precise rollback only when a high-risk state is detected; use a distributed lock mechanism to perform global concurrent mutual exclusion control on disaster recovery exercise operations.

[0016] S4. Drill Execution and Rollback: Sequentially execute cluster role switching and business traffic scheduling; real-time statistics and external display of atomic step execution progress; after normal completion of the drill or abnormal rollback, automatically clean up temporary configurations, release distributed locks, restore the original cluster topology and data synchronization links.

[0017] Furthermore, in the method of this application, the atomic steps in step S1 include: cluster state pre-inspection, distributed lock verification, synchronization state detection, data barrier marking, cluster role switching, business traffic switching, data consistency verification, abnormal state determination, topology rollback and resource cleanup.

[0018] Furthermore, in the method of this application, the step S1 of collecting and persistently storing running status information during the execution of atomic steps includes: collecting running status information before, during, and after the execution of each atomic step, and persistently storing the step status, timestamp, execution log, error code, and instantaneous cluster parameters to form a structured exercise progress snapshot.

[0019] Furthermore, in the method of this application, the dimensions of the global consistency verification in step S2 include: log file continuity, barrier site matching degree, cluster transaction status, and synchronization delay value;

[0020] The preset consistency conditions include: all backup nodes have consistent barrier points, the primary-backup synchronization delay does not exceed a preset threshold, and there are no uncommitted residual transactions.

[0021] Furthermore, in the method of this application, the rollback threshold in step S3 is set proportionally based on the total number of atomic steps.

[0022] Furthermore, in the method of this application, the cluster status indicators mentioned in step S3 include: master / slave role status, replication link connectivity, node heartbeat liveness status, and dual-master conflict risk status.

[0023] The distributed lock mechanism includes: generating a unique global identifier for each exercise task and creating exclusive lock resources; scanning the disaster recovery-related tasks running in the cluster before the exercise starts, and intercepting the exercise if a running disaster recovery task is detected; configuring an automatic lock recycling mechanism to automatically release lock resources in the event of normal task completion, active termination, or abnormal crash.

[0024] Furthermore, in the method of this application, the statistical formula for the execution progress of the atomic steps in step S4 is: progress value = number of atomic steps successfully executed / total number of atomic steps × 100%.

[0025] Furthermore, in the method of this application, the external display described in step S4 is implemented through a standardized HTTP progress query interface.

[0026] The second aspect of this application provides an automated disaster recovery drill and intelligent rollback device for primary and standby database clusters that supports breakpoint resumption. When the device is running, it implements the steps of the aforementioned automated disaster recovery drill and intelligent rollback method for primary and standby database clusters that supports breakpoint resumption, such as... Figure 2 As shown, the device includes:

[0027] The data status acquisition module is used to break down the disaster recovery drill process of the primary and backup cluster databases into multiple atomic steps, assign a unique step identifier to each atomic step, collect running status information during the execution of atomic steps and persist it to form a drill progress snapshot, and when the drill process is interrupted and restarted, it resumes execution from the breakpoint based on the drill progress snapshot.

[0028] The consistency decision evaluation module is used to freeze the incremental log synchronization of the standby cluster before the drill switch, insert a globally unified transaction barrier mark at the end position of the log replay; control each node of the standby cluster to force the replay of the existing logs until the barrier mark position is reached, lock the replay status of each node to unify the data view inside the standby cluster; perform a global consistency check on the primary and standby clusters, and enter the switch process after the preset consistency conditions are met.

[0029] The intelligent safety scheduling module is used to set rollback thresholds based on the exercise execution progress, dividing the exercise process into a pre-low-risk interval and a post-high-risk interval; when an anomaly occurs in the low-risk interval, a full reverse rollback is performed; when an anomaly occurs in the high-risk interval, a comprehensive judgment is made based on cluster status indicators, and a precise rollback is triggered only when a high-risk state is detected; and a distributed lock mechanism is used to perform global concurrency and mutual exclusion control on disaster recovery exercise operations.

[0030] The exercise execution and rollback module is used to sequentially execute cluster role switching and business traffic scheduling; to statistically analyze and display the execution progress of atomic steps in real time; and to automatically clean up temporary configurations, release distributed locks, and restore the original cluster topology and data synchronization links after the exercise ends normally or rolls back abnormally.

[0031] A third aspect of this application provides an electronic device, including: a memory and a processor;

[0032] Memory: Used to store computer programs;

[0033] Processor: Used to execute the computer program to implement the steps of the aforementioned method for automated disaster recovery drills and intelligent rollback of master-slave cluster databases that supports breakpoint resumption.

[0034] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the aforementioned method for automated disaster recovery drills and intelligent rollback of a master-slave cluster database that supports breakpoint resume.

[0035] In summary, this invention, through its five core mechanisms—atomic step management, two-stage data barriers, hierarchical intelligent rollback, global distributed locks, and progress visualization—offers the following technical advantages over existing technologies:

[0036] (1) Improved process fault tolerance and automation: By splitting atomic steps and persisting state, automatic breakpoint resume after exercise interruption is achieved, avoiding the overall rerun of the traditional fixed serial process in case of anomalies, and significantly improving the fault tolerance and automation level of the exercise.

[0037] (2) Strong data consistency guarantee: The active two-stage barrier alignment mechanism is adopted to forcibly lock the transaction boundary of the master and backup clusters, ensuring complete log playback, no half-transaction residues and data gaps, so that the recovery point objective (RPO) is controllable and predictable.

[0038] (3) Refined rollback strategy and improved security: Based on the risk interval division, the hierarchical intelligent rollback model implements differentiated rollback according to the exercise progress and cluster status, avoiding topology disorder and performance loss caused by full rollback or no rollback.

[0039] (4) Enhanced concurrency safety and system stability: Global mutual exclusion of disaster recovery operations is achieved through global distributed locks, eliminating the risks of split brain, synchronization failure and role conflict, and ensuring the stable operation of the cluster in a multi-tasking environment.

[0040] (5) Observability and Operation and Maintenance Efficiency Optimization: Provides progress quantitative calculation and standardized query interface, enabling operation and maintenance personnel to perceive the exercise status in real time, replacing the traditional log traceability method, and realizing full-process controllability and unattended closed-loop management.

[0041] Other features and advantages of this application will be set forth in detail in the following description, or will become apparent through the implementation of the relevant technical solutions of this application. The objectives and other advantages of this application can be achieved through the technical features and means explicitly pointed out in the description, claims, and drawings, and will be obtained through the implementation of these technical contents. Attached Figure Description

[0042] To more clearly illustrate the technical solution of this application, the accompanying drawings involved in the description of this invention will be briefly introduced below. It should be noted that the drawings only show some embodiments of the invention. For those skilled in the art, other related drawings can be derived from these drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating the overall implementation process of the automated disaster recovery drill and intelligent rollback method for primary and standby cluster databases that supports breakpoint resume operation in this application.

[0044] Figure 2 This is a structural diagram of the automated disaster recovery drill and intelligent rollback device for primary and standby cluster databases that supports breakpoint resume operation, as described in this application.

[0045] Figure 3 This is a schematic diagram of the system architecture for the automated disaster recovery switchover drill and state rollback method under the primary / standby cluster architecture of this application.

[0046] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.

[0048] In this document, the term "including" and any variations thereof (such as "including," "including," etc.) are open-ended expressions and should be understood as "including but not limited to," meaning that the listed content is not an exhaustive list and may include other content not explicitly mentioned. The term "based on" should be understood as "at least partially based on," meaning that the basis or condition referred to may not be the only factor and may involve other relevant factors.

[0049] In this application, the terms "a" and "a plurality of" are used to modify related elements or features, and their expression is illustrative rather than restrictive. Unless otherwise expressly stated in the context, "a" should be understood as "at least one," and "a plurality of" should be understood as "at least two." Those skilled in the art should reasonably interpret these terms based on the semantic and logical relationships of the context to ensure that they cover the possibility of "one or more."

[0050] Example: A method and apparatus for automated disaster recovery drills and intelligent rollback of master-slave cluster databases that supports breakpoint resumption

[0051] Figure 3 The figure shows a schematic diagram of the system architecture of the automated disaster recovery switching exercise and status rollback method under the primary and backup cluster architecture of the present invention. As shown in the figure, the overall solution adopts a top-down hierarchical design, which includes four core layers: data status acquisition layer, consistency decision evaluation layer, intelligent security scheduling layer, and exercise execution and visualization rollback layer. Each layer is linked sequentially through control flow and data flow to form a complete closed-loop management logic.

[0052] 1. Data Status Acquisition Layer: Atomic Step Management

[0053] This layer is responsible for the fine-grained control and state persistence of the exercise process. First, the exercise process is broken down into atomic steps, and a unique identifier is assigned to each step; second, the running status of each step is collected in real time and the data is persistently stored; finally, relying on the progress snapshot mechanism, breakpoint resume is realized after the exercise is interrupted.

[0054] This layer is the system's foundational perception layer, responsible for atomically decomposing and persistently storing the state of the entire disaster recovery exercise process, solving the problems of traditional exercises lacking progress tracking and requiring re-running after interruption. This invention breaks down the complete exercise process into nine fixed and ordered minimum execution units, including: cluster state pre-inspection, distributed lock verification, synchronization state detection, data barrier marking, cluster role switching, business traffic switching, data consistency verification, abnormal state determination, topology rollback, and resource cleanup. Each atomic step is assigned a unique and fixed Step-ID, and strict pre-defined sequential dependencies are implemented.

[0055] The system incorporates a high-frequency status acquisition mechanism, collecting runtime status information before, during, and after each atomic step. This information is persistently stored as step status (not executed / in progress / successful / failed), timestamps, execution logs, error codes, and instantaneous cluster parameters, forming a structured snapshot of the exercise progress. When the exercise process is interrupted due to network jitter, node outages, or process lag, restarting the task automatically skips all successfully executed steps by comparing the persistent Step-ID progress table, continuing execution only from the breakpoint, achieving breakpoint continuation capability without manual intervention.

[0056] 2. Consistency Decision Evaluation Layer: Two-Stage Data Barrier Alignment Determination Mechanism

[0057] This layer is responsible for ensuring data consistency during the switchover process. First, it freezes the incremental log synchronization of the standby cluster and inserts barrier markers into the global transaction flow; second, it unifies the log replay points of each node and solidifies the cluster data view; finally, it verifies the overall data status and can only proceed to the next step after meeting the preset consistency threshold.

[0058] This layer is the core of data security decision-making, replacing the traditional, passive, and coarse logic of waiting for synchronization to catch up. It achieves strong consistency locking of the primary and backup cluster transaction boundaries through proactive barrier marking and a global verification model. This invention designs a two-phase forced alignment protocol, with the specific execution logic and judgment rules as follows:

[0059] First stage: freeze and solidify the standby cluster barrier. The system issues a global synchronous freezing instruction, forbids standby cluster nodes from receiving new incremental logs from the primary cluster, and inserts a globally unified transaction barrier mark (Barrier-LSN) at the end position of current log replay. It controls all nodes in the standby cluster to forcibly replay the stored logs until the log positions of all nodes completely reach the barrier mark position, then locks the replay status of each node, forbids further replay, and unifies the internal data view of the standby cluster.

[0060] Second stage: global consistency check of the primary cluster. The primary cluster traverses and detects the barrier completion status of all standby nodes at a configurable polling period (3s / time by default), and the check dimensions include: log file continuity, barrier position matching degree, cluster transaction status, and synchronization delay value. The system presets an alignment timeout threshold (60s by default). Only when all three conditions that the barrier positions of all standby nodes are completely consistent, the synchronization delay ≤ 1s, and there are no uncommitted residual transactions are satisfied at the same time, it is determined that the data alignment is qualified, and the switching process is allowed to enter; if the threshold is not met after timeout or there are position differences, the drill is terminated immediately, the cluster synchronization status is frozen and a data inconsistency alarm is pushed.

[0061] 3. Intelligent security scheduling layer: hierarchical rollback decision + global concurrent mutual exclusion control mechanism

[0062] This layer is responsible for drill risk control and rollback strategy scheduling. First, thresholds are set according to the drill progress to divide high and low risk intervals; second, corresponding hierarchical differentiated rollback strategies are matched according to abnormal scenarios; finally, a distributed lock mechanism is adopted to realize global mutual exclusion control of disaster recovery operations and prevent concurrent conflicts.

[0063] This layer is the core of the system's intelligent decision-making, and includes two quantifiable decision models, which implement refined abnormal rollback and operation security isolation respectively.

[0064] (1) Graded threshold conditional intelligent rollback model

[0065] The system presets a safe rollback threshold T according to the total number of atomic steps (which is set at the position of 60% of the total steps by default), and divides the drill process into two front and rear risk intervals: the front low-risk interval (execution steps < T) and the rear high-risk interval (execution steps ≥ T). Abnormal signals are monitored in real time during the operation of the drill, and differentiated judgment logic is adopted:

[0066] (a) Abnormality in the low-risk interval: No cluster role change or traffic switching has occurred at this time. Once an error report, timeout or check failure is captured, the system directly executes unconditional full reverse rollback, restores traffic routing, cluster roles, synchronization links and node parameters in sequence according to reverse steps, and completely restores the initial state;

[0067] (b) High-risk interval anomaly: At this time, the cluster topology has changed. The system prohibits blind rollback and automatically collects four cluster status indicators (master and backup role status, replication link connectivity, node heartbeat liveness status, and dual master conflict risk status) for secondary comprehensive judgment. Accurate rollback is triggered only when high-risk statuses such as topology disorder, bidirectional replication, and node disconnection are detected. If there is no high risk, the on-site status is retained and the exercise is terminated to avoid cluster oscillation caused by ineffective rollback.

[0068] (2) Distributed lock global concurrency mutual exclusion model

[0069] The system generates a unique global UUID for each exercise task and creates exclusive lock resources based on the cluster's distributed lock. During the pre-exercise self-check phase, a full scan of all running task types within the cluster is performed, including high-risk operations such as disaster recovery exercises, automatic failover, topology adjustments, and VIP traffic switching. If a running disaster recovery task is detected, the exercise is directly intercepted, and a concurrency conflict error is returned, ensuring that only a single disaster recovery task is allowed to execute globally at any given time. An automatic lock recycling mechanism is also configured, automatically clearing lock resources and temporary configurations in three scenarios: normal task completion, voluntary termination, and abnormal crash. This prevents deadlocks from blocking routine data synchronization and completely avoids the risks of cluster split-brain, synchronization disorder, and role conflicts.

[0070] 4. Drill Execution and Visualized Rollback Layer: A Mechanism for Controllable Execution and Quantifiable Progress Throughout the Entire Process

[0071] This layer is responsible for the closed-loop management of cluster switching execution and drills. First, it orderly completes the switching of cluster roles and the scheduling of business traffic; second, it calculates the execution percentage of each step to quantitatively display the drill progress; finally, it automatically restores the cluster architecture and resumes normal synchronization mode after the drill is completed.

[0072] This layer is the final execution closed-loop layer, responsible for the orderly execution of the exercise process, quantitative reporting of progress, closed-loop rollback of anomalies, and cluster state restoration, achieving a fully unattended and controllable implementation. The system uses atomic steps as the smallest statistical unit, calculating the exercise completion progress in real time: Progress value = (Number of successfully executed atomic steps / Total number of steps) × 100%. It continuously updates the progress percentage and current stage label, accurately distinguishing between the pre-inspection stage, data alignment stage, switchover execution stage, verification stage, rollback stage, and closing stage. It also provides a standardized HTTP progress query interface, supporting the operation and maintenance platform to display the exercise status in real time, replacing the traditional pure log traceability method.

[0073] The normal execution flow of the exercise is as follows: state pre-inspection → lock verification → data barrier alignment → write protection isolation of the old primary cluster → standby cluster promotion to primary → traffic routing switch → full-dimensional data verification → exercise success determination. After the exercise ends or an anomaly rollback is completed, the system automatically executes the closed-loop recovery logic: clearing temporary barrier markers, releasing distributed locks, restoring the original primary and standby role topology, rebuilding the normalized data synchronization link, and clearing persistent temporary step records, so that the cluster completely returns to its initial running state, achieving a lossless, residue-free, and disorder-free automated exercise closed loop.

[0074] The flowcharts and block diagrams in the accompanying drawings illustrate possible implementations of apparatus, methods, and computer program products according to various embodiments of this application, including architecture, functionality, and operation. In these figures, each block may represent a module, program segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should be noted that each block in the block diagrams and / or flowcharts, and combinations thereof, can be implemented using either a dedicated hardware-based system or a combination of dedicated hardware and computer instructions to achieve the specified function or operation.

[0075] like Figure 4 As shown in the illustration, an embodiment of this application also discloses an electronic device, including: a processor 310, a communication interface 320, a memory 330 for storing processor-executable computer programs, and a communication bus 340. The processor 310, communication interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 executes the executable computer program to implement the steps of the aforementioned method for automated disaster recovery drills and intelligent rollback of a master-slave cluster database that supports breakpoint resumption.

[0076] It is understood that, in addition to memory and a processor, this electronic device may also include input devices (such as a keyboard), output devices (such as a display), and other communication modules. These input devices, output devices, and other communication modules all communicate with the processor through I / O interfaces (i.e., input / output interfaces).

[0077] The operations described in this application can be implemented by writing computer program code using one or more programming languages ​​or a combination thereof. The programming languages ​​include, but are not limited to, the following types:

[0078] Object-oriented programming languages, such as Java, Smalltalk, C++, etc.

[0079] Conventional procedural programming languages, such as "C" or similar programming languages.

[0080] The execution methods of program code include, but are not limited to:

[0081] It runs entirely on the user's computer;

[0082] Part of it executes on the user's computer, and part of it executes on a remote computer;

[0083] Execute as a standalone software package;

[0084] It is executed entirely on a remote computer or server.

[0085] In scenarios involving remote computers, the remote computer can connect to the user computer via any type of network, including but not limited to a local area network (LAN) or a wide area network (WAN). Furthermore, the remote computer can also connect to external computers via an internet service provider, for example, by utilizing the internet for connection.

[0086] Furthermore, this application also discloses a computer-readable storage medium, which, when the instructions in the computer-readable storage medium are executed by the processor of an electronic device, enables the electronic device to perform the various steps of the automated disaster recovery drill and intelligent rollback method for master-slave cluster databases that supports breakpoint resume, disclosed in this application.

[0087] In the context of this application, a computer-readable storage medium refers to a tangible medium capable of storing computer program code and related data. Specific examples include, but are not limited to, the following:

[0088] (1) Portable computer disk: such as floppy disks and other removable magnetic storage media.

[0089] (2) Hard disk: including mechanical hard disks and solid-state hard disks and other fixed storage devices.

[0090] (3) Random Access Memory (RAM): A volatile storage medium used for temporary storage of data and program code.

[0091] (4) Read-only memory (ROM): a non-volatile storage medium used to store fixed programs and data.

[0092] (5) Erasable programmable read-only memory (EPROM) or flash memory: non-volatile storage media that supports multiple erasures and reprogrammings.

[0093] (6) Fiber optic storage devices: storage media based on fiber optic technology.

[0094] (7) Portable compact disc read-only memory (CD-ROM): a read-only medium that stores data in the form of an optical disc.

[0095] (8) Optical storage devices: such as DVDs, Blu-ray discs and other storage media based on optical principles.

[0096] (9) Magnetic storage devices: such as magnetic tapes, disks and other storage media based on magnetic principles.

[0097] (10) Any suitable combination of the above: for example, combining multiple storage media to meet different storage needs.

[0098] These computer-readable storage media can be used to store the program code and related data described in this application to support program execution and persistent data storage.

[0099] Specifically, according to embodiments of this application, the processes described in the flowcharts can be implemented as computer software programs. For example, embodiments of this application relate to a computer program product comprising a computer program carried on a non-transitory computer-readable medium. This computer program includes program code for executing the automated disaster recovery drill and intelligent rollback method for primary / standby cluster databases that supports breakpoint resumption disclosed in this application. When this computer program is executed by a processing device, it can achieve the functions defined in the embodiments of this application.

[0100] While the foregoing discussion contains several specific implementation details, these details should not be construed as limiting the scope of this application. The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this application is not limited to technical solutions formed by specific combinations of the above-described technical features. Furthermore, this application should also cover other technical solutions formed by any combination of the above-described technical features or their equivalents without departing from the foregoing disclosed concept.

[0101] Those skilled in the art should also understand that modifications can be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features, without departing from the spirit and scope of the technical solutions of the embodiments of this application. These modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the core spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for automated disaster recovery drills and intelligent rollback of master-slave cluster databases that supports breakpoint resumption, characterized in that, Includes the following steps: S1. Data Status Acquisition: The disaster recovery exercise process of the primary and backup cluster databases is broken down into multiple atomic steps, and a unique step identifier is assigned to each atomic step. During the execution of atomic steps, running status information is collected and persistently stored to form a snapshot of the exercise progress; when the exercise process is interrupted and restarted, execution resumes from the breakpoint based on the exercise progress snapshot; S2. Consistency Decision Assessment: Before the drill switch, freeze the incremental log synchronization of the standby cluster, insert a globally unified transaction barrier mark at the end position of the log replay; control each node of the standby cluster to force the replay of the existing logs until the barrier mark position is reached, and lock the replay status of each node to unify the data view inside the standby cluster. Perform a full consistency check on the primary and standby clusters, and proceed with the switchover process once the preset consistency conditions are met. S3. Intelligent safety scheduling: Set rollback thresholds based on the progress of the exercise, and divide the exercise process into a low-risk zone before the exercise and a high-risk zone after the exercise. When an anomaly occurs in the low-risk zone, a full reverse rollback is performed. When an anomaly occurs in the high-risk zone, a comprehensive judgment is made based on cluster status indicators, and a precise rollback is triggered only when a high-risk state is detected. Global concurrency and mutual exclusion control are implemented for disaster recovery drill operations based on a distributed lock mechanism. S4. Drill Execution and Rollback: Sequentially execute cluster role switching and business traffic scheduling; real-time statistics and external display of atomic step execution progress; after normal completion of the drill or abnormal rollback, automatically clean up temporary configurations, release distributed locks, restore the original cluster topology and data synchronization links.

2. The method according to claim 1, characterized in that, The atomic steps described in step S1 include: cluster state pre-inspection, distributed lock verification, synchronization state detection, data barrier marking, cluster role switching, business traffic switching, data consistency verification, abnormal state determination, topology rollback and resource cleanup.

3. The method according to claim 1, characterized in that, The step S1, which involves collecting and persistently storing runtime status information during the execution of atomic steps, includes: collecting runtime status information before, during, and after each atomic step, and persistently storing the step status, timestamp, execution log, error code, and instantaneous cluster parameters to form a structured snapshot of the exercise progress.

4. The method according to claim 1, characterized in that, The dimensions of global consistency verification mentioned in step S2 include: log file continuity, barrier site matching degree, cluster transaction status, and synchronization delay value; The preset consistency conditions include: all backup nodes have consistent barrier points, the primary-backup synchronization delay does not exceed a preset threshold, and there are no uncommitted residual transactions.

5. The method according to claim 1, characterized in that, The rollback threshold mentioned in step S3 is set proportionally based on the total number of atomic steps.

6. The method according to claim 1, characterized in that, The cluster status indicators mentioned in step S3 include: primary / standby role status, replication link connectivity, node heartbeat liveness status, and dual-primary conflict risk status. The distributed lock mechanism includes: generating a unique global identifier for each exercise task and creating exclusive lock resources; scanning the disaster recovery-related tasks running in the cluster before the exercise starts, and intercepting the exercise if a running disaster recovery task is detected; configuring an automatic lock recycling mechanism to automatically release lock resources in the event of normal task completion, active termination, or abnormal crash.

7. The method according to claim 1, characterized in that, The statistical formula for the execution progress of the atomic steps in step S4 is: Progress value = Number of atomic steps successfully executed / Total number of atomic steps × 100%.

8. The method according to claim 1, characterized in that, The external display described in step S4 is achieved through a standardized HTTP progress query interface.

9. A primary / standby cluster database automated disaster recovery drill and intelligent rollback device supporting breakpoint resumption, characterized in that, When the device is in operation, it implements the steps of the automated disaster recovery drill and intelligent rollback method for primary and standby cluster databases that supports breakpoint resumption as described in any one of claims 1-8, including: The data status acquisition module is used to break down the disaster recovery drill process of the primary and backup cluster databases into multiple atomic steps, assign a unique step identifier to each atomic step, collect running status information during the execution of atomic steps and persist it to form a drill progress snapshot, and when the drill process is interrupted and restarted, it resumes execution from the breakpoint based on the drill progress snapshot. The consistency decision evaluation module is used to freeze the incremental log synchronization of the standby cluster before the drill switch, insert a globally unified transaction barrier mark at the end position of the log replay; control each node of the standby cluster to force the replay of the existing logs until the barrier mark position is reached, lock the replay status of each node to unify the data view inside the standby cluster; perform a global consistency check on the primary and standby clusters, and enter the switch process after the preset consistency conditions are met. The intelligent safety scheduling module is used to set rollback thresholds based on the exercise execution progress, dividing the exercise process into a pre-low-risk interval and a post-high-risk interval; when an anomaly occurs in the low-risk interval, a full reverse rollback is performed; when an anomaly occurs in the high-risk interval, a comprehensive judgment is made based on cluster status indicators, and a precise rollback is triggered only when a high-risk state is detected; and a distributed lock mechanism is used to perform global concurrency and mutual exclusion control on disaster recovery exercise operations. The exercise execution and rollback module is used to sequentially execute cluster role switching and business traffic scheduling; to statistically analyze and display the execution progress of atomic steps in real time; and to automatically clean up temporary configurations, release distributed locks, and restore the original cluster topology and data synchronization links after the exercise ends normally or rolls back abnormally.

10. An electronic device, characterized in that, include: Memory and processor; Memory: Used to store computer programs; Processor: Used to execute the computer program to implement the steps of the automated disaster recovery drill and intelligent rollback method for master-slave cluster databases that supports breakpoint resumption as described in any one of claims 1-8.

Images