Cross-cluster disaster recovery method and device, electronic equipment, storage medium and product

Abstract

The invention discloses a cross-cluster disaster recovery method and device, electronic equipment, a storage medium and a product. The method comprises the steps that in the process that a main cluster conducts window aggregation calculation on to-be-consumed data in a to-be-consumed database, intermediate state information of an aggregation window is captured through a dynamic state mirror image DSM, and the intermediate state information is synchronized to a standby cluster; detecting the consumption data offset of the main cluster to the to-be-consumed database, and storing the consumption data offset in a distributed file system HDFS; and when a fault of the main cluster is detected, the standby cluster reads the latest consumption data offset from the HDFS, and determines non-consumption data of the main cluster in the to-be-consumed database based on the latest intermediate state information and the latest consumption data offset so as to process the non-consumption data. According to the scheme, the cross-cluster data zero-loss and zero-repetition disaster recovery capability is realized.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a cross-cluster disaster recovery method, apparatus, electronic device, storage medium, and product. Background Technology

[0002] The foundation of current financial information system monitoring platforms is monitoring data, and distributed stream processing (primarily Flink) is a core step in the monitoring data processing process, requiring high stability, data integrity, and disaster recovery capabilities. To ensure business continuity and data integrity, dual-cluster disaster recovery is typically implemented for Flink clusters. However, current technology faces challenges such as loss of intermediate states during window aggregation, poor consistency between offsets and states, and low efficiency in cross-cluster synchronization. For example, in performance monitoring threshold alarm scenarios, traditional disaster recovery solutions (such as Flink Checkpoint) only save static snapshots. If the primary cluster crashes before the window is closed, the backup cluster needs to recalculate the incomplete window, resulting in the loss of intermediate results. If alarm calculation is triggered before window aggregation (e.g., 1-minute transaction volume) is completed, the backup cluster's recalculation may miss some transaction data, leading to false alarms.

[0003] In addition, the stream processing system has put forward higher requirements for disaster recovery solutions: (1) Decoupling of event time and processing time; it is necessary to support window calculation based on event time (such as sliding window, session window), and after the fault is recovered, the window timer and the unprocessed event must be consistent in time. (2) Exactly-Once semantic guarantee; data must be processed only once after the disaster recovery switch to avoid duplication or loss (such as financial transaction deduction scenario). (3) Dynamic scaling compatibility; the primary and backup clusters must support elastic scaling (such as Kubernetes automatic scaling), and the state synchronization is not interrupted due to changes in the cluster size. In response to the above technical pain points, the following technical problems need to be solved: (1) How to capture and synchronize the intermediate state of window aggregation in real time to ensure that the backup cluster can seamlessly take over the unclosed window; (2) How to achieve strong consistency between Checkpoint commit and Kafka offset commit through transaction atomicity guarantee; (3) How to complete state synchronization with low bandwidth consumption to support efficient disaster recovery of large-scale real-time data streams. Summary of the Invention

[0004] This invention provides a cross-cluster disaster recovery method, device, electronic device, storage medium and product, which can effectively solve the problems of loss of intermediate state of window aggregation, data duplication or loss caused by failure of the main cluster, and realize cross-cluster data zero loss and zero duplication disaster recovery capability.

[0005] According to one aspect of the present invention, a cross-cluster disaster recovery method is provided, comprising:

[0006] During the process of performing window aggregation calculation on the data to be consumed in the database to be consumed in the main cluster, the intermediate state information of the aggregation window is captured by the Dynamic State Mirror (DSM), and the intermediate state information is synchronized to the backup cluster.

[0007] The main cluster detects the consumption data offset of the database to be consumed and stores the consumption data offset in the distributed file system HDFS;

[0008] When a failure of the primary cluster is detected, the backup cluster reads the latest consumption data offset from the distributed file system HDFS, and determines the unconsumed data of the primary cluster in the database to be consumed based on the latest intermediate state information and the latest consumption data offset, so as to process the unconsumed data.

[0009] According to another aspect of the present invention, a cross-cluster disaster recovery device is provided, comprising:

[0010] The intermediate state information capture module is used to capture the intermediate state information of the aggregation window through the Dynamic State Mirror (DSM) during the window aggregation calculation of the data to be consumed in the database to be consumed in the main cluster, and synchronize the intermediate state information to the backup cluster.

[0011] The consumption data offset detection module is used to detect the consumption data offset of the main cluster to the database to be consumed, and store the consumption data offset in the distributed file system HDFS;

[0012] The standby cluster takeover module is used to read the latest consumption data offset from the distributed file system HDFS when the primary cluster failure is detected, and to determine the unconsumed data of the primary cluster in the database to be consumed based on the latest intermediate state information and the latest consumption data offset, so as to process the unconsumed data.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the cross-cluster disaster recovery method according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the cross-cluster disaster recovery method according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the cross-cluster disaster recovery method described in any embodiment of the present invention.

[0019] The cross-cluster disaster recovery scheme of this invention, during the window aggregation calculation of unconsumed data in the unconsumed database of the primary cluster, captures intermediate state information of the aggregation window through a Dynamic State Mirror (DSM) and synchronizes the intermediate state information to the backup cluster; detects the consumption data offset of the primary cluster relative to the unconsumed database and stores the consumption data offset in a Distributed File System (HDFS); when a failure of the primary cluster is detected, the backup cluster reads the latest consumption data offset from the HDFS and determines the unconsumed data of the primary cluster in the unconsumed database based on the latest intermediate state information and the latest consumption data offset, and processes the unconsumed data. The technical solution provided by this invention effectively solves the problems of lost intermediate states, duplicated data, or lost data due to primary cluster failure, achieving cross-cluster disaster recovery capability with zero data loss and zero duplication.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a cross-cluster disaster recovery method provided in an embodiment of the present invention;

[0023] Figure 2 An architecture diagram of a cross-cluster disaster recovery system provided in an embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram of the Kafka consumption process of the main cluster under normal conditions, provided by an embodiment of the present invention.

[0025] Figure 4 This is a schematic diagram of the structure of a cross-cluster disaster recovery device provided in an embodiment of the present invention;

[0026] Figure 5 A schematic diagram of the structure of an electronic device for implementing the cross-cluster disaster recovery method of this invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] Figure 1 This is a flowchart illustrating a cross-cluster disaster recovery method provided in an embodiment of the present invention. This embodiment is applicable to situations involving cross-cluster disaster recovery. The method can be executed by a cross-cluster disaster recovery device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0030] S110. During the process of performing window aggregation calculation on the data to be consumed in the database to be consumed in the main cluster, the intermediate state information of the aggregation window is captured by the Dynamic State Mirror (DSM), and the intermediate state information is synchronized to the backup cluster.

[0031] The database to be consumed can be a database stored on the Kafka platform. Kafka is a distributed streaming platform that supports high-throughput message publishing and subscription, and can be used in real-time data management and stream processing systems. Kafka serves as the data source for the data to be consumed in the database. The data contained in the database is the data to be consumed, which can also be understood as the data that needs to be processed. The master cluster can be a Flink master cluster. Flink is an open-source distributed stream processing framework that supports high-throughput, low-latency real-time data processing, providing exactly-once semantics and state management capabilities.

[0032] In this embodiment of the invention, the primary cluster reads data to be consumed from the database and performs window aggregation calculations on the data. The window aggregation calculation can include incremental aggregation calculations and full aggregation calculations. During the window aggregation calculation process, the primary cluster captures intermediate state information of the aggregation window through a Dynamic State Mirror (DSM) and sends this intermediate state information to the backup cluster, effectively synchronizing the intermediate state information to the backup cluster. It is understood that the DSM can dynamically capture the intermediate state information of the aggregation window, rather than simply obtaining a static snapshot of the aggregation window. Optionally, the intermediate state information includes information such as the window aggregation value, unprocessed event timestamps, and timer states.

[0033] Optionally, during the window aggregation calculation of the data to be consumed in the database to be consumed by the main cluster, intermediate state information of the aggregation window is captured through the Dynamic State Mirror (DSM). This includes: during the window aggregation calculation of the data to be consumed by the main cluster, if a preset detection period is reached or a closing operation of the aggregation window is detected, the intermediate state information of the window aggregation is captured through the Dynamic State Mirror (DSM). For example, during the window aggregation calculation of the data to be consumed by the main cluster, if a preset detection period (e.g., every 5 seconds) is reached, the intermediate state information of the window aggregation is captured through the Dynamic State Mirror (DSM). That is, the trigger condition for the Dynamic State Mirror (DSM) to capture the intermediate state information of the window aggregation is periodic, such as capturing the window aggregation value, timer status, and unprocessed event timestamps every 5 seconds. For example, during the window aggregation calculation of the data to be consumed in the database to be consumed by the main cluster, if a closing operation of the aggregation window is detected, the intermediate state information of the window aggregation is captured by the Dynamic State Mirror (DSM). That is, the trigger condition for the DSM to capture the intermediate state information of the window aggregation is event-driven; that is, when a closing operation of the aggregation window is detected, the DSM is forcibly controlled to capture the intermediate state information of the window aggregation. Optionally, during the window aggregation calculation of the data to be consumed by the database to be consumed by the main cluster, the intermediate state information of the aggregation window can also be captured in real time by the DSM.

[0034] Optionally, before synchronizing the intermediate state information to the standby cluster, the method further includes: the dynamic state mirror DSM acquiring a local state snapshot; wherein the local state snapshot is the previously captured aggregate window state information; comparing the intermediate state information with the local state snapshot to determine state difference data; and synchronizing the intermediate state information to the standby cluster, including: synchronizing the state difference data to the standby cluster. For example, the dynamic state mirror DSM acquires its maintained local state snapshot (i.e., the aggregate window state information previously captured by the dynamic state mirror DSM), and compares the local state snapshot with the acquired intermediate state information to generate state difference data (Delta). The generated state difference data reflects the change in the intermediate state information of the most recently captured aggregate window by the dynamic state mirror DSM compared to the intermediate state information of the previously captured aggregate window. RoaringBitmap can be used to mark the changed aggregate window ID and the corresponding state difference data. The dynamic state mirror DSM only synchronizes the state difference data to the standby cluster. The advantage of this setup is that it effectively reduces the amount of data transmitted from the primary cluster to the standby cluster, thereby reducing network bandwidth consumption and improving the performance of both the primary and standby clusters. Furthermore, the Dynamic State Mirror (DSM) can use Snappy compression technology to compress Delta data before synchronizing it to the standby cluster, further reducing network transmission overhead.

[0035] S120. Detect the consumption data offset of the master cluster to the database to be consumed, and store the consumption data offset in the distributed file system HDFS.

[0036] In this embodiment of the invention, as time progresses, the main cluster continuously processes the data to be consumed in the database to be consumed. Therefore, the data consumption offset of the main cluster in the database to be consumed can be detected in real time or according to a preset detection period. The data consumption offset reflects the data position where the main cluster has completed processing the data to be consumed in the database. It can be understood that based on the data consumption offset, it is possible to determine which data to be consumed in the database has been completed and which data has not yet been consumed. The detected data consumption offset is stored in the distributed file system HDFS.

[0037] Optionally, detecting the consumption data offset of the master cluster to the database to be consumed and storing the consumption data offset in the distributed file system HDFS includes: when the master cluster completes a window aggregation calculation, it sends a consumption data offset acquisition request to the data platform where the database to be consumed is located through a two-phase commit module; the two-phase commit module receives the consumption data offset returned by the data platform in response to the consumption data offset acquisition request and stores the consumption data offset in the fault recovery module; the fault recovery module stores the consumption data offset in the distributed file system HDFS.

[0038] For example, when the primary cluster detects that a window aggregation computation has been completed, the two-phase commit module (2PCCoordinator) sends a consumer data offset retrieval request to the data platform (such as Kafka) where the database to be consumed resides. It also controls the Dynamic State Mirror (DSM) to freeze the intermediate state of the current aggregation window, prohibiting subsequent modifications. The DSM then synchronizes the frozen intermediate state of the current aggregation window to the standby cluster. The data platform (such as Kafka) responds to the consumer data offset retrieval request, determines the consumer data offset, and feeds it back to the two-phase commit module (2PC Coordinator). The two-phase commit module (2PC Coordinator) stores the consumer data offset in the fault recovery module Checkpoint. Then, the fault recovery module Checkpoint stores the consumer data offset in the distributed file system HDFS. The fault recovery module Checkpoint can be understood as a fault tolerance mechanism in the Flink primary cluster, periodically saving job state snapshots to persistent storage. Understandably, the two-phase commit module (2PCCoordinator) storing the consumed data offset to the fault recovery module Checkpoint represents the prepare phase of the consumed data offset, while the fault recovery module Checkpoint storing the consumed data offset to the distributed file system HDFS represents the commit phase. The advantage of this setup is that it effectively guarantees the atomicity of the primary cluster consuming data from the database to be consumed.

[0039] S130. When the primary cluster failure is detected, the backup cluster reads the latest consumption data offset from the distributed file system HDFS, and determines the unconsumed data of the primary cluster in the database to be consumed based on the latest intermediate state information and the latest consumption data offset, so as to process the unconsumed data.

[0040] In this embodiment of the invention, when a primary cluster failure is detected, the standby cluster reads the latest consumption data offset from the distributed file system HDFS, and determines the unconsumed data of the primary cluster in the database to be consumed based on the latest intermediate state information and the latest consumption data offset, so as to process the unconsumed data. It is understood that when a primary cluster failure is detected, the standby cluster is activated to an Active state and takes over Kafka consumption. For example, if the timer status in the latest intermediate state information is triggered, it indicates that the aggregation window was closed when the primary cluster failed, meaning the aggregation window has completed window aggregation calculation, and the unconsumed data of the primary cluster in the database to be consumed can be directly determined based on the latest consumption data offset. If the timer status in the latest intermediate state information is not triggered, it indicates that the aggregation window was not closed when the primary cluster failed, meaning the aggregation window has not completed window aggregation calculation. In this case, the unconsumed data of the primary cluster in the database to be consumed is determined based on the unprocessed event timestamp in the intermediate state information and the latest consumption data offset.

[0041] Optionally, processing the unconsumed data includes: determining whether the current time of the standby cluster is later than the aggregation window closing time in the latest intermediate state information; if so, performing window aggregation calculation on the unconsumed data; if not, waiting for the standby cluster's time to reach the aggregation window closing time before performing window aggregation calculation on the unconsumed data. For example, the standby cluster determines its current time and the aggregation window closing time in the latest intermediate state information, where the current time is the time when the primary cluster failure is detected, and the aggregation window closing time is the time when the primary cluster performs window aggregation calculation and when the window aggregation calculation is completed. Determining whether the current time is later than the aggregation window closing time in the latest intermediate state information: if so, directly performing window aggregation calculation on the unconsumed data; if not, waiting for the standby cluster's time to reach the aggregation window closing time before performing window aggregation calculation on the unconsumed data. For example, if the main cluster fails and the time range for window aggregation calculation of a certain aggregation window is 10:00-10:05, it means that the closing time of the aggregation window in the latest intermediate status information is 10:05. If the current time is 10:03, the standby cluster needs to wait until 10:05 to trigger the aggregation window calculation; if the current time is 10:06, the standby cluster will immediately trigger the aggregation window calculation.

[0042] The cross-cluster disaster recovery method of this invention, during the window aggregation calculation of unconsumed data in the unconsumed database of the primary cluster, captures intermediate state information of the aggregation window through a Dynamic State Mirror (DSM) and synchronizes the intermediate state information to the standby cluster; detects the consumption data offset of the primary cluster relative to the unconsumed database and stores the consumption data offset in a Distributed File System (HDFS); when a failure of the primary cluster is detected, the standby cluster reads the latest consumption data offset from the HDFS and determines the unconsumed data of the primary cluster in the unconsumed database based on the latest intermediate state information and the latest consumption data offset, and processes the unconsumed data. The technical solution provided by this invention effectively solves the problems of lost intermediate states, duplicated data, or lost data due to primary cluster failure, achieving cross-cluster disaster recovery capability with zero data loss and zero duplication.

[0043] Figure 2 An architecture diagram of a cross-cluster disaster recovery system provided in an embodiment of the present invention is shown below. Figure 2 As shown, the cross-cluster disaster recovery system comprises a data layer, a data processing layer, and a synchronization algorithm layer. In the data layer, the Kafka platform serves as the data source, providing real-time data stream input and computation result output. The distributed file system HDFS stores the consumption data offsets of the primary and backup clusters for Kafka data. In the data processing layer, the primary cluster's Flink job performs window aggregation computation. During this computation, the Dynamic State Mirror Service (DSM) captures the intermediate state information of the aggregation window in real time (window aggregation value, timestamps of unprocessed events, and timer status); the two-phase commit coordinator (2PC) manages checkpoint commits and Kafka transaction boundaries. In the backup cluster, the checkpoint loader loads the latest consumption data offset from the primary cluster from HDFS; the time window alignment mechanism inherits the primary cluster's unclosed window state, compensates for timestamps, and continues processing Kafka data. In the synchronization algorithm layer, the state probes of the Dynamic State Mirror Service (DSM) are periodically injected into Flink operators to capture intermediate states (such as window aggregation values ​​and unprocessed event timestamps) and generate state difference data (Delta), which includes the changed window IDs and corresponding states. The incremental synchronization algorithm uses RoaringBitmap to mark changed windows and only synchronizes the changed parts (such as newly added windows or updated values). When the standby cluster detects a conflict between local and remote states, it adopts the LWW strategy (prioritizing intermediate state information synchronized from the primary cluster) to resolve the conflict.

[0044] Figure 3 This is a schematic diagram illustrating the Kafka consumption process of the main cluster under normal conditions, provided as an embodiment of the present invention. The description of the above embodiment can be used to... Figure 3 The meaning is clear and will not be elaborated upon here.

[0045] Figure 4 This is a schematic diagram of a cross-cluster disaster recovery device provided in an embodiment of the present invention. Figure 4 As shown, the device includes:

[0046] The intermediate state information capture module 410 is used to capture the intermediate state information of the aggregation window through the dynamic state mirror DSM during the window aggregation calculation of the data to be consumed in the database to be consumed in the main cluster, and synchronize the intermediate state information to the backup cluster.

[0047] The consumption data offset detection module 420 is used to detect the consumption data offset of the main cluster to the database to be consumed, and store the consumption data offset in the distributed file system HDFS;

[0048] The standby cluster takeover module 430 is used to, when the primary cluster failure is detected, read the latest consumption data offset from the distributed file system HDFS, and determine the unconsumed data of the primary cluster in the database to be consumed based on the latest intermediate state information and the latest consumption data offset, so as to process the unconsumed data.

[0049] Optional, an intermediate state information capture module, used for:

[0050] During the window aggregation calculation of the data to be consumed in the database to be consumed in the main cluster, if the preset detection period is reached or the closing operation of the aggregation window is detected, the intermediate state information of the window aggregation is captured by the dynamic state mirror DSM.

[0051] Optional, also includes:

[0052] The local state snapshot acquisition module is used to acquire a local state snapshot of the Dynamic State Mirror (DSM) before synchronizing the intermediate state information to the standby cluster; wherein, the local state snapshot is the aggregated window state information captured last time;

[0053] The state difference data determination module is used to compare the intermediate state information with the local state snapshot to determine the state difference data.

[0054] The intermediate state information capture module is used for:

[0055] The state difference data is synchronized to the standby cluster.

[0056] Optional, a consumption data offset detection module is used for:

[0057] When the main cluster completes a window aggregation calculation, it sends a consumption data offset acquisition request to the data platform where the database to be consumed is located through the two-phase commit module.

[0058] The two-stage submission module receives the consumption data offset amount from the data platform in response to the consumption data offset acquisition request, and stores the consumption data offset amount in the fault recovery module;

[0059] The fault recovery module stores the consumption data offset in the distributed file system HDFS.

[0060] Optional, a backup cluster takeover module is used for:

[0061] If the current time of the standby cluster is later than the aggregation window closing time in the latest intermediate state information, then perform window aggregation calculation on the unconsumed data; otherwise, wait until the time of the standby cluster reaches the aggregation window closing time before performing window aggregation calculation on the unconsumed data.

[0062] Optionally, the intermediate state information includes window aggregation value, unprocessed event timestamps, and timer status.

[0063] The cross-cluster disaster recovery device provided in this embodiment of the invention can execute the cross-cluster disaster recovery method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0064] Figure 5 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0065] like Figure 5As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0066] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0067] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as cross-cluster disaster recovery methods.

[0068] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication unit 19, or installed from storage unit 18, or installed from ROM 12. When the computer program is executed by processor 11, it performs the functions defined in the methods of the embodiments of the present invention.

[0069] In some embodiments, the cross-cluster disaster recovery method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the cross-cluster disaster recovery method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the cross-cluster disaster recovery method by any other suitable means (e.g., by means of firmware).

[0070] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0071] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0072] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0073] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0074] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0075] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0076] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0077] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A cross-cluster disaster recovery method, characterized in that, include: During the process of performing window aggregation calculation on the data to be consumed in the database to be consumed in the main cluster, the intermediate state information of the aggregation window is captured by the Dynamic State Mirror (DSM), and the intermediate state information is synchronized to the backup cluster. The main cluster detects the consumption data offset of the database to be consumed and stores the consumption data offset in the distributed file system HDFS; When a failure of the primary cluster is detected, the backup cluster reads the latest consumption data offset from the distributed file system HDFS, and determines the unconsumed data of the primary cluster in the database to be consumed based on the latest intermediate state information and the latest consumption data offset, so as to process the unconsumed data.

2. The method according to claim 1, characterized in that, During the window aggregation calculation of the data to be consumed in the database to be consumed in the main cluster, the intermediate state information of the aggregation window is captured by the Dynamic State Mirror (DSM), including: During the window aggregation calculation of the data to be consumed in the database to be consumed in the main cluster, if the preset detection period is reached or the closing operation of the aggregation window is detected, the intermediate state information of the window aggregation is captured by the dynamic state mirror DSM.

3. The method according to claim 1, characterized in that, Before synchronizing the intermediate state information to the backup cluster, the following steps are also included: The Dynamic State Mirror (DSM) acquires a local state snapshot; wherein, the local state snapshot is the previously captured aggregate window state information; The intermediate state information is compared with the local state snapshot to determine the state difference data; Synchronizing the intermediate state information to the backup cluster includes: The state difference data is synchronized to the standby cluster.

4. The method according to claim 1, characterized in that, Detecting the consumption data offset of the master cluster to the database to be consumed, and storing the consumption data offset in the distributed file system HDFS, includes: When the main cluster completes a window aggregation calculation, it sends a consumption data offset acquisition request to the data platform where the database to be consumed is located through the two-phase commit module. The two-stage submission module receives the consumption data offset amount from the data platform in response to the consumption data offset acquisition request, and stores the consumption data offset amount in the fault recovery module; The fault recovery module stores the consumption data offset in the distributed file system HDFS.

5. The method according to claim 1, characterized in that, Processing the unconsumed data includes: If the current time of the standby cluster is later than the aggregation window closing time in the latest intermediate state information, then perform window aggregation calculation on the unconsumed data; otherwise, wait until the time of the standby cluster reaches the aggregation window closing time before performing window aggregation calculation on the unconsumed data.

6. The method according to claim 1, characterized in that, The intermediate state information includes the window aggregation value, unprocessed event timestamps, and timer status.

7. A cross-cluster disaster recovery device, characterized in that, include: The intermediate state information capture module is used to capture the intermediate state information of the aggregation window through the Dynamic State Mirror (DSM) during the window aggregation calculation of the data to be consumed in the database to be consumed in the main cluster, and synchronize the intermediate state information to the backup cluster. The consumption data offset detection module is used to detect the consumption data offset of the main cluster to the database to be consumed, and store the consumption data offset in the distributed file system HDFS; The standby cluster takeover module is used to read the latest consumption data offset from the distributed file system HDFS when the primary cluster failure is detected, and to determine the unconsumed data of the primary cluster in the database to be consumed based on the latest intermediate state information and the latest consumption data offset, so as to process the unconsumed data.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the cross-cluster disaster recovery method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the cross-cluster disaster recovery method according to any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the cross-cluster disaster recovery method according to any one of claims 1-6.

Images