Disaster recovery method and device in active-active system and active-active system

Abstract

The present application relates to a disaster recovery method, device and active-active system in an active-active system. After the consumer client agrees on the consumption switching proposal through negotiation, the second consumer cluster is notified to perform consumption switching, thereby realizing disaster recovery control of the active-active system. The mechanisms of consumption status synchronization, consumption cluster switching negotiation and consumption cluster switching notification are introduced in the consumer client, so that status synchronization and consumption switching negotiation only rely on the communication between consumer clients, and no additional centralized components or services other than the message middleware are required to be deployed. The decentralized state synchronization and consumption switching negotiation methods are adopted to avoid potential performance problems in massive applications using centralized services or distributed consistency coordination services. There is no need for the message middleware server to handle the disaster recovery process, which avoids the performance pressure on the server caused by a large number of businesses sharing the message middleware, and can save computing resources.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a disaster recovery method and device in an active-active system, and an active-active system. Background Art

[0002] Traditional solutions for high availability and disaster recovery of message middleware consumer clients rely on the network heartbeat of the consumer instance to determine if it is abnormal. Within a single data center, the message middleware server maintains a list of all consumer instances with normal heartbeats and can eliminate abnormal consumers based on heartbeat detection to implement failover.

[0003] In cross-data center disaster recovery scenarios, traditional solutions can solve the master election problem by monitoring consumer network heartbeats and combining them with distributed consistency coordination services such as Zookeeper (distributed application coordination service software) and etcd (distributed key-value storage system). By monitoring master-slave switching events, consumer threads within different data centers can be started and shut down.

[0004] Currently, the number of consumers in a single consumer cluster is usually related to the number of single business application instances. A data center usually contains a large number of different businesses. Using additional customized centralized services or distributed consistency coordination services to handle customized disaster recovery strategies for a large number of clusters outside of consumer heartbeats poses performance risks. Summary of the Invention

[0005] Based on this, it is necessary to provide a disaster recovery method, device and active-active system in an active-active system to address the above technical problems, which can avoid potential performance problems in massive applications using centralized services or distributed consistency coordination services.

[0006] In a first aspect, the present application provides a disaster recovery method in an active-active system, the active-active system including a first data center and a second data center, the first data center including a first message middleware and a first consumer cluster, and the second data center including a second message middleware and a second consumer cluster; the method is applied to any consumer client in the first consumer cluster, the method comprising:

[0007] Receive first synchronization data; the first synchronization data includes the status of other consumer clients in the first consumer cluster;

[0008] Determining, based on the first synchronization data, a first number of consumer clients in an abnormal state in the first consumer cluster;

[0009] When the first number is greater than a first set threshold, broadcasting a first consumption switching proposal in the first consumer cluster;

[0010] Receive second synchronization data; the second synchronization data includes consumption switching voting results provided by other consumer clients in the first consumer cluster;

[0011] Determining, based on the second synchronization data, a second number of consumer clients that agree to the first consumption switching proposal;

[0012] When the second number is greater than the second set threshold, the consumption thread is closed, and a first consumption switching instruction is sent to any consumer client in the second consumer cluster.

[0013] In one embodiment, the method further comprises:

[0014] Get current running information;

[0015] Based on the current operation information, determine whether the preset consumption abnormality conditions are met;

[0016] If satisfied, the current state is updated to an abnormal state, and the abnormal state is broadcast to other consumer clients in the first consumer cluster.

[0017] In one embodiment, the current operation information includes the number of consumption failure exceptions, the consumption retry ratio per unit time, and the consumption timeout ratio; if the number of consumption failure exceptions is greater than a preset threshold, the consumption retry ratio per unit time is greater than a preset ratio, and the consumption timeout ratio is greater than a preset ratio, it is determined that the preset consumption exception conditions are currently met.

[0018] In one embodiment, the method further comprises:

[0019] Receive third synchronization data; the third synchronization data includes a consumption switching proposal initiated by other consumer clients in the first consumer cluster;

[0020] Verify whether the consumption switching condition is currently met based on the third synchronization data;

[0021] Generate consumption switching voting results based on the verification results;

[0022] The generated consumption switching voting results are synchronized to other consumer clients in the first consumer cluster.

[0023] In one embodiment, the method further comprises:

[0024] If a second consumption switching instruction is received, the cross-data center consumption thread is started, and the consumption data of the second message middleware is consumed based on the cross-data center consumption thread; the second consumption switching instruction is provided by any consumer client in the second consumer cluster or other consumer clients in the first consumer cluster.

[0025] In one embodiment, the method further comprises:

[0026] Determining a third number of consumer clients that initiate cross-data center consumption threads in the first consumer cluster;

[0027] When the third number is greater than the third set threshold, a consumption switching success response is sent to any consumer client in the second consumer cluster.

[0028] In one embodiment, the method further comprises:

[0029] Periodically send a request to the first message middleware to obtain the network address of each consumer client in the first consumer cluster and the second consumer cluster.

[0030] In one embodiment, the first synchronization data further includes the status of each consumer client in the second consumer cluster; the method further includes:

[0031] Determining, based on the first synchronization data, a fourth number of consumer clients in the second consumer cluster that are in an abnormal state;

[0032] When the fourth number is greater than a fourth set threshold, broadcasting a second consumption switching proposal in the first consumer cluster;

[0033] If the number of consumer clients in the first consumer cluster that agree to the second consumption switching proposal is greater than a second set threshold, a third consumption switching instruction is sent to any consumer client in the second consumer cluster.

[0034] In a second aspect, the present application further provides a disaster recovery device in an active-active system, the active-active system including a first data center and a second data center, the first data center including a first message middleware and a first consumer cluster, and the second data center including a second message middleware and a second consumer cluster; the device is applied to any consumer client in the first consumer cluster, the device including:

[0035] A consumer communication module, configured to receive first synchronization data; the first synchronization data includes the status of other consumer clients in the first consumer cluster;

[0036] A consumption switching module is configured to determine, based on the first synchronization data, a first number of consumer clients in an abnormal state in a first consumer cluster; and broadcast a first consumption switching proposal in the first consumer cluster when the first number is greater than a first set threshold;

[0037] The consumer communication module is further configured to receive second synchronization data; the second synchronization data includes consumption switching voting results provided by other consumer clients in the first consumer cluster;

[0038] The consumption switching module is also used to determine the second number of consumer clients that agree to the first consumption switching proposal based on the second synchronization data; when the second number is greater than the second set threshold, the consumption thread is closed and the first consumption switching instruction is sent to any consumer client in the second consumer cluster.

[0039] In a third aspect, the present application also provides an active-active system, which includes a first data center and a second data center, the first data center includes a first message middleware and a first consumer cluster, and the second data center includes a second message middleware and a second consumer cluster; when any consumer client in the first consumer cluster or the second consumer cluster executes a computer program, the steps of the method described in the first aspect are implemented.

[0040] The disaster recovery method, device and active-active system in the above-mentioned active-active system include a first data center and a second data center, the first data center includes a first message middleware and a first consumer cluster, and the second data center includes a second message middleware and a second consumer cluster; any consumer client in the first consumer cluster receives the first synchronization data; the first synchronization data includes the status of other consumer clients in the first consumer cluster; based on the first synchronization data, the first number of consumer clients in the first consumer cluster that are in an abnormal state is determined; when the first number is greater than the first set threshold, the first consumption switching proposal is broadcast in the first consumer cluster; the second synchronization data is received; the second synchronization data includes the consumption switching voting results provided by other consumer clients in the first consumer cluster; based on the second synchronization data, the second number of consumer clients that agree with the first consumption switching proposal is determined; when the second number is greater than the second set threshold, the consumption thread is closed, and the first consumption switching instruction is sent to any consumer client in the second consumer cluster. Through the above method, after the consumer client negotiates and agrees to the consumption switching proposal, the second consumer cluster is notified to perform consumption switching, thereby realizing disaster recovery control of the active-active system. By introducing mechanisms for consumer state synchronization, consumer cluster switch negotiation, and consumer cluster switch notification within the consumer client, state synchronization and consumer switch negotiation rely solely on communication between consumer clients, eliminating the need for additional centralized components or services beyond the messaging middleware. This decentralized approach to state synchronization and consumer switch negotiation mitigates potential performance issues associated with using centralized or distributed consistency coordination services for large volumes of applications. This eliminates the need for message middleware servers to handle disaster recovery processes, preventing server-side performance pressure caused by a large number of businesses sharing the messaging middleware and conserving computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 This is a diagram of the disaster recovery process for the traditional solution;

[0043] Figure 2 Schematic diagram of the drawbacks of heartbeat detection in traditional solutions;

[0044] Figure 3 This is a diagram of an application environment of a disaster recovery method in an active-active system according to an embodiment;

[0045] Figure 4 1 is a flow chart of a disaster recovery method in an active-active system according to an embodiment;

[0046] Figure 5 A schematic diagram of the module structure of a consumer client in one embodiment;

[0047] Figure 6 A schematic diagram of consumer status synchronization in one embodiment;

[0048] Figure 7 A schematic diagram of consumer cluster switching in one embodiment;

[0049] Figure 8 1 is a flow chart of a disaster recovery method in an active-active system in another embodiment;

[0050] Figure 9 FIG. 1 is a structural block diagram of a disaster recovery device in an active-active system in an embodiment. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0052] It is understood that the terms "first," "second," and the like used herein may be used to describe various elements herein, but these elements are not limited by these terms. These terms are used only to distinguish a first element from another element. For example, a first consumer cluster may be referred to as a second consumer cluster, and similarly, a second consumer cluster may be referred to as a first consumer cluster, without departing from the scope of this application. Both the first consumer cluster and the second consumer cluster are consumer clusters, but they are not the same consumer cluster.

[0053] To facilitate understanding of the embodiments provided in this application, some terms that appear in the specific description are introduced below:

[0054] Message-based middleware (MQ), also known as a message queue, is a software system that facilitates message delivery and exchange within a distributed system. It uses asynchronous messaging to enable communication and data exchange between different applications, services, or components without requiring direct connections. MQ is also used to decouple systems, improving reliability and scalability. Currently, mainstream message-based middleware includes Kafka, Pulsar, and RocketMQ.

[0055] Topic: In the publish / subscribe model, a name used to categorize messages. It represents the classification identifier for messages stored on the message middleware server. Producers publish messages to a topic, and consumers who subscribe to the topic receive the messages.

[0056] Producer: An application or component responsible for creating and sending messages. Producers send messages to message queues for consumers to consume.

[0057] Consumer: An application or component that receives and processes messages from a message queue. A consumer is typically a program module within a business service instance.

[0058] Publish / Subscribe: A messaging model where producers publish messages to topics, and consumers subscribe to topics to receive messages for business processing. Messages can be received by multiple consumers.

[0059] Cluster consumption: Cluster consumption is a common message consumption model. In this model, multiple consumers form a consumer cluster to jointly consume messages from MQ. Each consumer in the cluster is equivalent in terms of business processing logic. Each message will only be successfully processed by one consumer in the cluster, improving system throughput and reliability.

[0060] Message backlog: This refers to a situation where the number of messages in MQ continues to increase, but message consumers fail to process these messages in a timely manner. Message backlog can lead to system performance degradation and even message loss, causing business problems.

[0061] An Internet Data Center (IDC) is a facility used to centrally manage computing and storage resources. It includes servers, storage systems, network equipment, and other infrastructure to support an organization's computing, storage, and data processing needs. In the active-active system proposed in this embodiment, IDC1 and IDC2 can represent the primary and secondary data centers, respectively.

[0062] Aspect-Oriented Programming (AOP): Its primary purpose is to add additional functionality to an operation or set of operations without modifying the source code. Features like exception handling and monitoring can be elegantly implemented using AOP, separating these additional features from the actual business logic and providing a clearer software structure.

[0063] Understandably, the traditional high-availability and disaster recovery solution for message middleware consumer clients relies on determining consumer anomalies based on the consumer instance's network heartbeat. Within a single data center, the message middleware server maintains a list of all consumer instances with normal heartbeats and can eliminate anomalies based on heartbeat detection to facilitate failover.

[0064] In a cross-data center disaster recovery scenario, such as Figure 1 As shown, traditional solutions solve the master election problem by monitoring consumer network heartbeats and combining them with distributed consistency coordination services such as Zookeeper (distributed application coordination service software) and etcd (distributed key-value storage system). By monitoring master-slave switchover events, consumer threads within different data centers can be started and shut down. For example, if a consumer in IDC1 goes down, a custom switchover service or distributed consistency service will detect the heartbeat anomaly and elect the consumer in IDC2 as the master, triggering a consumer switchover.

[0065] Currently, the number of consumers in a single consumer cluster is usually related to the number of single business application instances. A data center usually contains a large number of different businesses. Using additional customized centralized services or distributed consistency coordination services to handle customized disaster recovery strategies for a large number of clusters outside of consumer heartbeats poses performance risks.

[0066] In addition, if Figure 2 As shown, existing solutions only detect consumer anomalies at the network heartbeat level. However, if problems occur in the dependent links of consumer business processing and network isolation occurs, resulting in business processing failure, this will not affect the consumer's heartbeat reporting, will not trigger disaster recovery, and will not cause consumer backlog problems.

[0067] Based on this, in the disaster recovery method, device and active-active system provided by this embodiment, after the consumer client negotiates and agrees on the consumption switching proposal, the second consumer cluster is notified to perform consumption switching, thereby realizing disaster recovery control of the active-active system. The mechanisms of consumption status synchronization, consumption cluster switching negotiation and consumption cluster switching notification are introduced in the consumer client, so that status synchronization and consumption switching negotiation only rely on communication between consumer clients, and no additional deployment of centralized components or services other than the message middleware is required. The decentralized state synchronization and consumption switching negotiation methods are adopted to avoid potential performance problems in massive applications using centralized services or distributed consistency coordination services. There is no need for the message middleware server to handle the disaster recovery process, which avoids the performance pressure on the server caused by a large number of businesses sharing the message middleware, and can save computing resources.

[0068] Furthermore, this embodiment defines three indicators: consumption failure exceptions, consumption retry times, and consumption timeouts to determine consumption anomalies. Compared with the traditional solution that only determines consumption anomalies through network heartbeats, this solution is closer to real business scenarios, supplements the shortcomings of traditional solutions in applicable scenarios, and can cope with consumption accumulation problems caused by various reasons.

[0069] The disaster recovery method in the active-active system provided in the embodiment of the present application can be applied to Figure 3 In the application environment shown. The active-active system includes a first data center 10 (abbreviated as IDC1 in the embodiment of this application) and a second data center 20 (abbreviated as IDC2 in the embodiment of this application). The first data center 10 includes a first message middleware 101 and a first consumer cluster 102, and the second data center 20 includes a second message middleware 201 and a second consumer cluster 202. The first consumer cluster 102 is composed of multiple consumer clients 1021. The second consumer cluster 202 is composed of multiple consumer clients 2021. The consumer client 1021 is used to execute the steps of the method provided in any embodiment of this application. The consumer logic of the consumer client 2021 is the same as that of the consumer client 1021.

[0070] Optionally, the first data center 10 and the second data center 20 each include multiple consumer clusters. The first consumer cluster 102 and the second consumer cluster 202 are two consumer clusters with the same consumer cluster name. Optionally, the first data center 10 and the second data center 20 can be located in different geographical locations, forming a remote active-active system.

[0071] In an exemplary embodiment, in combination Figure 3 ,like Figure 4 As shown, a disaster recovery method in a dual-active system is provided, and the method is applied to Figure 3 Taking the consumer client 1021 in the example as an example, the following steps are included:

[0072] Step 402: Receive first synchronization data; the first synchronization data includes the status of other consumer clients in the first consumer cluster.

[0073] Data synchronization is performed regularly between any two consumer clients in the first consumer cluster. The synchronized data includes at least one of the following: consumer status, consumer switching proposal, consumer switching voting result, consumer cluster switching notification, and consumer cluster switching response. The consumer status includes the status of other consumer clients in the first consumer cluster.

[0074] Optionally, the consumer status also includes the status of each consumer client in the second consumer cluster. Data synchronization covers all consumers in the same consumer cluster in both data centers. For example, a consumer client in the IDC1 consumer cluster periodically and randomly synchronizes its consumer status (normal or abnormal) with other consumer clients in the cluster and other consumer clients in IDC2. The synchronization logic for IDC2 consumer clients is the same as that for IDC1, achieving a decentralized data synchronization approach.

[0075] In an optional implementation, referring to Figure 5 In the consumer client, the basic functional modules of the consumer client are expanded with the consumer communication module, the consumer monitoring module, and the consumer switching module. Optionally, the consumer communication module builds a network application framework based on Netty and uses a custom protocol to achieve timed data synchronization between consumer clients in the consumer cluster. This program module is introduced by the consumer code program and runs in the same process as the consumer.

[0076] In a specific implementation, the current consumer client receives the first synchronization data and determines the status of other consumer clients in the first consumer cluster, which may be a normal state or an abnormal state. The following is an example of the data protocol for the consumer status:

[0077] {

[0078] "version": 1, / / protocol version

[0079] "type": 1, / / message type, state synchronization

[0080] "clusterId": 10, / / IDC cluster ID, here IDC1 ID is 10

[0081] "consumerGroup": "groupX", / / Consumer cluster name, IDC1 and IDC2 have the same name

[0082] "consumerId": "192.……", / / Uniquely identifies the consumer client of the current cluster

[0083] "status": "normal", / / The current status of the consumer (normal / abnormal)

[0084] "timestamp": ... / / Timestamp of status sending

[0085] }

[0086] In an optional implementation, if the current consumer client newly joins the first consumer cluster or recovers from an abnormal state to a normal state, it is necessary to synchronize the state with all consumer clients in the first consumer cluster (and the second consumer cluster with the same name).

[0087] Step 404: Determine a first number of consumer clients in an abnormal state in the first consumer cluster based on the first synchronization data.

[0088] Based on the first synchronization data, the current consumer client collects the status of all consumers in the first consumer cluster and determines a first number of consumer clients in an abnormal state.

[0089] Step 406 : When the first number is greater than a first set threshold, broadcast the first consumption switching proposal in the first consumer cluster.

[0090] The first threshold is a pre-set parameter that can be adjusted based on actual circumstances. Optionally, the first threshold is half the total number of consumer clients in the first consumer cluster. If the current consumer client discovers that more than a certain number of consumer clients in the first consumer cluster are in an abnormal state (i.e., the first number is greater than the first threshold), a consumer switching proposal is triggered and the first consumer switching proposal is initiated to other consumer clients in the first consumer cluster.

[0091] In an optional implementation, to avoid repeated disaster recovery proposals, when the current consumer client detects that the first number is greater than a first set threshold, it waits for a random time and then initiates a first consumption switching proposal to other consumer clients in the first consumer cluster.

[0092] For example, referring to Figure 5 The consumer client is extended with a consumer switching module. This module triggers a consumer switching proposal when it finds that more than a certain number of consumer clients in the first consumer cluster are in an abnormal state. This program module is introduced by the consumer code program and runs in the same process as the consumer.

[0093] In an optional implementation, referring to Figure 6 Assume that a consumer cluster includes consumer 1, consumer 2, and consumer 3. The two consumers synchronize their status regularly. Consumer 1 performs the following steps: 1. Regularly monitors its own consumer status and the status of other consumers in the cluster, and synchronizes its own consumer status to consumers 2 and 3; 2. Regularly monitors whether the number of abnormal consumers in the cluster exceeds the threshold. If so, it initiates a consumption switching proposal to consumers 2 and 3.

[0094] Step 408: Receive second synchronization data; the second synchronization data includes consumption switching voting results provided by other consumer clients in the first consumer cluster.

[0095] Data synchronization is performed regularly between consumer clients. The synchronized data includes the consumer switching voting results, which include approval or rejection. The following example illustrates the data protocol for the consumer switching voting results:

[0096] {

[0097] "version": 1, / / protocol version

[0098] "type": 3, / / message type, consumption switching voting

[0099] "clusterId": 10, / / IDC cluster ID

[0100] "consumerGroup": "groupX", / / Consumer cluster name

[0101] "consumerId": "192.……", / / Uniquely identifies the consumer client of the current cluster

[0102] "result": "agree", / / Voting result (agree / reject)

[0103] "timestamp": ... / / Timestamp of when the consumer switching voting result is sent

[0104] }

[0105] Step 410: Determine a second number of consumer clients that agree to the first consumption switching proposal based on the second synchronization data.

[0106] Based on the second synchronization data, the current consumer client collects consumption switching voting results of all consumer clients in the first consumer cluster, and determines a second number of consumer clients that agree with the first consumption switching proposal.

[0107] Step 412: When the second number is greater than the second set threshold, the consumption thread is closed, and a first consumption switching instruction is sent to any consumer client in the second consumer cluster.

[0108] Among them, the second set threshold is a parameter set in advance and can be adjusted according to actual conditions. Optionally, the second set threshold is half of the total number of consumer clients included in the first consumer cluster. If the current consumer client finds that the number of consumer clients in the first consumer cluster that agree to the first consumption switching proposal is greater than a certain number (that is, the second number is greater than the second set threshold), then the cross-data center consumption switching is triggered. Specifically, the consumption thread is closed, and the first consumption switching instruction is sent to any consumer client in the second consumer cluster. Optionally, before the IDC2 consumer cluster responds to agree to the switch, each consumer client in the IDC1 consumer cluster sends a consumption switching instruction to IDC2 in a periodic cycle.

[0109] In an optional implementation, consumer client A in the first consumer cluster performs steps 402 to 406, and consumer client B in the first consumer cluster performs steps 408 to 412. Consumer client A and consumer client B are two different consumer clients in the first consumer cluster.

[0110] In the above-mentioned disaster recovery method in the active-active system, after the consumer client negotiates and agrees on the consumption switching proposal, the second consumer cluster is notified to perform the consumption switching, thereby realizing the disaster recovery control of the active-active system. Introducing the mechanisms of consumption status synchronization, consumption cluster switching negotiation, and consumption cluster switching notification in the consumer client makes it so that status synchronization and consumption switching negotiation rely only on communication between consumer clients, without the need to deploy additional centralized components or services other than the message middleware. The use of decentralized state synchronization and consumption switching negotiation avoids potential performance issues in the use of centralized services or distributed consistency coordination services under massive applications. There is no need for the message middleware server to handle the disaster recovery process, avoiding the performance pressure on the server caused by a large number of businesses sharing the message middleware, and saving computing resources.

[0111] In an exemplary embodiment, the method further includes: obtaining current operation information; judging whether a preset consumption abnormality condition is currently met based on the current operation information; if so, updating the current state to an abnormal state, and broadcasting the abnormal state to other consumer clients in the first consumer cluster.

[0112] Among them, reference Figure 5The consumer client is extended with a consumption monitoring module. Based on this consumption monitoring module, the current operation information of the current consumer client is monitored to determine whether the current consumer client meets the preset consumption abnormality conditions. If the preset consumption abnormality conditions are met, the current consumer client is set to an abnormal state and the abnormal state is broadcast to other consumer clients in the first consumer cluster. Optionally, the abnormal state is also broadcast to each consumer client in the second consumer cluster.

[0113] In an optional implementation, AOP (Aspect-Oriented Programming) is used to extend the original consumer client program, monitor the requests and responses of the consumption function, and implement a consumption monitoring module. This program module is introduced by the consumer code program and runs in the same process as the consumer. When the consumer client is started, the consumption monitoring module reads the preset consumption exception conditions pre-defined in the consumer configuration file and monitors whether the current consumer meets the preset consumption exception conditions. The implementation method of the consumption monitoring module is related to the programming language of the message middleware client. This embodiment is implemented using AOP aspect-oriented programming and is compatible with common message middleware such as Kafka, RocketMQ, and Pulsar. If the message middleware client programming language does not have native AOP support, similar functions can be implemented through decorator patterns, middleware, reflection, function wrappers, etc. The specific implementation method can be adjusted according to specific needs and technology stacks, and this embodiment does not limit this.

[0114] The current operation information is used to characterize the consumption status of the consumer client. Optionally, the current operation information includes: the number of consumption failure exceptions, the consumption retry ratio per unit time, and the consumption timeout ratio.

[0115] In this embodiment, a consumer client exception monitoring solution is provided, which can monitor whether the consumer client is abnormal and synchronize the current abnormal status to other consumer clients in a timely manner.

[0116] In an exemplary embodiment, the current operating information includes the number of consumption failure exceptions, the consumption retry ratio per unit time, and the consumption timeout ratio; if the number of consumption failure exceptions is greater than a preset threshold, the consumption retry ratio per unit time is greater than a preset ratio, and the consumption timeout ratio is greater than a preset ratio, it is determined that the preset consumption exception conditions are currently met.

[0117] Among them, the preset consumption abnormality conditions include preset thresholds, preset ratios and preset proportions. The preset thresholds, preset ratios and preset proportions refer to specific critical parameters set for different monitoring indicators, which are used to determine whether the operating status of the consumer client deviates from the normal range, thereby triggering the corresponding abnormality handling mechanism. In a specific implementation, if the number of consumption failure abnormalities is greater than the preset threshold, the consumption retry ratio per unit time is greater than the preset ratio, and the consumption timeout ratio is greater than the preset ratio, it is determined that the preset consumption abnormality conditions are currently met, the current state is updated to an abnormal state, and the abnormal state is broadcast to other consumer clients in the first consumer cluster. If the number of consumption failure abnormalities is not greater than the preset threshold, the consumption retry ratio per unit time is not greater than the preset ratio, or the consumption timeout ratio is not greater than the preset ratio, it is determined that the preset consumption abnormality conditions are not currently met, and the current state is updated to a normal state.

[0118] In this embodiment, this embodiment defines three indicators: consumption failure exception, consumption retry number, and consumption timeout to determine consumption anomalies. Compared with the traditional solution that only determines consumption anomalies through network heartbeat, this embodiment is closer to the real business scenario, supplements the shortcomings of the traditional solution in applicable scenarios, and can deal with the problem of consumption accumulation caused by various reasons.

[0119] In an exemplary embodiment, the method also includes: receiving third synchronization data; the third synchronization data includes consumption switching proposals initiated by other consumer clients in the first consumer cluster; based on the third synchronization data, verifying whether the consumption switching conditions are currently met; generating a consumption switching voting result based on the verification result; and synchronizing the generated consumption switching voting result to other consumer clients in the first consumer cluster.

[0120] The consumption switching proposal can be initiated by other consumer clients in the first consumer cluster and propagated based on the communication method between the consumer clients. The current consumer client receives the third synchronization data, responds to the consumption switching proposal initiated by the other consumer clients, verifies the monitoring data received by the current consumer client, and determines whether the consumption switching conditions are currently met.

[0121] Optionally, the current consumer client determines, based on the received monitoring data, whether more than half of the consumer clients in the first consumer cluster are in an abnormal state. If so, it confirms that the abnormal situation described in the consumer switching proposal actually exists and that the consumer switching conditions are met. Proposal negotiation ensures the accuracy of the switching proposal and avoids false alarms or erroneous operations. In specific implementations, other proposal negotiation methods may also be used, and this embodiment does not limit this.

[0122] In the specific implementation, if the consumption switching conditions are currently met, the generated consumption switching voting result is approval, and the approved consumption switching voting result is broadcast to all consumer clients in the first consumer cluster; if the switching conditions are currently not met, the generated consumption switching voting result is rejection, and the rejected consumption switching voting result is broadcast to all consumer clients in the first consumer cluster.

[0123] In an exemplary embodiment, the method further includes: if a second consumption switching instruction is received, starting a cross-data center consumption thread, and consuming the consumption data of the second message middleware based on the cross-data center consumption thread; the second consumption switching instruction is provided by any consumer client in the second consumer cluster or other consumer clients in the first consumer cluster.

[0124] Among them, when the second consumer cluster fails, by issuing a second consumption switching instruction, the first consumer cluster with the same name can be quickly switched to consume the data of the second data center to ensure business continuity.

[0125] In an optional implementation, the consumer clients in the second consumer cluster follow the same logic as steps 202 to 412 above. If it is found that more than a certain number (e.g., more than half) of the consumer clients in the second consumer cluster agree to the consumption switching proposal, cross-data center consumption switching is triggered: the consumer thread is shut down, and a second consumption switching instruction is sent to any consumer client in the first consumer cluster. The current consumer client in the first consumer cluster receives the second consumption switching instruction based on communication with any consumer client in the second consumer cluster.

[0126] In an optional implementation, other consumer clients in the first consumer cluster receive the second consumption switching instruction based on communication with any consumer client in the second consumer cluster, and then propagate the second consumption switching instruction based on communication between any two consumer clients in the first consumer cluster.

[0127] Optionally, the consumer client in the first consumer cluster receives fourth synchronization data; the fourth synchronization data includes the first consumer switching instruction or the second consumer switching instruction, i.e., a consumer cluster switching notification, which is used to notify the consumer client to close the consumer thread or start the cross-data center consumer thread. The following example illustrates the data protocol of the consumer cluster switching notification:

[0128] {

[0129] "version": 1, / / protocol version

[0130] "type": 4, / / Message type, consumer cluster switching notification

[0131] "clusterId": 10, / / IDC cluster ID

[0132] "consumerGroup": "groupX", / / Consumer cluster name

[0133] "consumerId": "192.……", / / Uniquely identifies the consumer client of the current cluster

[0134] "topic": "topicA", / / The topic of the message middleware in IDC1

[0135] "timestamp": ... / / Timestamp of status sending

[0136] }

[0137] After any consumer client in the first consumer cluster receives the second consumption switching instruction, it starts the cross-IDC consumption thread (this thread is closed by default) and consumes the message data in the IDC2 message middleware.

[0138] In an exemplary embodiment, the method further includes: determining a third number of consumer clients in the first consumer cluster that start cross-data center consumption threads; and when the third number is greater than a third set threshold, sending a consumption switching success response to any consumer client in the second consumer cluster.

[0139] The third threshold is a pre-set parameter that can be adjusted based on actual circumstances. Optionally, the third threshold is half the total number of consumer clients in the first consumer cluster. If the current consumer client detects that the number of consumer clients in the first consumer cluster that have initiated cross-data center consumption threads exceeds a certain number (i.e., the third number exceeds the third threshold), it sends a consumption switch success response to the consumer clients in the second consumer cluster.

[0140] In an optional implementation, referring to Figure 7The first consumer cluster of IDC1 includes consumers 1, 2, and 3, while the second consumer cluster of IDC2 with the same name includes consumers 4, 5, and 6. Consumer 1 notifies consumer 2 to shut down its consumption thread. After shutting down, consumer 2 sends an ACK (acknowledgement) back to consumer 1. Similarly, consumers 1 and 3 also shut down their consumption threads. Consumer 2 sends a consumption cluster switch notification (the first consumption switch instruction) to consumer 5. Consumer 5 notifies consumer 4 to start a cross-IDC consumption thread. After starting, consumer 4 sends an ACK. Similarly, consumers 5 and 6 also start cross-IDC consumption threads. Consumer 5 sends a successful message switch response back to consumer 2. The consumption thread of IDC2 consumers on the IDC1 message topic is disabled by default and is enabled after receiving the consumption cluster switch notification. After more than half of IDC2 consumers successfully start cross-IDC consumption threads, they send a successful consumption switch response to IDC1. The initiation of cross-IDC consumption threads by IDC2 consumers does not affect the existing threads consuming the IDC2 message topic.

[0141] In an optional implementation, after step 412, the method further includes: receiving fifth synchronization data; the fifth synchronization data includes a consumption switching success response; the consumption switching success response is provided by any consumer client in the second consumer cluster or other consumer clients in the first consumer cluster. The following example illustrates the data protocol of the consumption switching success response:

[0142] {

[0143] "version": 1, / / protocol version

[0144] "type": 5, / / message type, consumption switching successful response

[0145] "clusterId": 20, / / IDC cluster ID. IDC2's ID is 20, indicating that the response comes from IDC2

[0146] "consumerGroup": "groupX", / / Consumer cluster name, IDC1 and IDC2 have the same name

[0147] "consumerId": "192.……", / / Uniquely identifies the consumer client of the current cluster

[0148] "topic": "topicA", / / The topic name of the message middleware in IDC1

[0149] "status": "normal", / / The current status of the consumer (normal / abnormal)

[0150] "timestamp": ... / / Timestamp of status sending

[0151] }

[0152] In an exemplary embodiment, the method further includes: regularly sending a request to the first message middleware to obtain the network address of each consumer client in the first consumer cluster and the second consumer cluster.

[0153] Among them, since consumer clients need to know the network addresses (IP) of consumers in the consumer cluster in order to synchronize data between each other, in this embodiment, the consumer clients periodically send requests to the message middleware server to obtain the network addresses of all consumer clients in the two consumer clusters with the same name.

[0154] In an exemplary embodiment, the first synchronization data also includes the status of each consumer client in the second consumer cluster; Figure 8 As shown, the method further includes:

[0155] Step 802: Determine a fourth number of consumer clients in an abnormal state in the second consumer cluster based on the first synchronization data.

[0156] Based on the first synchronization data, the current consumer client collects the status of all consumers in the second consumer cluster and determines a fourth number of consumer clients in an abnormal state.

[0157] Step 804 : When the fourth number is greater than a fourth set threshold, broadcast a second consumption switching proposal in the first consumer cluster.

[0158] The fourth threshold value is a pre-set parameter that can be adjusted according to actual conditions. Optionally, the fourth threshold value is half of the total number of consumer clients included in the second consumer cluster. The first threshold value, the second threshold value, the third threshold value, and the fourth threshold value provided in the embodiment of the present application can be different values ​​or the same value, and this embodiment does not limit this.

[0159] If the current consumer client finds that the number of consumer clients in the second consumer cluster that are in an abnormal state is greater than a certain number (that is, the fourth number is greater than the fourth set threshold), it triggers a consumption switching proposal and initiates a consumption switching proposal to other consumer clients in the first consumer cluster.

[0160] Step 806: If the number of consumer clients in the first consumer cluster that agree to the second consumption switching proposal is greater than a second set threshold, a third consumption switching instruction is sent to any consumer client in the second consumer cluster.

[0161] Among them, after receiving the second consumption switching proposal, other consumer clients in the first consumer cluster verify whether the consumption switching conditions are currently met based on the received monitoring data, generate consumption switching voting results based on the verification results, and broadcast the consumption switching voting results.

[0162] The current consumer client receives synchronization data, which includes the consumption switching voting results provided by other consumer clients in the first consumer cluster. Based on the synchronization data, it is determined that the number of consumer clients in the first consumer cluster that agree to the second consumption switching proposal is greater than a certain number (that is, the number is greater than the second set threshold), then cross-data center consumption switching is triggered. In this embodiment, the normal first consumer cluster actively triggers the consumption switching, and notifies the second consumer cluster to perform the consumption switching through the third consumption switching instruction. Specifically, the consumer clients in the second consumer cluster are notified to close the consumption thread, and the other consumer clients in the first consumer cluster are notified to start the cross-IDC consumption thread.

[0163] It is understandable that in some extreme cases, no consumer node in the faulty consumer cluster can propose disaster recovery switching. In this embodiment, the normal consumer cluster actively proposes disaster recovery switching, which can avoid business interruption and enhance the fault tolerance capability of the active-active system.

[0164] In an exemplary embodiment, in combination Figure 6 and Figure 7 ,The disaster recovery method applied to the active-active system includes the following steps:

[0165] 1. When a consumer in the IDC1 consumer cluster detects that more than half of the consumer nodes are in an abnormal state, in order to avoid repeated disaster recovery proposals, it waits for a random period of time and then initiates a consumption switching proposal to other consumers in the cluster.

[0166] 2. After receiving the proposal, other consumers in the cluster verify the monitoring data received by this consumer. If the consumption switching conditions are met, the consumer switching voting result approving the proposal is broadcast to all consumers in the consumer cluster. Otherwise, the consumer switching voting result rejecting the proposal is broadcast to all consumers in the consumer cluster.

[0167] 3. If more than half of the consumers in the IDC1 consumer cluster agree to the consumption switching proposal, the IDC1 consumer closes its own consumption thread and sends a consumption switching instruction to a consumer in the IDC2 consumer cluster. Before the IDC2 consumer cluster responds and agrees to the switch, any IDC1 consumer will periodically send consumption switching instructions to IDC2.

[0168] 4. A consumer in the IDC2 consumer cluster receives a consumption switching instruction and broadcasts the consumption switching instruction to all consumers in the cluster.

[0169] 5. After receiving the consumption switching instruction, the IDC2 consumer starts the cross-IDC consumption thread (this thread is closed by default) and consumes the message data in the IDC1 message middleware cluster.

[0170] 6. In the IDC2 consumer cluster, if more than half of the consumers have started cross-IDC consumption threads, a consumption switching success response is sent to the consumers in IDC1.

[0171] 7. After any consumer in the IDC1 consumer cluster receives a successful consumption switching response, it stops sending consumption switching instructions periodically and broadcasts the successful consumption switching response to other consumers in the cluster.

[0172] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0173] Based on the same inventive concept, embodiments of the present application also provide a disaster recovery device in an active-active system for implementing the disaster recovery method in the active-active system described above. The implementation solution provided by this device is similar to the implementation solution described in the method described above. Therefore, the specific limitations of the disaster recovery device in one or more active-active systems provided below can be found in the limitations of the disaster recovery method in the active-active system described above and will not be repeated here.

[0174] In an exemplary embodiment, in combination Figure 1 ,like Figure 9 As shown, a disaster recovery device in a dual-active system is provided, and the device is applied to Figure 1 Taking the consumer client in the example as an example, the device includes:

[0175] The consumer communication module 902 is configured to receive first synchronization data; the first synchronization data includes the status of other consumer clients in the first consumer cluster.

[0176] The consumption switching module 904 is configured to determine a first number of consumer clients in an abnormal state in a first consumer cluster based on the first synchronization data; and broadcast a first consumption switching proposal in the first consumer cluster when the first number is greater than a first set threshold.

[0177] The consumer communication module 902 is further configured to receive second synchronization data; the second synchronization data includes consumption switching voting results provided by other consumer clients in the first consumer cluster.

[0178] The consumption switching module 904 is also used to determine the second number of consumer clients that agree to the first consumption switching proposal based on the second synchronization data; when the second number is greater than the second set threshold, close the consumption thread and send the first consumption switching instruction to any consumer client in the second consumer cluster.

[0179] In the disaster recovery device of the above-mentioned active-active system, after the consumer client negotiates and agrees on the consumption switching proposal, the second consumer cluster is notified to perform consumption switching, thereby realizing disaster recovery control of the active-active system. Introducing the mechanisms of consumption status synchronization, consumption cluster switching negotiation, and consumption cluster switching notification in the consumer client makes it so that status synchronization and consumption switching negotiation only rely on communication between consumer clients, and does not require the additional deployment of centralized components or services other than the message middleware. The use of decentralized state synchronization and consumption switching negotiation avoids potential performance issues in the use of centralized services or distributed consistency coordination services under massive applications. There is no need for the message middleware server to handle the disaster recovery process, avoiding the performance pressure on the server caused by a large number of businesses sharing the message middleware, and saving computing resources.

[0180] In an exemplary embodiment, referring to Figure 9 The device further includes a consumption monitoring module 906; the consumption monitoring module 906 is used to obtain current operation information; based on the current operation information, determine whether the preset consumption abnormality condition is currently met; if so, update the current state to an abnormal state;

[0181] The consumer communication module 902 is further configured to broadcast the abnormal status to other consumer clients in the first consumer cluster.

[0182] In an exemplary embodiment, the current operating information includes the number of consumption failure exceptions, the consumption retry ratio per unit time, and the consumption timeout ratio; if the number of consumption failure exceptions is greater than a preset threshold, the consumption retry ratio per unit time is greater than a preset ratio, and the consumption timeout ratio is greater than a preset ratio, it is determined that the preset consumption exception conditions are currently met.

[0183] In an exemplary embodiment, the consumer communication module 902 is further configured to receive third synchronization data; the third synchronization data includes a consumption switching proposal initiated by other consumer clients in the first consumer cluster;

[0184] The consumption switching module 904 is further configured to verify whether the consumption switching condition is currently met based on the third synchronization data; and generate a consumption switching voting result based on the verification result;

[0185] The consumer communication module 902 is further configured to synchronize the generated consumer switching voting result to other consumer clients in the first consumer cluster.

[0186] In an exemplary embodiment, the consumption switching module 904 is also used to start a cross-data center consumption thread if a second consumption switching instruction is received, and consume the consumption data of the second message middleware based on the cross-data center consumption thread; the second consumption switching instruction is provided by any consumer client in the second consumer cluster or other consumer clients in the first consumer cluster.

[0187] In an exemplary embodiment, the consumption switching module 904 is also used to determine the third number of consumer clients in the first consumer cluster that start cross-data center consumption threads; when the third number is greater than a third set threshold, a consumption switching success response is sent to any consumer client in the second consumer cluster.

[0188] In an exemplary embodiment, the consumer communication module 902 is further configured to periodically send a request to the first message middleware to obtain the network address of each consumer client in the first consumer cluster and the second consumer cluster.

[0189] In an exemplary embodiment, the first synchronization data also includes the status of each consumer client in the second consumer cluster; the consumption switching module 904 is also used to determine the fourth number of consumer clients in the second consumer cluster that are in an abnormal state based on the first synchronization data; when the fourth number is greater than the fourth set threshold, the second consumption switching proposal is broadcast in the first consumer cluster; if the number of consumer clients in the first consumer cluster that agree to the second consumption switching proposal is greater than the second set threshold, a third consumption switching instruction is sent to any consumer client in the second consumer cluster.

[0190] Each module in the disaster recovery device of the active-active system can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0191] In an exemplary embodiment, Figure 1 As shown, an embodiment of the present application provides an active-active system, which includes a first data center 10 and a second data center 20. The first data center 10 includes a first message middleware 101 and a first consumer cluster 102, and the second data center 20 includes a second message middleware 201 and a second consumer cluster 202; when any consumer client in the first consumer cluster 102 or the second consumer cluster 202 executes a computer program, the steps of the method described in any embodiment of the present application are implemented.

[0192] In an exemplary embodiment, when any consumer client in the first consumer cluster 102 or the second consumer cluster 202 executes a computer program, the following steps are implemented: receiving first synchronization data; the first synchronization data includes the status of other consumer clients in the first consumer cluster; based on the first synchronization data, determining a first number of consumer clients in the first consumer cluster that are in an abnormal state; when the first number is greater than a first set threshold, broadcasting a first consumption switching proposal in the first consumer cluster; receiving second synchronization data; the second synchronization data includes consumption switching voting results provided by other consumer clients in the first consumer cluster; based on the second synchronization data, determining a second number of consumer clients that agree with the first consumption switching proposal; when the second number is greater than a second set threshold, closing the consumption thread and sending a first consumption switching instruction to any consumer client in the second consumer cluster.

[0193] In an exemplary embodiment, when any consumer client in the first consumer cluster 102 or the second consumer cluster 202 executes a computer program, the following steps are implemented: obtaining current operation information; determining whether a preset consumption abnormality condition is currently satisfied based on the current operation information; if satisfied, updating the current state to an abnormal state, and broadcasting the abnormal state to other consumer clients in the first consumer cluster.

[0194] In an exemplary embodiment, when any consumer client in the first consumer cluster 102 or the second consumer cluster 202 executes a computer program, the following steps are implemented: receiving third synchronization data; the third synchronization data includes consumption switching proposals initiated by other consumer clients in the first consumer cluster; verifying whether the consumption switching conditions are currently met based on the third synchronization data; generating a consumption switching voting result based on the verification result; and synchronizing the generated consumption switching voting result to other consumer clients in the first consumer cluster.

[0195] In an exemplary embodiment, when any consumer client in the first consumer cluster 102 or the second consumer cluster 202 executes a computer program, the following steps are implemented: if a second consumption switching instruction is received, a cross-data center consumption thread is started, and consumption data of the second message middleware is consumed based on the cross-data center consumption thread; the second consumption switching instruction is provided by any consumer client in the second consumer cluster or other consumer clients in the first consumer cluster.

[0196] In an exemplary embodiment, when any consumer client in the first consumer cluster 102 or the second consumer cluster 202 executes a computer program, the following steps are implemented: determining a third number of consumer clients in the first consumer cluster that have started cross-data center consumption threads; and sending a consumption switching success response to any consumer client in the second consumer cluster when the third number is greater than a third set threshold.

[0197] In an exemplary embodiment, when any consumer client in the first consumer cluster 102 or the second consumer cluster 202 executes a computer program, the following steps are implemented: regularly sending a request to the first message middleware to obtain the network address of each consumer client in the first consumer cluster and the second consumer cluster.

[0198] In an exemplary embodiment, when any consumer client in the first consumer cluster 102 or the second consumer cluster 202 executes a computer program, the following steps are implemented: based on the first synchronization data, a fourth number of consumer clients in the second consumer cluster that are in an abnormal state is determined; when the fourth number is greater than a fourth set threshold, a second consumption switching proposal is broadcast in the first consumer cluster; if the number of consumer clients in the first consumer cluster that agree to the second consumption switching proposal is greater than the second set threshold, a third consumption switching instruction is sent to any consumer client in the second consumer cluster.

[0199] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0200] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0201] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A disaster recovery method in an active-active system, characterized in that: The active-active system includes a first data center and a second data center, the first data center includes a first message middleware and a first consumer cluster, and the second data center includes a second message middleware and a second consumer cluster; The method is applied to any consumer client in the first consumer cluster, and the method includes: Receive first synchronization data; the first synchronization data includes the status of other consumer clients in the first consumer cluster; Determining, based on the first synchronization data, a first number of consumer clients in the first consumer cluster that are in an abnormal state; When the first number is greater than a first set threshold, broadcasting a first consumption switching proposal in the first consumer cluster; Receive second synchronization data; the second synchronization data includes consumption switching voting results provided by other consumer clients in the first consumer cluster; determining, based on the second synchronization data, a second number of consumer clients that agree to the first consumption switching proposal; When the second number is greater than a second set threshold, the consumption thread is closed, and a first consumption switching instruction is sent to any consumer client in the second consumer cluster.

2. The method according to claim 1, characterized in that The method further comprises: Get current running information; Determine whether a preset consumption abnormality condition is currently met based on the current operation information; If satisfied, the current state is updated to an abnormal state, and the abnormal state is broadcasted to other consumer clients in the first consumer cluster.

3. The method according to claim 2, characterized in that The current operation information includes the number of consumption failure exceptions, the consumption retry ratio per unit time, and the consumption timeout ratio; if the number of consumption failure exceptions is greater than the preset threshold, the consumption retry ratio per unit time is greater than the preset ratio, and the consumption timeout ratio is greater than the preset ratio, it is determined that the preset consumption exception conditions are currently met.

4. The method according to claim 1, wherein The method further comprises: Receive third synchronization data; the third synchronization data includes consumption switching proposals initiated by other consumer clients in the first consumer cluster; Verifying, based on the third synchronization data, whether a consumption switching condition is currently satisfied; Generate consumption switching voting results based on the verification results; The generated consumption switching voting result is synchronized to other consumer clients in the first consumer cluster.

5. The method according to claim 1, wherein The method further comprises: If a second consumption switching instruction is received, a cross-data center consumption thread is started, and the consumption data of the second message middleware is consumed based on the cross-data center consumption thread; the second consumption switching instruction is provided by any consumer client in the second consumer cluster or other consumer clients in the first consumer cluster.

6. The method according to claim 5, characterized in that The method further comprises: Determining a third number of consumer clients that initiate cross-data center consumption threads in the first consumer cluster; When the third number is greater than a third set threshold, a consumption switching success response is sent to any consumer client in the second consumer cluster.

7. The method according to claim 1, characterized in that The method further comprises: Periodically send a request to the first message middleware to obtain the network address of each consumer client in the first consumer cluster and the second consumer cluster.

8. The method according to any one of claims 1 to 7, characterized in that The first synchronization data also includes the status of each consumer client in the second consumer cluster; the method further includes: Determining, based on the first synchronization data, a fourth number of consumer clients in the second consumer cluster that are in an abnormal state; When the fourth number is greater than a fourth set threshold, broadcasting a second consumption switching proposal in the first consumer cluster; If the number of consumer clients in the first consumer cluster that agree to the second consumption switching proposal is greater than the second set threshold, a third consumption switching instruction is sent to any consumer client in the second consumer cluster.

9. A disaster recovery device in an active-active system, characterized in that: The active-active system includes a first data center and a second data center, the first data center includes a first message middleware and a first consumer cluster, and the second data center includes a second message middleware and a second consumer cluster; The device is applied to any consumer client in the first consumer cluster, and the device includes: A consumer communication module, configured to receive first synchronization data; the first synchronization data including the status of other consumer clients in the first consumer cluster; a consumption switching module, configured to determine, based on the first synchronization data, a first number of consumer clients in an abnormal state in the first consumer cluster; and broadcast a first consumption switching proposal in the first consumer cluster when the first number is greater than a first set threshold; The consumer communication module is further configured to receive second synchronization data; the second synchronization data includes consumption switching voting results provided by other consumer clients in the first consumer cluster; The consumption switching module is also used to determine the second number of consumer clients that agree to the first consumption switching proposal based on the second synchronization data; when the second number is greater than a second set threshold, close the consumption thread and send the first consumption switching instruction to any consumer client in the second consumer cluster.

10. A dual active system, characterized in that: The active-active system includes a first data center and a second data center, the first data center includes a first message middleware and a first consumer cluster, and the second data center includes a second message middleware and a second consumer cluster; when any one consumer client in the first consumer cluster or the second consumer cluster executes a computer program, the steps of the method described in any one of claims 1 to 8 are implemented.

Images