Disaster recovery method and device in active-active system and active-active system

Abstract

The invention relates to a disaster recovery method and device in an active-active system and the active-active system. And after the consumer client negotiates to agree with the consumption switching proposal, the second consumer cluster is notified to perform consumption switching, so that disaster recovery control of the dual-active system is realized. A consumption state synchronization mechanism, a consumption cluster switching negotiation mechanism and a consumption cluster switching notification mechanism are introduced into consumer clients, so that state synchronization and consumption switching negotiation only depend on communication between the consumer clients, and centralized components or services except message middleware do not need to be additionally deployed. A decentralized state synchronization and consumption switching negotiation mode is adopted, and the potential performance problem of using centralized services or distributed consistency coordination services under mass applications is avoided. A message-oriented middleware server side does not need to process a disaster recovery process, server side performance pressure caused by a large number of services sharing the message-oriented middleware is avoided, and computing resources can be saved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a disaster recovery method, device and active-active system in an active-active system. Background Art

[0002] For high availability and disaster recovery of message middleware consumer clients, the traditional solution is to determine whether the consumer is abnormal based on the network heartbeat of the consumer instance. Within a single data center, the message middleware server maintains a list of all consumer instances with normal heartbeats, and can remove abnormal consumers based on the heartbeat detection situation to achieve failover.

[0003] In cross-data center disaster recovery scenarios, traditional solutions can solve the master election problem by monitoring consumer network heartbeats and combining distributed consistency coordination services such as zookeeper (distributed application coordination service software) and etcd (distributed key-value storage system). By monitoring master-slave switching events, consumer threads in different data centers can be turned on and off.

[0004] Currently, the number of consumers contained in a single consumer cluster is usually related to the number of a single business application instance. A data center usually contains a large number of different businesses. There is a performance risk in using additional customized centralized services or distributed consistency coordination services to handle customized disaster recovery strategies for a large number of clusters other than consumer heartbeats. Summary of the invention

[0005] Based on this, it is necessary to provide a disaster recovery method, device and active-active system in an active-active system to address the above technical problems, which can avoid potential performance problems in massive applications using centralized services or distributed consistency coordination services.

[0006] In a first aspect, the present application provides a disaster recovery method in an active-active system, the active-active system comprising a first data center and a second data center, the first data center comprising a first message middleware and a first consumer cluster, the second data center comprising a second message middleware and a second consumer cluster; the method is applied to any consumer client in the first consumer cluster, the method comprising:

[0007] Receive first synchronization data; the first synchronization data includes the status of other consumer clients in the first consumer cluster;

[0008] Determine, based on the first synchronization data, a first number of consumer clients in an abnormal state in the first consumer cluster;

[0009] When the first number is greater than a first set threshold, broadcasting a first consumption switching proposal in a first consumer cluster;

[0010] receiving second synchronization data; the second synchronization data including consumption switching voting results provided by other consumer clients in the first consumer cluster;

[0011] Determining, based on the second synchronization data, a second number of consumer clients that agree to the first consumption switching proposal;

[0012] When the second number is greater than the second set threshold, the consumption thread is closed, and a first consumption switching instruction is sent to any consumer client in the second consumer cluster.

[0013] In one embodiment, the method further comprises:

[0014] Get current running information;

[0015] According to the current operation information, determine whether the preset consumption abnormality conditions are met;

[0016] If satisfied, the current state is updated to an abnormal state, and the abnormal state is broadcast to other consumer clients in the first consumer cluster.

[0017] In one of the embodiments, the current operating information includes the number of consumption failure exceptions, the consumption retry ratio per unit time, and the consumption timeout ratio; if the number of consumption failure exceptions is greater than a preset threshold, the consumption retry ratio per unit time is greater than a preset ratio, and the consumption timeout ratio is greater than a preset ratio, it is determined that the preset consumption exception conditions are currently met.

[0018] In one embodiment, the method further comprises:

[0019] receiving third synchronization data; the third synchronization data includes consumption switching proposals initiated by other consumer clients in the first consumer cluster;

[0020] According to the third synchronization data, verify whether the consumption switching condition is currently met;

[0021] Generate consumption switching voting results based on verification results;

[0022] The generated consumption switching voting results are synchronized to other consumer clients in the first consumer cluster.

[0023] In one embodiment, the method further comprises:

[0024] If a second consumption switching instruction is received, a cross-data center consumption thread is started, and consumption data of the second message middleware is consumed based on the cross-data center consumption thread; the second consumption switching instruction is provided by any consumer client in the second consumer cluster or other consumer clients in the first consumer cluster.

[0025] In one embodiment, the method further comprises:

[0026] Determining a third number of consumer clients that start cross-data center consumption threads in the first consumer cluster;

[0027] When the third number is greater than the third set threshold, a consumption switching success response is sent to any consumer client in the second consumer cluster.

[0028] In one embodiment, the method further comprises:

[0029] Periodically send a request to the first message middleware to obtain the network address of each consumer client in the first consumer cluster and the second consumer cluster.

[0030] In one embodiment, the first synchronization data also includes the status of each consumer client in the second consumer cluster; the method further includes:

[0031] Determining, based on the first synchronization data, a fourth number of consumer clients in the second consumer cluster that are in an abnormal state;

[0032] When the fourth number is greater than a fourth set threshold, broadcasting a second consumption switching proposal in the first consumer cluster;

[0033] If the number of consumer clients in the first consumer cluster that agree to the second consumption switching proposal is greater than the second set threshold, a third consumption switching instruction is sent to any consumer client in the second consumer cluster.

[0034] In a second aspect, the present application further provides a disaster recovery device in an active-active system, the active-active system comprising a first data center and a second data center, the first data center comprising a first message middleware and a first consumer cluster, the second data center comprising a second message middleware and a second consumer cluster; the device is applied to any consumer client in the first consumer cluster, the device comprising:

[0035] A consumer communication module, configured to receive first synchronization data; the first synchronization data includes the status of other consumer clients in the first consumer cluster;

[0036] A consumption switching module, configured to determine, based on the first synchronization data, a first number of consumer clients in an abnormal state in a first consumer cluster; and broadcast a first consumption switching proposal in the first consumer cluster when the first number is greater than a first set threshold;

[0037] The consumer communication module is further used to receive second synchronization data; the second synchronization data includes consumption switching voting results provided by other consumer clients in the first consumer cluster;

[0038] The consumption switching module is also used to determine the second number of consumer clients that agree to the first consumption switching proposal based on the second synchronization data; when the second number is greater than the second set threshold, the consumption thread is closed and the first consumption switching instruction is sent to any consumer client in the second consumer cluster.

[0039] In the third aspect, the present application also provides an active-active system, which includes a first data center and a second data center, the first data center includes a first message middleware and a first consumer cluster, and the second data center includes a second message middleware and a second consumer cluster; when any consumer client in the first consumer cluster or the second consumer cluster executes a computer program, the steps of the method described in the first aspect are implemented.

[0040] The disaster recovery method, device and active-active system in the above active-active system, the active-active system includes a first data center and a second data center, the first data center includes a first message middleware and a first consumer cluster, and the second data center includes a second message middleware and a second consumer cluster; any consumer client in the first consumer cluster receives the first synchronization data; the first synchronization data includes the status of other consumer clients in the first consumer cluster; based on the first synchronization data, the first number of consumer clients in the first consumer cluster that are in an abnormal state is determined; when the first number is greater than the first set threshold, the first consumption switching proposal is broadcast in the first consumer cluster; the second synchronization data is received; the second synchronization data includes the consumption switching voting results provided by other consumer clients in the first consumer cluster; based on the second synchronization data, the second number of consumer clients that agree to the first consumption switching proposal is determined; when the second number is greater than the second set threshold, the consumption thread is closed, and the first consumption switching instruction is sent to any consumer client in the second consumer cluster. Through the above method, after the consumer client negotiates and agrees to the consumption switching proposal, the second consumer cluster is notified to perform consumption switching, thereby realizing disaster recovery control of the active-active system. Introducing the mechanisms of consumption status synchronization, consumption cluster switching negotiation, and consumption cluster switching notification in the consumer client makes the status synchronization and consumption switching negotiation rely only on the communication between consumer clients, and does not require the deployment of additional centralized components or services other than the message middleware. The decentralized state synchronization and consumption switching negotiation methods are adopted to avoid potential performance problems in massive applications using centralized services or distributed consistency coordination services. There is no need for the message middleware server to handle the disaster recovery process, avoiding the performance pressure on the server caused by a large number of businesses sharing the message middleware, which can save computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 This is a diagram of the disaster recovery process of the traditional solution;

[0043] Figure 2 Schematic diagram of the drawbacks of heartbeat detection in traditional solutions;

[0044] Figure 3 An application environment diagram of a disaster recovery method in an active-active system in one embodiment;

[0045] Figure 4 A schematic diagram of a process flow of a disaster recovery method in an active-active system in one embodiment;

[0046] Figure 5 A schematic diagram of the module structure of a consumer client in one embodiment;

[0047] Figure 6 A schematic diagram of consumer state synchronization in one embodiment;

[0048] Figure 7 A schematic diagram of consumer cluster switching in an embodiment;

[0049] Figure 8 is a flow chart of a disaster recovery method in an active-active system in another embodiment;

[0050] Fig. 9 The figure is a structural block diagram of a disaster recovery device in an active-active system in an embodiment. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first consumer cluster may be referred to as a second consumer cluster, and similarly, a second consumer cluster may be referred to as a first consumer cluster. Both the first consumer cluster and the second consumer cluster are consumer clusters, but they are not the same consumer cluster.

[0053] In order to facilitate understanding of the embodiments provided in this application, some terms that appear in the specific description process are introduced below:

[0054] Message middleware: also known as message queue (MQ), is a software system that is responsible for the transmission and exchange of messages in a distributed system. Through asynchronous message transmission, different applications, services or components can communicate and exchange data without the need for direct connection between them. MQ is also used to decouple systems and improve system reliability and scalability. Currently, the mainstream message middleware includes Kafka, Pulsar, RocketMQ, etc.

[0055] Topic: In the publish / subscribe model, the name used to classify messages, representing the classification identifier of messages stored on the message middleware server. Producers publish messages to topics, and consumers who subscribe to the topic receive messages.

[0056] Producer: An application or component responsible for creating and sending messages. Producers send messages to message queues for consumers to consume.

[0057] Consumer: An application or component that receives and processes messages from a message queue. A consumer is usually a program module in a business service instance.

[0058] Publish / Subscribe: A messaging model where producers publish messages to topics and consumers subscribe to topics to receive messages for business processing. Messages can be received by multiple consumers.

[0059] Cluster consumption: Cluster consumption is a common message consumption mode. In this mode, multiple consumers form a consumer cluster to jointly consume messages in MQ. Each consumer in the cluster is equivalent in business processing logic. Each message will only be successfully processed by one consumer in the cluster, improving the throughput and reliability of the system.

[0060] Message Backlog: refers to the situation where the number of messages in MQ keeps increasing, but the message consumers fail to process these messages in time. Message backlog may lead to system performance degradation or even message loss, causing business problems.

[0061] Internet Data Center (IDC): is a facility for centralized management of computing and storage resources. It includes servers, storage systems, network equipment, and other infrastructure to support the computing, storage, and data processing needs of an organization. In the active-active system proposed in this embodiment, IDC1 and IDC2 can represent the first data center and the second data center, respectively.

[0062] Aspect Oriented Programming (AOP): Its main function is to add additional functions to a certain operation or a group of operations without modifying the source code. Functions such as exception handling and monitoring can be implemented "elegantly" with AOP, so that these additional functions are separated from the real business logic, and the structure of the software will be clearer.

[0063] It is understandable that for high availability and disaster recovery of message middleware consumer clients, the traditional solution is to determine whether the consumer is abnormal based on the network heartbeat of the consumer instance. Within a single data center, the message middleware server maintains a list of all consumer instances with normal heartbeats, and can remove abnormal consumers based on the heartbeat detection situation to achieve failover.

[0064] In a cross-data center disaster recovery scenario, such as Figure 1 As shown in the figure, the traditional solution can solve the master selection problem by monitoring the consumer network heartbeat and combining distributed consistency coordination services such as zookeeper (distributed application coordination service software) and etcd (distributed key-value storage system). By monitoring the master-slave switching event, the consumer threads in different data centers can be turned on and off. For example, if the consumer of IDC1 goes down, the custom switching service or distributed consistency service senses the abnormal heartbeat and selects the consumer of IDC2 as the master to switch the consumer.

[0065] Currently, the number of consumers contained in a single consumer cluster is usually related to the number of a single business application instance. A data center usually contains a large number of different businesses. There is a performance risk in using additional customized centralized services or distributed consistency coordination services to handle customized disaster recovery strategies for a large number of clusters other than consumer heartbeats.

[0066] In addition, if Figure 2 As shown in the figure, the existing solution only detects consumer anomalies from the network heartbeat level. However, if there is a problem in the dependent link of consumer business processing and the network is isolated, the business processing will fail. However, it will not affect the consumer's heartbeat report, will not trigger disaster recovery, and will not cause consumer accumulation problems.

[0067] Based on this, in the disaster recovery method, device and active-active system provided in the present embodiment, after the consumer client negotiates and agrees to the consumption switching proposal, the second consumer cluster is notified to perform consumption switching, thereby realizing disaster recovery control of the active-active system. The mechanisms of consumption status synchronization, consumption cluster switching negotiation and consumption cluster switching notification are introduced in the consumer client, so that status synchronization and consumption switching negotiation only rely on communication between consumer clients, and no centralized components or services other than the message middleware need to be deployed additionally. The decentralized state synchronization and consumption switching negotiation methods are adopted to avoid potential performance problems in massive applications using centralized services or distributed consistency coordination services. There is no need for the message middleware server to handle the disaster recovery process, which avoids the performance pressure on the server caused by a large number of businesses sharing the message middleware, and can save computing resources.

[0068] Furthermore, this embodiment defines three indicators, namely, consumption failure exceptions, consumption retry times, and consumption timeouts, to determine consumption anomalies. Compared with the traditional solution that only determines consumption anomalies through network heartbeats, this solution is closer to real business scenarios, makes up for the shortcomings of traditional solutions in applicable scenarios, and can cope with consumption accumulation problems caused by various reasons.

[0069] The disaster recovery method in the active-active system provided in the embodiment of the present application can be applied to Figure 3 In the application environment shown. Among them, the active-active system includes a first data center 10 (abbreviated as IDC1 in the embodiment of the present application) and a second data center 20 (abbreviated as IDC2 in the embodiment of the present application), the first data center 10 includes a first message middleware 101 and a first consumer cluster 102, and the second data center 20 includes a second message middleware 201 and a second consumer cluster 202. The first consumer cluster 102 is composed of a plurality of consumer clients 1021. The second consumer cluster 202 is composed of a plurality of consumer clients 2021. The consumer client 1021 is used to execute the steps of the method provided in any embodiment of the present application. The consumer logic of the consumer client 2021 is the same as that of the consumer client 1021.

[0070] Optionally, the first data center 10 and the second data center 20 each include multiple consumer clusters. The first consumer cluster 102 and the second consumer cluster 202 are two consumer clusters with the same consumer cluster name. Optionally, the first data center 10 and the second data center 20 can be set in different geographical locations to form a remote active-active system.

[0071] In an exemplary embodiment, in combination Figure 3 ,like Figure 4 As shown, a disaster recovery method in a dual-active system is provided, and the method is applied to Figure 3 The consumer client 1021 in the example is used as an example to illustrate, including the following steps:

[0072] Step 402, receiving first synchronization data; the first synchronization data includes the status of other consumer clients in the first consumer cluster.

[0073] Wherein, data synchronization is performed regularly between any two consumer clients in the first consumer cluster. The synchronized data includes at least one of consumer status, consumer switching proposal, consumer switching voting result, consumer cluster switching notification, and consumer cluster switching response. The consumer status includes the status of other consumer clients in the first consumer cluster.

[0074] Optionally, the consumer status also includes the status of each consumer client in the second consumer cluster. The scope of data synchronization is all consumers in the consumer clusters with the same name in the two data centers. For example, the consumer client of the IDC1 consumer cluster periodically and randomly synchronizes its own consumer status (normal or abnormal) to other consumer clients in the cluster and other consumer clients of IDC2. The synchronization logic of the IDC2 consumer client is the same as that of IDC1, realizing a decentralized data synchronization method.

[0075] In an optional implementation, refer to Figure 5 In the consumer client, based on the basic function module of the consumer client, the consumer communication module, the consumer monitoring module and the consumer switching module are extended. Optionally, the consumer communication module builds a network application framework based on Netty, and uses a custom protocol to achieve timed data synchronization between consumer clients in the consumer cluster. The program module is introduced by the consumer code program and runs in the same process as the consumer.

[0076] In a specific implementation, the current consumer client receives the first synchronization data and determines the status of other consumer clients in the first consumer cluster, where the status includes a normal status or an abnormal status. The following is an example of the data protocol of the consumer status:

[0077] {

[0078] "version": 1, / / protocol version

[0079] "type": 1, / / message type, state synchronization

[0080] "clusterId": 10, / / IDC cluster ID, here IDC1 ID is 10

[0081] "consumerGroup": "groupX", / / Consumer cluster name, IDC1 and IDC2 are the same

[0082] "consumerId": "192……", / / Uniquely identifies the consumer client of the current cluster

[0083] "status": "normal", / / The current status of the consumer (normal / abnormal)

[0084] "timestamp": ... / / Timestamp of status sending

[0085] }

[0086] In an optional implementation, if the current consumer client newly joins the first consumer cluster or recovers from an abnormal state to a normal state, it is necessary to synchronize the state with all consumer clients in the first consumer cluster (and the second consumer cluster with the same name).

[0087] Step 404: Determine a first number of consumer clients in an abnormal state in a first consumer cluster based on the first synchronization data.

[0088] Based on the first synchronization data, the current consumer client collects the status of all consumers in the first consumer cluster and determines the first number of consumer clients in an abnormal state.

[0089] Step 406 : when the first number is greater than a first set threshold, broadcast a first consumption switching proposal in the first consumer cluster.

[0090] Among them, the first set threshold is a parameter set in advance and can be adjusted according to actual conditions. Optionally, the first set threshold is half of the total number of consumer clients included in the first consumer cluster. If the current consumer client finds that the number of consumer clients in the first consumer cluster that are in an abnormal state is greater than a certain number (that is, the first number is greater than the first set threshold), a consumption switching proposal is triggered, and the first consumption switching proposal is initiated to other consumer clients in the first consumer cluster.

[0091] In an optional implementation, to avoid repeated disaster recovery proposals, when the current consumer client detects that the first number is greater than a first set threshold, it waits for a random time and then initiates a first consumption switching proposal to other consumer clients in the first consumer cluster.

[0092] For example, refer to Figure 5 The consumer client is extended with a consumer switching module, through which a consumer switching proposal is triggered when it is found that the number of consumer clients in the first consumer cluster that are in an abnormal state exceeds a certain number. This program module is introduced by the consumer code program and runs in the same process as the consumer.

[0093] In an optional implementation, refer to Figure 6 , assuming that a consumer cluster includes consumer 1, consumer 2 and consumer 3, and the status of the two consumers is synchronized regularly, consumer 1 performs the following steps: 1. Regularly monitor its own consumer status and the status of other consumers in the cluster, and synchronize its own consumer status to consumers 2 and 3; 2. Regularly monitor whether the number of abnormal consumers in the cluster exceeds the threshold. If so, initiate a consumption switching proposal to consumers 2 and 3.

[0094] Step 408, receiving second synchronization data; the second synchronization data includes consumption switching voting results provided by other consumer clients in the first consumer cluster.

[0095] Among them, data synchronization is performed regularly between consumer clients. The synchronized data includes the consumer switching voting results, and the consumer switching voting results include approval or rejection. The following is an example of the data protocol for the consumer switching voting results:

[0096] {

[0097] "version": 1, / / protocol version

[0098] "type": 3, / / message type, consumer switch voting

[0099] "clusterId": 10, / / IDC cluster ID

[0100] "consumerGroup": "groupX", / / Consumer cluster name

[0101] "consumerId": "192……", / / Uniquely identifies the consumer client of the current cluster

[0102] "result": "agree", / / Voting result (agree / reject)

[0103] "timestamp": ... / / Timestamp of when the consumer switching voting result is sent

[0104] }

[0105] Step 410: Determine a second number of consumer clients that agree to the first consumption switching proposal based on the second synchronization data.

[0106] Based on the second synchronization data, the current consumer client collects consumption switching voting results of all consumer clients in the first consumer cluster, and determines a second number of consumer clients that agree to the first consumption switching proposal.

[0107] Step 412: When the second number is greater than the second set threshold, close the consumption thread and send a first consumption switching instruction to any consumer client in the second consumer cluster.

[0108] Among them, the second set threshold is a parameter set in advance and can be adjusted according to actual conditions. Optionally, the second set threshold is half of the total number of consumer clients included in the first consumer cluster. If the current consumer client finds that the number of consumer clients in the first consumer cluster that agree to the first consumption switching proposal is greater than a certain number (that is, the second number is greater than the second set threshold), the cross-data center consumption switching is triggered. Specifically, the consumption thread is closed, and the first consumption switching instruction is sent to any consumer client in the second consumer cluster. Optionally, before the IDC2 consumer cluster responds to agree to the switch, each consumer client in the IDC1 consumer cluster sends a consumption switching instruction to IDC2 in a periodic cycle.

[0109] In an optional implementation, the A consumer client in the first consumer cluster performs the above steps 402 to 406, and the B consumer client in the first consumer cluster performs the above steps 408 to 412. The A consumer client and the B consumer client are two different consumer clients in the first consumer cluster.

[0110] In the above-mentioned disaster recovery method in the active-active system, after the consumer client negotiates and agrees to the consumption switching proposal, the second consumer cluster is notified to perform consumption switching, thereby realizing disaster recovery control of the active-active system. The mechanisms of consumption status synchronization, consumption cluster switching negotiation, and consumption cluster switching notification are introduced in the consumer client, so that status synchronization and consumption switching negotiation only rely on communication between consumer clients, and no centralized components or services other than the message middleware need to be deployed additionally. The decentralized state synchronization and consumption switching negotiation methods are adopted to avoid potential performance problems in massive applications using centralized services or distributed consistency coordination services. There is no need for the message middleware server to handle the disaster recovery process, which avoids the performance pressure on the server caused by a large number of businesses sharing the message middleware, and can save computing resources.

[0111] In an exemplary embodiment, the method further includes: obtaining current operation information; judging whether a preset consumption abnormality condition is currently met based on the current operation information; if so, updating the current state to an abnormal state, and broadcasting the abnormal state to other consumer clients in the first consumer cluster.

[0112] Among them, refer to Figure 5The consumer client is extended with a consumption monitoring module. Based on the consumption monitoring module, the current operation information of the current consumer client is monitored to determine whether the current consumer client meets the preset consumption abnormality condition. If the preset consumption abnormality condition is met, the current consumer client is set to an abnormal state and broadcast to other consumer clients in the first consumer cluster. Optionally, the abnormal state is also broadcast to each consumer client in the second consumer cluster.

[0113] In an optional implementation, the original consumer client program is extended using AOP (aspect-oriented programming) to monitor the request and response of the consumption function and implement a consumption monitoring module. The program module is introduced by the consumer code program and runs in the same process as the consumer. When the consumer client starts, the consumption monitoring module reads the preset consumption exception conditions pre-defined in the consumer configuration file and monitors whether the current consumer meets the preset consumption exception conditions. The implementation method of the consumption monitoring module is related to the programming language of the message middleware client. This embodiment is implemented using AOP aspect-oriented programming and is compatible with common message middleware such as kafka, rocketmq, and pulsar. If the message middleware client programming language does not have native AOP support, similar functions can be implemented through decorator mode, middleware, reflection, function wrapper, etc. The specific implementation method can be adjusted according to specific needs and technology stacks, and this embodiment does not limit this.

[0114] The current operation information is used to characterize the consumption status of the consumer client. Optionally, the current operation information includes: the number of consumption failure exceptions, the consumption retry ratio per unit time, and the consumption timeout ratio.

[0115] In this embodiment, an abnormality monitoring solution for the consumer client is provided, which can monitor whether the consumer client is abnormal and synchronize the current abnormal status to other consumer clients in a timely manner.

[0116] In an exemplary embodiment, the current operating information includes the number of consumption failure exceptions, the consumption retry ratio per unit time, and the consumption timeout ratio; if the number of consumption failure exceptions is greater than a preset threshold, the consumption retry ratio per unit time is greater than a preset ratio, and the consumption timeout ratio is greater than a preset ratio, it is determined that the preset consumption exception conditions are currently met.

[0117] Among them, the preset consumption abnormality conditions include preset thresholds, preset ratios and preset proportions. The preset thresholds, preset ratios and preset proportions refer to specific critical parameters set for different monitoring indicators, which are used to determine whether the operating status of the consumer client deviates from the normal range, thereby triggering the corresponding abnormality handling mechanism. In the specific implementation, if the number of consumption failure exceptions is greater than the preset threshold, the consumption retry ratio per unit time is greater than the preset ratio, and the consumption timeout ratio is greater than the preset ratio, it is determined that the preset consumption abnormality conditions are currently met, the current state is updated to an abnormal state, and the abnormal state is broadcast to other consumer clients in the first consumer cluster. If the number of consumption failure exceptions is not greater than the preset threshold, the consumption retry ratio per unit time is not greater than the preset ratio, or the consumption timeout ratio is not greater than the preset ratio, it is determined that the preset consumption abnormality conditions are not currently met, and the current state is updated to a normal state.

[0118] In this embodiment, this embodiment defines three indicators, namely, consumption failure exception, consumption retry number, and consumption timeout, to determine consumption exception. Compared with the traditional solution that only determines consumption exception through network heartbeat, this embodiment is closer to the real business scenario, supplements the shortcomings of the traditional solution in applicable scenarios, and can cope with consumption accumulation problems caused by various reasons.

[0119] In an exemplary embodiment, the method also includes: receiving third synchronization data; the third synchronization data includes consumption switching proposals initiated by other consumer clients in the first consumer cluster; based on the third synchronization data, verifying whether the consumption switching conditions are currently met; generating a consumption switching voting result based on the verification result; and synchronizing the generated consumption switching voting result to other consumer clients in the first consumer cluster.

[0120] The consumption switching proposal may be initiated by other consumer clients in the first consumer cluster and propagated based on the communication method between the consumer clients. The current consumer client receives the third synchronization data, responds to the consumption switching proposal initiated by other consumer clients, verifies the monitoring data received by the current consumer client, and determines whether the consumption switching condition is currently met.

[0121] Optionally, the current consumer client determines whether the number of consumer clients in the first consumer cluster that are in an abnormal state exceeds half based on the received monitoring data. If so, it confirms that the abnormal situation described in the consumption switching proposal does exist and satisfies the consumption switching conditions. Through proposal negotiation, the accuracy of the switching proposal can be ensured to avoid false alarms or erroneous operations. In a specific implementation, other proposal negotiation methods can also be used, which is not limited in this embodiment.

[0122] In the specific implementation, if the consumption switching conditions are currently met, the generated consumption switching voting result is approval, and the approved consumption switching voting result is broadcast to all consumer clients in the first consumer cluster; if the switching conditions are currently not met, the generated consumption switching voting result is rejection, and the rejected consumption switching voting result is broadcast to all consumer clients in the first consumer cluster.

[0123] In an exemplary embodiment, the method also includes: if a second consumption switching instruction is received, starting a cross-data center consumption thread, and consuming consumption data of the second message middleware based on the cross-data center consumption thread; the second consumption switching instruction is provided by any consumer client in the second consumer cluster or other consumer clients in the first consumer cluster.

[0124] Among them, when the second consumer cluster fails, by issuing a second consumer switching instruction, the first consumer cluster with the same name can be quickly switched to consume the data of the second data center to ensure business continuity.

[0125] In an optional implementation, the consumer clients in the second consumer cluster follow the same logic as the aforementioned steps 202 to 412, and when it is found that the number of consumer clients in the second consumer cluster that agree to the consumption switching proposal is greater than a certain number (for example, more than half), the consumption switching across data centers is triggered: the consumption thread is closed, and a second consumption switching instruction is sent to any consumer client in the first consumer cluster. The current consumer client in the first consumer cluster receives the second consumption switching instruction based on the communication with any consumer client in the second consumer cluster.

[0126] In an optional implementation, other consumer clients in the first consumer cluster receive the second consumption switching instruction based on communication with any consumer client in the second consumer cluster, and then propagate the second consumption switching instruction based on communication between any two consumer clients in the first consumer cluster.

[0127] Optionally, the consumer client in the first consumer cluster receives the fourth synchronization data; the fourth synchronization data includes the first consumption switching instruction or the second consumption switching instruction, that is, the consumption cluster switching notification, which is used to notify the consumer client to close the consumption thread or start the cross-data center consumption thread. The following is an example of the data protocol of the consumption cluster switching notification:

[0128] {

[0129] "version": 1, / / protocol version

[0130] "type": 4, / / message type, consumer cluster switching notification

[0131] "clusterId": 10, / / IDC cluster ID

[0132] "consumerGroup": "groupX", / / Consumer cluster name

[0133] "consumerId": "192……", / / Uniquely identifies the consumer client of the current cluster

[0134] "topic": "topicA", / / The topic of the message middleware in IDC1

[0135] "timestamp": ... / / Timestamp of status sending

[0136] }

[0137] After any consumer client in the first consumer cluster receives the second consumption switching instruction, it starts the cross-IDC consumption thread (this thread is closed by default) and consumes the message data in the IDC2 message middleware.

[0138] In an exemplary embodiment, the method also includes: determining a third number of consumer clients in the first consumer cluster that start cross-data center consumption threads; when the third number is greater than a third set threshold, sending a consumption switching success response to any consumer client in the second consumer cluster.

[0139] The third set threshold is a parameter set in advance and can be adjusted according to actual conditions. Optionally, the third set threshold is half of the total number of consumer clients included in the first consumer cluster. If the current consumer client finds that the number of consumer clients that start cross-data center consumption threads in the first consumer cluster is greater than a certain number (that is, the third number is greater than the third set threshold), it sends a consumption switching success response to the consumer clients in the second consumer cluster.

[0140] In an optional implementation, refer to Figure 7, the first consumer cluster of IDC1 includes consumers 1, 2 and 3, and the second consumer cluster of IDC2 with the same name includes consumers 4, 5 and 6. Among them, consumer 1 notifies consumer 2 to close the consumption thread. After consumer 2 closes, it feedbacks ACK (Acknowledgement) to consumer 1. Similarly, consumers 1 and 3 both close the consumption thread. Consumer 2 initiates a consumption cluster switching notification to consumer 5 (i.e., the first consumption switching instruction). Consumer 5 notifies consumer 4 to start a cross-IDC consumption thread. After consumer 4 starts, it feedbacks ACK. Similarly, consumers 5 and 6 both start cross-IDC consumption threads. Consumer 5 feedbacks a successful message switching response to consumer 2. Among them, the consumption thread of IDC2 consumers on the IDC1 message topic is closed by default, and is opened after receiving the consumption cluster switching notification; after more than half of the consumers of IDC2 successfully start the cross-IDC consumption thread, a successful consumption switching response is sent to IDC1; IDC2 consumers start the cross-IDC consumption thread, which does not affect the original thread's consumption of the IDC2 message topic.

[0141] In an optional implementation, after step 412, the method further includes: receiving fifth synchronization data; the fifth synchronization data includes a consumption switching success response; the consumption switching success response is provided by any consumer client in the second consumer cluster or other consumer clients in the first consumer cluster. The following is an example of a data protocol for a consumption switching success response:

[0142] {

[0143] "version": 1, / / protocol version

[0144] "type": 5, / / message type, consumption switching successful response

[0145] "clusterId": 20, / / IDC cluster ID. The ID of IDC2 is 20, indicating that the response comes from IDC2

[0146] "consumerGroup": "groupX", / / Consumer cluster name, IDC1 and IDC2 are the same

[0147] "consumerId": "192……", / / Uniquely identifies the consumer client of the current cluster

[0148] "topic": "topicA", / / The topic name of the message middleware in IDC1

[0149] "status": "normal", / / The current status of the consumer (normal / abnormal)

[0150] "timestamp": ... / / Timestamp of status sending

[0151] }

[0152] In an exemplary embodiment, the method further includes: regularly sending a request to the first message middleware to obtain the network address of each consumer client in the first consumer cluster and the second consumer cluster.

[0153] Among them, since the consumer clients need to know the network addresses (IP) of consumers in the consumer cluster to synchronize data between each other, in this embodiment, the consumer clients periodically send requests to the message middleware server to obtain the network addresses of all consumer clients in the two consumer clusters with the same name.

[0154] In an exemplary embodiment, the first synchronization data also includes the status of each consumer client in the second consumer cluster; Figure 8 As shown, the method also includes:

[0155] Step 802: Determine a fourth number of consumer clients in an abnormal state in the second consumer cluster based on the first synchronization data.

[0156] Based on the first synchronization data, the current consumer client collects the status of all consumers in the second consumer cluster and determines a fourth number of consumer clients in an abnormal state.

[0157] Step 804: when the fourth number is greater than a fourth set threshold, broadcast a second consumption switching proposal in the first consumer cluster.

[0158] Among them, the fourth set threshold is a parameter set in advance and can be adjusted according to actual conditions. Optionally, the fourth set threshold is half of the total number of consumer clients included in the second consumer cluster. The first set threshold, the second set threshold, the third set threshold, and the fourth set threshold provided in the embodiment of the present application can be different values ​​or the same value, and this embodiment does not limit this.

[0159] If the current consumer client finds that the number of consumer clients in the second consumer cluster that are in an abnormal state is greater than a certain number (that is, the fourth number is greater than the fourth set threshold), a consumption switching proposal is triggered, and a consumption switching proposal is initiated to other consumer clients in the first consumer cluster.

[0160] Step 806: If the number of consumer clients in the first consumer cluster that agree to the second consumption switching proposal is greater than the second set threshold, a third consumption switching instruction is sent to any consumer client in the second consumer cluster.

[0161] Among them, after receiving the second consumption switching proposal, other consumer clients in the first consumer cluster verify whether the consumption switching conditions are currently met based on the received monitoring data, generate consumption switching voting results based on the verification results, and broadcast the consumption switching voting results.

[0162] The current consumer client receives synchronization data, which includes the consumption switching voting results provided by other consumer clients in the first consumer cluster. According to the synchronization data, it is determined that the number of consumer clients in the first consumer cluster that agree to the second consumption switching proposal is greater than a certain number (that is, the number is greater than the second set threshold), then the cross-data center consumption switching is triggered. In this embodiment, the normal first consumer cluster actively triggers the consumption switching, and notifies the second consumer cluster to perform the consumption switching through the third consumption switching instruction. Specifically, the consumer clients in the second consumer cluster are notified to close the consumption thread, and other consumer clients in the first consumer cluster are notified to start the cross-IDC consumption thread.

[0163] It is understandable that in some extreme cases, no consumer node in the faulty consumer cluster can propose disaster recovery switching. In this embodiment, the normal consumer cluster actively proposes disaster recovery switching, which can avoid business interruption and enhance the fault tolerance capability of the active-active system.

[0164] In an exemplary embodiment, in combination Figure 6 and Figure 7 ,The disaster recovery method applied to the active-active system includes the following steps:

[0165] 1. When a consumer in the IDC1 consumer cluster collects that more than half of the consumer nodes are in an abnormal state, in order to avoid repeated disaster recovery proposals, it waits for a random time and then initiates a consumption switching proposal to other consumers in the cluster.

[0166] 2. After receiving the proposal, other consumers in the cluster verify the monitoring data received by this consumer. If the consumption switching conditions are met, the consumption switching voting results of approving the proposal are broadcast to all consumers in the consumer cluster. Otherwise, the consumption switching voting results of rejecting the proposal are broadcast to all consumers in the consumer cluster.

[0167] 3. If more than half of the consumers in the IDC1 consumer cluster agree to the consumption switching proposal, the IDC1 consumer closes its own consumption thread and sends a consumption switching instruction to a consumer in the IDC2 consumer cluster. Before the IDC2 consumer cluster responds and agrees to the switch, any IDC1 consumer sends a consumption switching instruction to IDC2 in a regular loop.

[0168] 4. A consumer in the IDC2 consumer cluster receives a consumption switching instruction and broadcasts the consumption switching instruction to all consumers in the cluster.

[0169] 5. After receiving the consumption switching instruction, the IDC2 consumer starts the cross-IDC consumption thread (this thread is closed by default) and consumes the message data in the IDC1 message middleware cluster.

[0170] 6. In the IDC2 consumer cluster, if more than half of the consumers have started cross-IDC consumption threads, a consumption switching success response is sent to the consumers in IDC1.

[0171] 7. After any consumer in the IDC1 consumer cluster receives a successful consumption switching response, it stops sending consumption switching instructions at regular intervals and broadcasts a successful consumption switching response to other consumers in the cluster.

[0172] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0173] Based on the same inventive concept, the embodiment of the present application also provides a disaster recovery device in an active-active system for implementing the disaster recovery method in the active-active system mentioned above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of the disaster recovery device in one or more active-active systems provided below can refer to the limitations of the disaster recovery method in the active-active system above, and will not be repeated here.

[0174] In an exemplary embodiment, in combination Figure 1 ,like Fig. 9 As shown, a disaster recovery device in a dual-active system is provided, and the device is applied to Figure 1 Taking the consumer client in the example as an example, the device includes:

[0175] The consumer communication module 902 is used to receive first synchronization data; the first synchronization data includes the status of other consumer clients in the first consumer cluster.

[0176] The consumption switching module 904 is used to determine the first number of consumer clients in an abnormal state in the first consumer cluster based on the first synchronization data; when the first number is greater than a first set threshold, broadcast a first consumption switching proposal in the first consumer cluster.

[0177] The consumer communication module 902 is further configured to receive second synchronization data; the second synchronization data includes consumption switching voting results provided by other consumer clients in the first consumer cluster.

[0178] The consumption switching module 904 is also used to determine the second number of consumer clients that agree to the first consumption switching proposal based on the second synchronization data; when the second number is greater than the second set threshold, close the consumption thread and send the first consumption switching instruction to any consumer client in the second consumer cluster.

[0179] In the disaster recovery device in the above-mentioned active-active system, after the consumer client negotiates and agrees to the consumption switching proposal, the second consumer cluster is notified to perform consumption switching, thereby realizing disaster recovery control of the active-active system. The mechanisms of consumption status synchronization, consumption cluster switching negotiation, and consumption cluster switching notification are introduced in the consumer client, so that status synchronization and consumption switching negotiation only rely on communication between consumer clients, and no centralized components or services other than the message middleware need to be deployed additionally. The decentralized state synchronization and consumption switching negotiation methods are adopted to avoid potential performance problems in massive applications using centralized services or distributed consistency coordination services. There is no need for the message middleware server to handle the disaster recovery process, which avoids the performance pressure on the server caused by a large number of businesses sharing the message middleware, and can save computing resources.

[0180] In an exemplary embodiment, referring to Fig. 9 , the device also includes a consumption monitoring module 906; the consumption monitoring module 906 is used to obtain current operation information; according to the current operation information, determine whether the preset consumption abnormality condition is currently met; if so, update the current state to an abnormal state;

[0181] The consumer communication module 902 is further configured to broadcast the abnormal status to other consumer clients in the first consumer cluster.

[0182] In an exemplary embodiment, the current operating information includes the number of consumption failure exceptions, the consumption retry ratio per unit time, and the consumption timeout ratio; if the number of consumption failure exceptions is greater than a preset threshold, the consumption retry ratio per unit time is greater than a preset ratio, and the consumption timeout ratio is greater than a preset ratio, it is determined that the preset consumption exception conditions are currently met.

[0183] In an exemplary embodiment, the consumer communication module 902 is further configured to receive third synchronization data; the third synchronization data includes a consumption switching proposal initiated by other consumer clients in the first consumer cluster;

[0184] The consumption switching module 904 is further used to verify whether the consumption switching condition is currently met according to the third synchronization data; and generate a consumption switching voting result according to the verification result;

[0185] The consumer communication module 902 is further used to synchronize the generated consumer switching voting result to other consumer clients in the first consumer cluster.

[0186] In an exemplary embodiment, the consumption switching module 904 is also used to start a cross-data center consumption thread if a second consumption switching instruction is received, and consume the consumption data of the second message middleware based on the cross-data center consumption thread; the second consumption switching instruction is provided by any consumer client in the second consumer cluster or other consumer clients in the first consumer cluster.

[0187] In an exemplary embodiment, the consumption switching module 904 is also used to determine the third number of consumer clients that start cross-data center consumption threads in the first consumer cluster; when the third number is greater than a third set threshold, a consumption switching success response is sent to any consumer client in the second consumer cluster.

[0188] In an exemplary embodiment, the consumer communication module 902 is further configured to periodically send a request to the first message middleware to obtain the network address of each consumer client in the first consumer cluster and the second consumer cluster.

[0189] In an exemplary embodiment, the first synchronization data also includes the status of each consumer client in the second consumer cluster; the consumption switching module 904 is also used to determine the fourth number of consumer clients in the second consumer cluster that are in an abnormal state based on the first synchronization data; when the fourth number is greater than the fourth set threshold, broadcast the second consumption switching proposal in the first consumer cluster; if the number of consumer clients in the first consumer cluster that agree to the second consumption switching proposal is greater than the second set threshold, send a third consumption switching instruction to any consumer client in the second consumer cluster.

[0190] Each module in the disaster recovery device in the above active-active system can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0191] In an exemplary embodiment, Figure 1 As shown, an embodiment of the present application provides an active-active system, which includes a first data center 10 and a second data center 20, the first data center 10 includes a first message middleware 101 and a first consumer cluster 102, and the second data center 20 includes a second message middleware 201 and a second consumer cluster 202; when any consumer client in the first consumer cluster 102 or the second consumer cluster 202 executes a computer program, the steps of the method described in any embodiment of the present application are implemented.

[0192] In an exemplary embodiment, when any consumer client in the first consumer cluster 102 or the second consumer cluster 202 executes a computer program, the following steps are implemented: receiving first synchronization data; the first synchronization data includes the status of other consumer clients in the first consumer cluster; based on the first synchronization data, determining a first number of consumer clients in the first consumer cluster that are in an abnormal state; when the first number is greater than a first set threshold, broadcasting a first consumption switching proposal in the first consumer cluster; receiving second synchronization data; the second synchronization data includes consumption switching voting results provided by other consumer clients in the first consumer cluster; based on the second synchronization data, determining a second number of consumer clients that agree with the first consumption switching proposal; when the second number is greater than a second set threshold, closing the consumption thread and sending a first consumption switching instruction to any consumer client in the second consumer cluster.

[0193] In an exemplary embodiment, when any consumer client in the first consumer cluster 102 or the second consumer cluster 202 executes a computer program, the following steps are implemented: obtaining current operation information; judging whether a preset consumption abnormality condition is currently met based on the current operation information; if so, updating the current state to an abnormal state, and broadcasting the abnormal state to other consumer clients in the first consumer cluster.

[0194] In an exemplary embodiment, when any consumer client in the first consumer cluster 102 or the second consumer cluster 202 executes a computer program, the following steps are implemented: receiving third synchronization data; the third synchronization data includes consumption switching proposals initiated by other consumer clients in the first consumer cluster; based on the third synchronization data, verifying whether the consumption switching conditions are currently met; generating a consumption switching voting result based on the verification result; and synchronizing the generated consumption switching voting result to other consumer clients in the first consumer cluster.

[0195] In an exemplary embodiment, when any consumer client in the first consumer cluster 102 or the second consumer cluster 202 executes a computer program, the following steps are implemented: if a second consumption switching instruction is received, a cross-data center consumption thread is started, and consumption data of the second message middleware is consumed based on the cross-data center consumption thread; the second consumption switching instruction is provided by any consumer client in the second consumer cluster or other consumer clients in the first consumer cluster.

[0196] In an exemplary embodiment, when any consumer client in the first consumer cluster 102 or the second consumer cluster 202 executes a computer program, the following steps are implemented: determining a third number of consumer clients in the first consumer cluster that start cross-data center consumption threads; and sending a consumption switching success response to any consumer client in the second consumer cluster when the third number is greater than a third set threshold.

[0197] In an exemplary embodiment, when any consumer client in the first consumer cluster 102 or the second consumer cluster 202 executes a computer program, the following steps are implemented: regularly sending a request to the first message middleware to obtain the network address of each consumer client in the first consumer cluster and the second consumer cluster.

[0198] In an exemplary embodiment, when any consumer client in the first consumer cluster 102 or the second consumer cluster 202 executes a computer program, the following steps are implemented: based on the first synchronization data, a fourth number of consumer clients in the second consumer cluster that are in an abnormal state is determined; when the fourth number is greater than a fourth set threshold, a second consumption switching proposal is broadcast in the first consumer cluster; if the number of consumer clients in the first consumer cluster that agree to the second consumption switching proposal is greater than the second set threshold, a third consumption switching instruction is sent to any consumer client in the second consumer cluster.

[0199] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0200] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0201] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A disaster recovery method in an active-active system, characterized in that: The active-active system includes a first data center and a second data center, the first data center includes a first message middleware and a first consumer cluster, and the second data center includes a second message middleware and a second consumer cluster; The method is applied to any consumer client in the first consumer cluster, and the method includes: Receive first synchronization data; the first synchronization data includes the status of other consumer clients in the first consumer cluster; Determine, based on the first synchronization data, a first number of consumer clients in the first consumer cluster that are in an abnormal state; When the first number is greater than a first set threshold, broadcasting a first consumption switching proposal in the first consumer cluster; receiving second synchronization data; the second synchronization data including consumption switching voting results provided by other consumer clients in the first consumer cluster; Determining, based on the second synchronization data, a second number of consumer clients that agree to the first consumption switching proposal; When the second number is greater than a second set threshold, the consumption thread is closed, and a first consumption switching instruction is sent to any consumer client in the second consumer cluster.

2. The method according to claim 1, characterized in that: The method further comprises: Get current running information; According to the current operation information, determining whether a preset consumption abnormality condition is currently met; If satisfied, the current state is updated to an abnormal state, and the abnormal state is broadcasted to other consumer clients in the first consumer cluster.

3. The method according to claim 2, characterized in that The current operation information includes the number of consumption failure exceptions, the consumption retry ratio per unit time, and the consumption timeout ratio; if the number of consumption failure exceptions is greater than a preset threshold, the consumption retry ratio per unit time is greater than a preset ratio, and the consumption timeout ratio is greater than a preset ratio, it is determined that the preset consumption exception conditions are currently met.

4. The method according to claim 1, characterized in that The method further comprises: receiving third synchronization data; the third synchronization data including consumption switching proposals initiated by other consumer clients in the first consumer cluster; Verifying, according to the third synchronization data, whether a consumption switching condition is currently met; Generate consumption switching voting results based on verification results; The generated consumption switching voting result is synchronized to other consumer clients in the first consumer cluster.

5. The method according to claim 1, characterized in that The method further comprises: If a second consumption switching instruction is received, a cross-data center consumption thread is started, and consumption data of the second message middleware is consumed based on the cross-data center consumption thread; the second consumption switching instruction is provided by any consumer client in the second consumer cluster or other consumer clients in the first consumer cluster.

6. The method according to claim 5, characterized in that The method further comprises: Determining a third number of consumer clients that start cross-data center consumption threads in the first consumer cluster; When the third number is greater than a third set threshold, a consumption switching success response is sent to any consumer client in the second consumer cluster.

7. The method according to claim 1, characterized in that The method further comprises: Periodically send a request to the first message middleware to obtain the network address of each consumer client in the first consumer cluster and the second consumer cluster.

8. The method according to any one of claims 1 to 7, characterized in that The first synchronization data also includes the status of each consumer client in the second consumer cluster; the method also includes: Determining, based on the first synchronization data, a fourth number of consumer clients in the second consumer cluster that are in an abnormal state; When the fourth number is greater than a fourth set threshold, broadcasting a second consumption switching proposal in the first consumer cluster; If the number of consumer clients in the first consumer cluster that agree to the second consumption switching proposal is greater than the second set threshold, a third consumption switching instruction is sent to any consumer client in the second consumer cluster.

9. A disaster recovery device in an active-active system, characterized in that: The active-active system includes a first data center and a second data center, the first data center includes a first message middleware and a first consumer cluster, and the second data center includes a second message middleware and a second consumer cluster; The device is applied to any consumer client in the first consumer cluster, and the device includes: A consumer communication module, configured to receive first synchronization data; the first synchronization data includes the status of other consumer clients in the first consumer cluster; A consumption switching module, configured to determine, based on the first synchronization data, a first number of consumer clients in an abnormal state in the first consumer cluster; and broadcast a first consumption switching proposal in the first consumer cluster when the first number is greater than a first set threshold; The consumer communication module is further used to receive second synchronization data; the second synchronization data includes consumption switching voting results provided by other consumer clients in the first consumer cluster; The consumption switching module is also used to determine the second number of consumer clients that agree to the first consumption switching proposal based on the second synchronization data; when the second number is greater than a second set threshold, close the consumption thread and send the first consumption switching instruction to any consumer client in the second consumer cluster.

10. A dual active system, characterized in that: The active-active system includes a first data center and a second data center, the first data center includes a first message middleware and a first consumer cluster, and the second data center includes a second message middleware and a second consumer cluster; when any one consumer client in the first consumer cluster or the second consumer cluster executes a computer program, the steps of the method described in any one of claims 1 to 8 are implemented.

Images