Active-active system of service system

Abstract

The invention discloses an active-active system of a service system. Comprising a main data center and a standby data center, and the main data center and the standby data center are respectively deployed with a business application, a Redis cluster, a Kafka cluster, a business library and a business calculation application; the main data center and the standby data center are respectively connected with the terminal; the two business libraries are connected; the two Redis clusters are connected; the two Kafka clusters are connected with each other; the Redis cluster and the Kafka cluster of the main data center are respectively connected with the business computing application of the main data center; the business calculation application of the main data center is connected with the business library and the business application respectively; the Redis cluster and the Kafka cluster of the standby data center are respectively connected with the service computing application; the service computing application of the standby data center is connected with the service library and the service application. By implementing the method provided by the invention, the availability, elasticity and resource utilization rate of the service system can be effectively improved.

Description

Technical Field

[0001] The present invention relates to a business system architecture, and more specifically to an active-active system of a business system. Background Art

[0002] In traditional business systems, disaster recovery architecture is usually implemented by deploying a set of independent applications, databases, and middleware in the primary and backup data centers. The basic process of this architecture is as follows: The primary data center is responsible for carrying daily business, while the backup data center is used as a disaster recovery backup. It usually does not provide services to the outside world and is mainly used to receive data from the primary data center.

[0003] The real-time data synchronization between the primary and standby databases and the middleware is achieved through the database's own synchronization tools or some open source components such as Oracle OGG, Debezium, etc. This synchronization ensures that the database in the standby data center can maintain consistency with the database in the primary data center.

[0004] Under normal circumstances, all business requests are processed by the primary data center, and the backup data center only serves as a data receiver. When the primary data center fails or goes down, the system will perform a failover and switch the business request to the backup data center to ensure business continuity.

[0005] However, this traditional disaster recovery architecture has a serious waste of computing and storage resources because the backup data center is idle most of the time and is only used to receive data. When a failure occurs, it may take a certain amount of time to switch to the backup environment, which will cause business interruption and a decline in user experience. In particular, when a failure occurs, business continuity cannot be effectively guaranteed. The traditional architecture has a single point of failure risk during the switching process. The backup data center will not be activated until the primary data center fails. This delay may affect the availability of the system.

[0006] Therefore, it is necessary to design a new system to effectively improve the availability, elasticity and resource utilization of the business system and achieve true business continuity and disaster recovery capabilities. Summary of the invention

[0007] The purpose of the present invention is to overcome the defects of the prior art and provide an active-active system for a business system.

[0008] To achieve the above-mentioned objectives, the present invention adopts the following technical scheme: a dual-active system of a business system, comprising a main data center and a standby data center, wherein the main data center and the standby data center are respectively deployed with business applications, Redis clusters, Kafka clusters, business libraries and business computing applications for data synchronization management; the main data center and the standby data center are respectively connected to the terminal; the business library of the main data center is connected to the business library of the standby data; the Redis cluster of the main data center is connected to the Redis cluster of the standby data; the Kafka cluster of the main data center is connected to the Kafka cluster of the standby data; the Redis cluster and the Kafka cluster of the main data center are respectively connected to the business computing application of the main data center; the business computing application of the main data center is respectively connected to the business library and the business application of the main data center; the Redis cluster and the Kafka cluster of the standby data center are respectively connected to the business computing application of the standby data center; the business computing application of the standby data center is respectively connected to the business library and the business application of the standby data center;

[0009] A further technical solution is: the business application is used for status monitoring, data activation and disaster recovery switching between the main data center and the backup data center.

[0010] Its further technical solution is: the business application is used to monitor the status of the Kafka cluster and Redis cluster of the main data center and the backup data center in real time, and make exception judgments according to preset rules to achieve dynamic switching and customized cache logic. If the main data center is restored, the data synchronization component is started through a script to perform full or incremental data synchronization, and automatically restart when the data synchronization component is abnormal to ensure continuous availability of the service.

[0011] Its further technical solution is: the business application is used to achieve active-active deployment across computer rooms through a soft load cluster combined with Keepalived and Nginx, and introduce global load balancing for intelligent domain name resolution; users access the business system through the Xintong DNS domain name, provide two-way services for the production environment and the disaster recovery environment, and automatically switch traffic in the event of a computer room disaster.

[0012] Its further technical solution is: the business computing application is used to connect to the message queue of the Kafka cluster and the cache library of the Redis cluster through the internal DNS according to the instructions issued by the business application, and access the business library to perform data warehousing operations at the same time.

[0013] Its further technical solution is: the business library includes an Oracle database.

[0014] Its further technical solution is: the two Redis clusters are bidirectionally synchronized through Redis-Shake, and when the production environment fails over to the backup data center, the Redis cluster of the backup data center adds a KEY with the characteristics of the primary data center to continue consuming data.

[0015] Its further technical solution is: the two Redis clusters use filtering LUA scripts and provide data caching capabilities at the same time when no abnormal situation occurs; when a problem occurs in the primary data center, the Redis whitelist of the backup data center will add a KEY with the characteristics of the primary data center; after the primary data center is restored, the backup data center will synchronize the full or incremental data to the primary data center through Redis-Shake.

[0016] Its further technical solution is: the two Kafka clusters perform real-time bidirectional data synchronization at the topic granularity through MirrorMaker2, and synchronize consumption submission pointers regularly. When the production environment fails and switches to the disaster recovery environment, the unconsumed data continues to be consumed.

[0017] Its further technical solution is: when the two Kafka clusters are working, a computer room prefix is ​​added before the Topic name; when the primary data center fails, the business computing application of the backup data center adds a subscription to the Topic synchronized with the primary data center, and after the primary data center is restored, the unconsumed data of the backup data center is synchronized to the primary data center.

[0018] The beneficial effects of the present invention compared with the prior art are as follows: the present invention deploys business applications, Redis clusters, Kafka clusters and business libraries through the main data center and the backup data center respectively to achieve real-time data synchronization and high availability. The two data centers are interconnected through the business library, Redis and Kafka to ensure data consistency and fast access, while supporting terminal connection. The flexible configuration and dynamic resource management of business computing applications between the main and backup data centers improves the elasticity and resource utilization of the system. It effectively improves the availability, elasticity and resource utilization of the business system and realizes true business continuity and disaster recovery capabilities.

[0019] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0021] Figure 1A schematic block diagram of a dual-active system of a business system provided by an embodiment of the present invention; Figure 2 A schematic diagram of the process of Redis cross-cluster bidirectional data synchronization provided by an embodiment of the present invention; Figure 3 A schematic diagram of the process of Kafka cross-cluster bidirectional data synchronization provided by an embodiment of the present invention; Figure 4 A schematic diagram of a workflow of a business application provided by an embodiment of the present invention; Figure 5 A schematic diagram of a workflow of a business application provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0024] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0025] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0026] See also Figure 1 , Figure 1 A schematic block diagram of a dual-active system of a business system provided by an embodiment of the present invention; solves the problem of idle computing resources in the backup data center computer room, improves resource utilization, and realizes a dual-active architecture of applications, databases, and middleware, which can reduce the long-term unavailability of business caused by failures in the primary data center, thereby improving business continuity See also Figure 1The above-mentioned active-active system of a business system includes a main data center 10 and a standby data center 20, and the main data center 10 and the standby data center 20 are respectively deployed with business applications, Redis clusters, Kafka clusters, business libraries and business computing applications for data synchronization management; the main data center 10 and the standby data center 20 are respectively connected to the terminal; the business library of the main data center 10 is connected to the business library of the standby data; the Redis cluster of the main data center 10 is connected to the Redis cluster of the standby data; the Kafka cluster of the main data center 10 is connected to the Kafka cluster of the standby data; the Redis cluster and Kafka cluster of the main data center 10 are respectively connected to the business computing application of the main data center 10; the business computing application of the main data center 10 is respectively connected to the business library and business application of the main data center 10; the Redis cluster and Kafka cluster of the standby data center 20 are respectively connected to the business computing application of the standby data center 20; the business computing application of the standby data center 20 is respectively connected to the business library and business application of the standby data center 20.

[0027] Specifically, the active-active system architecture includes a primary data center 10 and a backup data center 20, each of which deploys various components required for data synchronization management, such as business applications, Redis clusters, Kafka clusters, business libraries, and business computing applications.

[0028] The main data center 10 and the backup data center 20 achieve real-time data sharing through data synchronization management. Connections are established between the business libraries, Redis clusters, and Kafka clusters in the two locations to ensure data consistency and availability. The data centers in the two locations are connected to end users respectively, and user requests can be processed by any data center to ensure high availability of the business.

[0029] Business computing applications process user requests and perform data operations by connecting to business libraries and business applications.

[0030] Redis cluster is used to cache commonly used data and improve access speed.

[0031] Kafka clusters are used for messaging, supporting asynchronous processing and real-time data flow.

[0032] The business computing applications of the backup data center 20 are also connected to their own business libraries and business applications, ensuring that the business can be quickly taken over when the primary data center 10 fails.

[0033] When a failure occurs in the primary data center 10, the backup data center 20 can immediately switch to the primary service node, using the backup business applications and data processing capabilities to ensure business continuity and availability.

[0034] The active-active architecture ensures that the business can continue to run regardless of failures in the primary data center 10 or the backup data center 20, significantly improving the availability of the system. Through reasonable configuration and synchronization management, idle resources are avoided and the utilization rate of computing and storage resources is improved. Through real-time data synchronization management, data consistency between the two locations is ensured, reducing errors and problems caused by inconsistent data. Even in the event of a failure, the backup data center 20 can quickly take over the business, reducing downtime and improving user experience. The system design supports future expansion, and resources and service nodes can be added at any time according to business needs, which improves the flexibility of the system.

[0035] The design and implementation of this dual-active system greatly enhances the resilience and adaptability of enterprises in the face of emergencies. It realizes two-way synchronization of business systems, and synchronizes data between clusters across data centers through multiple synchronization components, without the need to establish a third data center as an arbitration center. Relying on the cross-cluster synchronization capabilities of various open source components, specific synchronization rules, and adaptive transformation of business and data synchronization management applications, the system effectively supports the dual-active architecture.

[0036] In daily operation, data components in both the primary and backup data centers 20 can provide services. When a failure occurs in the primary data center 10, the data synchronization management application can quickly switch to the business application in the backup data center, thereby improving the stability and robustness of the active-active architecture system. This design ensures high availability of the system and business continuity in various situations.

[0037] In one embodiment, see Figure 1 The above-mentioned business applications are used for status monitoring, data activation and disaster recovery switching between the primary data center 10 and the backup data center 20.

[0038] In one embodiment, see Figure 1 and Figure 5 The above-mentioned business application is used to monitor the status of the Kafka cluster and Redis cluster of the primary data center 10 and the backup data center 20 in real time, and make abnormal judgments according to preset rules to achieve dynamic switching and customized cache logic. If the primary data center 10 is restored, the data synchronization component is started through the script to perform full or incremental data synchronization, and automatically restarts when the data synchronization component is abnormal to ensure continuous service availability.

[0039] In one embodiment, the business computing application is used to connect to the message queue of the Kafka cluster and the cache library of the Redis cluster through the internal DNS according to the instructions issued by the business application, and access the business library to perform data warehousing operations.

[0040] Specifically, the business application integrates a built-in monitoring module to facilitate the configuration and management of various key components, including Kafka clusters, Redis clusters, MirrorMaker2 components, Redis-Shake components, and application API addresses. The core of this mechanism is to achieve real-time monitoring and management of resources deployed in the primary data center 10 (computer room A) and the backup data center 20 (computer room B).

[0041] The monitoring module is configured in the following ways: Kafka cluster configuration: Configure the Kafka cluster of the primary and standby data centers 20 to ensure reliable message delivery.

[0042] Redis Cluster Configuration: Set up a Redis cluster to manage access to cached data.

[0043] MirrorMaker2 and Redis-Shake components: Configure these two components to achieve synchronization and real-time processing of data.

[0044] API address configuration: Set up an API interface for business applications to facilitate data interaction.

[0045] By setting the timer, the monitoring module can regularly execute the data collection script to obtain the operating status of the Kafka cluster and Redis cluster in computer room A and computer room B. At the same time, it also monitors the status of the data synchronization components MirrorMaker2 and Redis-Shake.

[0046] During the monitoring process, the monitoring module will determine the operating status of each component according to the preset rules. If an abnormality is found in the Kafka cluster or Redis cluster, the system will trigger the switching module and take the following measures: Dynamically add consumption topics: Dynamically add consumption topics to business applications to ensure that message processing is not affected.

[0047] Customize Redis cache value logic: Adjust Redis cache logic according to business needs to achieve smooth business switching and ensure that necessary data can still be accessed under abnormal circumstances.

[0048] When the Kafka cluster or Redis cluster of the primary data center 10 is repaired, the monitoring module will restart and configure the data synchronization component through the built-in script to achieve full or incremental synchronization of data. The specific process is as follows: Full data synchronization: When the cluster status is restored, the system will perform full data synchronization to ensure that all data is accurately synchronized to the target cluster.

[0049] Incremental data synchronization: If only part of the data needs to be synchronized, the system will perform incremental synchronization to reduce the amount of data transferred and the synchronization time.

[0050] If the running status of MirrorMaker2 or Redis-Shake is abnormal, the monitoring module will also use the built-in script to start and configure the data synchronization component and perform a restart operation to ensure the continuous synchronization of incremental data. This process includes: Monitoring status: Continuously monitor the status of MirrorMaker2 and Redis-Shake to detect abnormalities in a timely manner.

[0051] Automatic restart: When the status is abnormal, the restart process is automatically triggered to ensure that the data synchronization component can quickly resume normal operation.

[0052] Through this refined monitoring and management mechanism, data synchronization management business applications can flexibly respond to various emergencies while ensuring high availability and data consistency of the primary and standby data centers 20, thereby improving system reliability and user experience. This automated management and recovery process not only improves work efficiency, but also reduces the need for manual intervention, ensuring the smooth operation of the system.

[0053] In one embodiment, see Figure 4 The above-mentioned business applications are used to achieve cross-computer room active-active deployment through soft load cluster combined with Keepalived and Nginx, and introduce global load balancing for intelligent domain name resolution; users access the business system through the Xintong DNS domain name, and provide two-way services for the production environment and the disaster recovery environment, while automatically switching traffic in the event of a computer room disaster.

[0054] In this embodiment, Keepalived and Nginx are used to implement a soft load balancing cluster (SLB), and combined with the global load balancing (GSLB) technology, to ensure active-active deployment of business applications across computer rooms; specifically, Keepalived and Nginx are configured as a soft load balancing cluster to ensure intelligent distribution of traffic between the two computer rooms. Two virtual IPs (VIP1 and VIP2) are used in the system as the entrance to business access, and users access through the Xintong DNS domain name.

[0055] Global Load Balancing (GSLB) implements intelligent domain name resolution, and distributes user traffic to different computer rooms through DNS polling and weight distribution, ensuring load balancing and high availability of services. In the event of a computer room-level disaster, business traffic will automatically switch to another computer room to ensure that user experience is not affected.

[0056] Business systems, or business applications, are deployed in both computer rooms. Normally, applications in the production environment and the disaster recovery environment provide external services.

[0057] The upper-layer load balancing implements active-active operation at the application layer, allowing user requests to be processed in any computer room.

[0058] The business applications of the two computer rooms connect to the Kafka message queues and Redis cache libraries of their respective computer rooms through internal DNS domain names to realize data consumption and operations.

[0059] The business applications in the two computer rooms are simultaneously connected to the Oracle database in the production environment to perform computing tasks and store data.

[0060] Business applications have the ability to dynamically manage Kafka topics. Using the consumer.subscribe() method, applications can dynamically adjust the currently subscribed topic list as needed.

[0061] When Data Center A fails, the business applications in Data Center B can modify the subscribed Topic list to consume the unconsumed data in the Topic synchronized from Data Center A to Data Center B. After Data Center A is repaired, Data Center B will remove the newly added Topic subscription. Data Center A refers to the primary data center 10, and Data Center B refers to the backup data center 20.

[0062] Business applications can dynamically add or delete whitelists of Redis cache values ​​and limit the searchable keys through customized value retrieval logic.

[0063] If a failure occurs in computer room A, the business application in computer room B can adjust the whitelist configuration to allow retrieval of keys with characteristic values ​​of computer room A. After computer room A is repaired, the keys with these characteristic values ​​will be removed.

[0064] API interface: The whitelist function of dynamically adding or deleting Kafka topics and Redis cache values ​​is provided to the outside world through API, allowing other applications to call it. This design enhances the flexibility of the system and makes operation, maintenance and management more efficient.

[0065] This architecture ensures high availability and flexibility of the business system through active-active deployment across data centers and intelligent load balancing. The dynamic management capabilities of Kafka and Redis, coupled with custom value logic, enable the system to quickly adapt to changes and failures, ensuring continuous service experience for users. The design of the overall architecture fully considers business continuity, resource utilization, and fault recovery capabilities, providing solid technical support for enterprises.

[0066] In one embodiment, the business database includes an Oracle database.

[0067] Specifically, in the technical selection of business databases, taking Oracle as an example, clusters are deployed in the two computer rooms through the RAC (Real Application Clusters) mode to ensure high availability and reliability. This architecture uses a unified SCANVIP or domain name address to facilitate access by applications in the two computer rooms.

[0068] The underlying ADG (Active Data Guard) technology is used to achieve real-time data synchronization between the primary database and the disaster recovery environment. This design not only provides business continuity, but also allows for rapid switching to the backup environment in the event of a host failure, ensuring data consistency and real-time performance. The overall architecture effectively improves the availability and flexibility of the system, meeting the needs of modern enterprises for database systems.

[0069] In one embodiment, if Figure 2 As shown, the two Redis clusters are synchronized bidirectionally through Redis-Shake. When the production environment fails over to the standby data center 20, the Redis cluster of the standby data center 20 adds a key with the characteristics of the primary data center 10 to continue consuming data.

[0070] In one embodiment, if Figure 2 As shown, the two Redis clusters use filtering LUA scripts and provide data caching capabilities at the same time when no abnormal situation occurs; when a problem occurs in the primary data center 10, the Redis whitelist of the backup data center 20 will add a key with the characteristics of the primary data center 10; after the primary data center 10 is restored, the backup data center 20 synchronizes the full or incremental data to the primary data center 10 through Redis-Shake.

[0071] In this embodiment, in the dual-computer room deployment, computer room A (primary data center 10) and computer room B (backup data center 20) independently deploy Redis sentinel clusters to ensure high availability and load balancing. This deployment enables the Redis clusters in the two computer rooms to provide data caching capabilities in daily operation, thereby improving the reliability and performance of the system.

[0072] Redis Sentinel Cluster: A Redis Sentinel Cluster is deployed in each computer room to monitor the status of Redis instances and automatically perform failover when a failure occurs.

[0073] Redis-Shake: used to achieve two-way data synchronization between computer room A and computer room B. Redis-Shake can ensure data consistency between the Redis instances in the two computer rooms and support real-time data update and synchronization.

[0074] In the two computer rooms, when business applications generate the KEY / VALUE of the task queue, they will add the computer room alias as the characteristic value at the beginning of the KEY. For example, the KEY generated by a business application was originally collq, and after the transformation: The KEY stored in computer room A becomes A.collq; The KEY stored in computer room B becomes B.collq; This naming strategy not only makes the KEY have the characteristics of the computer room, but also avoids KEY conflicts during data synchronization and enhances the flexibility of data management.

[0075] Redis-Shake is used to achieve bidirectional data synchronization between two computer rooms. The specific operations are as follows: Bidirectional synchronization configuration: Configure Redis-Shake to support bidirectional data synchronization from data center A to data center B and from data center B to data center A.

[0076] Filter LUA script: Add a filter LUA script in Redis-Shake to perform synchronization based on the characteristic value of the KEY (i.e. the alias of the data center). This mechanism can effectively prevent infinite recursion during the synchronization process, such as the mutual influence between A.collq of data center A and B.collq of data center B.

[0077] Under normal circumstances, the Redis clusters in Data Center A and Data Center B will provide services at the same time. Once Data Center A fails, the business application will add a key with the characteristics of Data Center A to the Redis whitelist of Data Center B. In this way, the application in Data Center B can continue to consume data related to Data Center A, ensuring business continuity.

[0078] During the failure of computer room A, computer room B maintains normal business operation by adding a whitelist of characteristic value KEY.

[0079] Once Data Center A is restored, Redis-Shake will be responsible for synchronizing the full or incremental data of Data Center B back to Data Center A. This process ensures the consistency of the data in Data Center A and Data Center B, while ensuring the normal use of business applications.

[0080] By adopting the Redis dual-data center cluster deployment and Redis-Shake's two-way data synchronization mechanism, enterprises can achieve high-availability and high-reliability cache solutions. The KEY naming strategy effectively avoids data conflicts and ensures the continuous operation of the business in the event of a failure. In addition, the fault switching and recovery mechanism ensures the real-time and consistency of data, providing enterprises with a solid data infrastructure.

[0081] In one embodiment, if Figure 3As shown in the figure, the two Kafka clusters use MirrorMaker2 to perform real-time bidirectional data synchronization at the topic granularity, and regularly synchronize consumption submission pointers. When the production environment fails over to the disaster recovery environment, the unconsumed data continues to be consumed.

[0082] In one embodiment, if Figure 3 As shown, when the two Kafka clusters are working, the computer room prefix is ​​added before the Topic name; when the primary data center 10 fails, the business computing application of the backup data center 20 increases the subscription to the Topic synchronized with the primary data center 10, and after the primary data center 10 is restored, the unconsumed data of the backup data center 20 is synchronized to the primary data center 10.

[0083] Specifically, in the dual-computer room deployment, two sets of Kafka clusters are independently deployed in Computer Room A (primary data center 10) and Computer Room B (backup data center 20). Bidirectional data synchronization between the two computer rooms is achieved through MirrorMaker2 to ensure data consistency between different computer rooms.

[0084] Kafka cluster: Two Kafka clusters are deployed in computer room A and computer room B to store and manage message data.

[0085] MirrorMaker2: As Kafka's official tool, MirrorMaker2 is used to implement data replication and synchronization between Kafka clusters.

[0086] Consumer group offset management: Through Kafka's consumer group mechanism, the last consumption position of each consumer in each topic is recorded to ensure that unprocessed data can continue to be consumed when a failure occurs.

[0087] MirrorMaker2 is configured for bidirectional synchronization to support the following scenarios: Synchronization from computer room A to computer room B: Data flow: The producer writes data to the Kafka cluster in computer room A, and MirrorMaker2 replicates the data to the Kafka cluster in computer room B.

[0088] Topic naming: Set replication.policy.class to DefaultReplicationPolicy in the MirrorMaker2 configuration file so that the topics synchronized to Data Center B are prefixed with the source cluster alias. For example, the topic bi-log in Data Center A becomes A.bi-log in Data Center B.

[0089] Synchronization from computer room B to computer room A: Reverse flow: The same configuration is also applicable to the synchronization from computer room B to computer room A, ensuring the consistency of data on both sides.

[0090] The topic prefix mechanism is used to avoid data dead loops during the two-way synchronization process. For example: After the subject bi-log of computer room A is synchronized to computer room B, it becomes A.bi-log.

[0091] After the subject bi-log of computer room B is synchronized to computer room A, it becomes B.bi-log.

[0092] This naming strategy effectively prevents the same topic from being replicated between the two clusters, thus avoiding an infinite loop.

[0093] In order to ensure that consumers can continue to consume from the correct position during a failover, MirrorMaker2 provides an offset synchronization function. The specific configuration is as follows: In the MirrorMaker2 configuration, set sync.group.offsets.enabled=true to enable consumer group offset synchronization.

[0094] By configuring sync.group.offsets.interval.seconds to define the time interval for offset synchronization, ensure that the consumer group offset of the Kafka cluster in computer room A is regularly converted to the __consumer_offsets topic of computer room B.

[0095] When a failure occurs in the production environment and a switch to the disaster recovery environment is required, the following process will be triggered: Switch to Data Center B: The business applications in Data Center A add subscriptions to the topics synchronized from Data Center A in the Kafka cluster of Data Center B, such as A.bi-log.

[0096] When starting a consumer, the consumer group will start consuming data from the unconsumed offset to ensure business continuity.

[0097] Fault recovery: Once data center A is restored, MirrorMaker2 will continue to synchronize the unconsumed data in data center B back to data center A.

[0098] The same topic will have the same name in Data Center A to ensure data consistency.

[0099] Through the Kafka dual-data center architecture and MirrorMaker2's bidirectional data synchronization mechanism, enterprises can achieve high availability and data consistency. In the event of a failure, consumers can start consuming data from the correct offset to ensure that the business is not affected. The effective combination of topic naming strategy and offset synchronization mechanism makes the entire architecture excellent in fault tolerance and data management.

[0100] The above-mentioned active-active system of a business system realizes real-time synchronization and high availability of data by deploying business applications, Redis clusters, Kafka clusters and business libraries in the primary data center 10 and the backup data center 20 respectively, ensuring that when a failure occurs in any data center, the other center can quickly take over the service; using the Kafka cluster to realize two-way synchronization of data, ensuring data consistency between the primary and backup data centers 20, reducing the risk of business interruption caused by data inconsistency, and realizing fast data caching and efficient access through the Redis cluster; the business computing applications of the primary and backup data centers 20 are connected to their respective Redis clusters and Kafka clusters to ensure that when the data centers are switched, the computing resources can quickly respond to requests and improve the elasticity of the system; through direct connection with the terminal, it is ensured that user requests can be processed in any data center, enhancing the availability and fault tolerance of the system; by monitoring and dynamically adjusting the resource allocation of the primary and backup data centers 20, efficient utilization of resources is realized, ensuring that the system can automatically expand during peak load moments, and effectively releasing resources when the load is reduced, maintaining the flexibility and efficiency of the system.

[0101] The above-mentioned active-active system of a business system deploys business applications, Redis clusters, Kafka clusters and business libraries through the main data center 10 and the backup data center 20 respectively to achieve real-time data synchronization and high availability. The two data centers are interconnected through the business library, Redis and Kafka to ensure data consistency and fast access, while supporting terminal connection. The flexible configuration and dynamic resource management of business computing applications between the main and backup data centers 20 improve the elasticity and resource utilization of the system. It effectively improves the availability, elasticity and resource utilization of the business system and realizes true business continuity and disaster recovery capabilities.

[0102] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0103] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0104] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the system of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0105] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0106] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A dual-active system of a business system, characterized in that: It includes a main data center and a backup data center, and the main data center and the backup data center are respectively deployed with business applications, Redis clusters, Kafka clusters, business libraries and business computing applications for data synchronization management; the main data center and the backup data center are respectively connected to the terminal; the business library of the main data center is connected to the business library of the backup data; the Redis cluster of the main data center is connected to the Redis cluster of the backup data; the Kafka cluster of the main data center is connected to the Kafka cluster of the backup data; the Redis cluster and Kafka cluster of the main data center are respectively connected to the business computing application of the main data center; the business computing application of the main data center is respectively connected to the business library and the business application of the main data center; the Redis cluster and Kafka cluster of the backup data center are respectively connected to the business computing application of the backup data center; the business computing application of the backup data center is respectively connected to the business library and the business application of the backup data center.

2. The active-active system of a business system according to claim 1, characterized in that: The business application is used to perform status monitoring, data activation and disaster recovery switching between the primary data center and the backup data center.

3. The active-active system of a business system according to claim 2, characterized in that: The business application is used to monitor the status of the Kafka cluster and Redis cluster of the primary data center and the backup data center in real time, and make exception judgments based on preset rules to achieve dynamic switching and customized cache logic. If the primary data center is restored, the data synchronization component is started through a script to perform full or incremental data synchronization, and automatically restarts when the data synchronization component is abnormal to ensure continuous service availability.

4. The active-active system of a business system according to claim 3, characterized in that: The business application is used to achieve active-active deployment across computer rooms through a soft load cluster combined with Keepalived and Nginx, and introduce global load balancing for intelligent domain name resolution; users access the business system through the Xintong DNS domain name, provide two-way services for the production environment and the disaster recovery environment, and automatically switch traffic in the event of a computer room disaster.

5. The active-active system of a business system according to claim 1, characterized in that: The business computing application is used to connect to the message queue of the Kafka cluster and the cache library of the Redis cluster through the internal DNS according to the instructions issued by the business application, and access the business library to perform data warehousing operations.

6. The active-active system of a business system according to claim 1, characterized in that: The business database includes an Oracle database.

7. The active-active system of a business system according to claim 1, characterized in that: The two Redis clusters are synchronized bidirectionally through Redis-Shake. When the production environment fails over to the standby data center, the Redis cluster in the standby data center adds a key with the characteristics of the primary data center to continue consuming data.

8. The active-active system of a business system according to claim 7, characterized in that: The two Redis clusters use filtering LUA scripts and provide data caching capabilities at the same time when no abnormal situation occurs; when a problem occurs in the primary data center, the Redis whitelist of the backup data center will add a key with the characteristics of the primary data center; after the primary data center is restored, the backup data center will synchronize the full or incremental data to the primary data center through Redis-Shake.

9. The active-active system of a business system according to claim 1, characterized in that: The two Kafka clusters perform real-time bidirectional data synchronization at the topic granularity through MirrorMaker2, and synchronize consumption submission pointers regularly. When the production environment fails and switches to the disaster recovery environment, the unconsumed data continues to be consumed.

10. The active-active system of a business system according to claim 9, characterized in that: When the two Kafka clusters are working, the computer room prefix is ​​added before the Topic name; when the primary data center fails, the business computing application of the backup data center adds a subscription to the Topic synchronized with the primary data center, and after the primary data center is restored, the unconsumed data of the backup data center is synchronized to the primary data center.

Images