A method for realizing ESB two-site three-center double-living

Abstract

The application relates to the technical field of storage, and discloses a method for realizing ESB two-site three-center dual-active, which realizes a dual-active architecture, improves the availability and fault tolerance of a system by deploying two data replicas in a production center, a city disaster recovery center and a remote disaster recovery center, and configuring complete business systems and data processing capabilities. Meanwhile, the real-time synchronization of data changes is realized by using the broadcast mechanism of the ESB, the number of data replication is reduced, data overlap and resource waste are avoided, and the operation cost is reduced. In addition, the method is also based on the fault detection and recovery mechanism of the ESB, when any center fails, the standby replica can be quickly switched to, so that the continuity and stability of the business are ensured. The architecture and mechanism not only solve the single-point failure and performance bottleneck problems in the traditional ESB architecture, but also meet the high availability and scalability requirements of the business, and provide a more efficient, flexible and reliable solution for large enterprises and Internet companies.

Description

Technical Field

[0001] This invention relates to the field of storage technology, specifically to a method for implementing dual-active ESB in two locations and three centers. Background Technology

[0002] With the rapid development of information technology and the continuous expansion of enterprise businesses, distributed system architecture has gradually become the mainstream choice for large enterprises and internet companies. Among them, ESB (Enterprise Service Bus), as an important integration framework, provides an efficient and flexible solution for communication and data exchange between various systems within an enterprise. However, when faced with the ever-increasing volume of data and user access demands, traditional ESB architecture has gradually exposed problems such as single points of failure and performance bottlenecks, failing to meet the high availability and scalability requirements of business operations.

[0003] To address these issues, the industry has proposed the concept of an ESB (Enhanced Service Provider) dual-site, three-center active-active architecture. This architecture improves system availability and fault tolerance by deploying data centers in different geographical locations and achieving data synchronization and load balancing between them. However, implementing an ESB dual-site, three-center active-active architecture still faces a series of technical challenges.

[0004] Existing ESB two-site three-center active-active architectures typically include a production center, a local disaster recovery center, and a remote disaster recovery center. The first data replication route involves the production center synchronously replicating data to the local disaster recovery center, which then replicates the entire data to the remote disaster recovery center. The second data replication route involves synchronous replication between the production center and the local disaster recovery center, followed by replication of the entire data from both centers to the remote disaster recovery center. However, these existing data backup methods require significant bandwidth. For example, in the first route, the production center sends data to the local disaster recovery center, which then replicates the received data along with its own data to the remote disaster recovery center—a process involving two replications. Similarly, in the second route, both the production center and the local disaster recovery center need to replicate the data from the other center, along with their own data, to the remote disaster recovery center. This excessive replication also leads to significant data overlap, wasting resources and increasing operating costs. Therefore, a method for implementing ESB two-site three-center active-active architecture is proposed. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method for implementing dual-active ESB in two locations with three centers, thereby resolving the problems in the background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for implementing dual-active ESB with two locations and three centers, the method comprising:

[0007] A production center and a local disaster recovery center are respectively established in two different geographical positions in the same city, and a remote disaster recovery center is established in a different place. An enterprise service bus (ESB) for data exchange and integration is deployed in the production center, the local disaster recovery center, and the remote disaster recovery center.

[0008] Two data replicas are deployed in the production center, the local disaster recovery center, and the remote disaster recovery center, respectively denoted as replicas a and b in the production center, replicas c and d in the local disaster recovery center, and replicas e and f in the remote disaster recovery center.

[0009] The production center, the local disaster recovery center, and the remote disaster recovery center are all equipped with complete business systems and data processing capabilities, and can simultaneously provide services to realize dual active.

[0010] When data changes occur in the production center replica a and the local disaster recovery center replica c, the production center replica a and the local disaster recovery center replica c send these change data to the local disaster recovery center replica d, the remote disaster recovery center replica e, and the production center replica b, the remote disaster recovery center replica f, respectively based on the broadcast mechanism of the ESB.

[0011] Fault detection and recovery mechanism based on ESB:

[0012] When the local disaster recovery center fails, the production center replicas a and b automatically switch to active state to receive and continue processing the data read and write tasks previously handled by the local disaster recovery center replica c. Meanwhile, the production center replica a sends the change data to the remote disaster recovery center replica e, and the production center replica b sends the change data to the remote disaster recovery center replica f.

[0013] When the production center fails, the local disaster recovery center replicas c and d automatically switch to active state to receive and continue processing the data read and write tasks previously handled by the production center replica a. Meanwhile, the local disaster recovery center replica c sends the change data to the remote disaster recovery center replica f, and the local disaster recovery center replica d sends the change data to the remote disaster recovery center replica e.

[0014] Preferably, the ESB is used as the core of service routing, and intelligent DNS technology and health route injection (RHI) mechanism are used to dynamically distribute user requests to the production center replica a or the local disaster recovery center replica c for data shunting.

[0015] Preferably, the ESB is used to monitor the load, health status, and geographical position factors of each data center in real time, and based on these information, dynamic routing adjustment is performed to ensure balanced distribution of business traffic and availability of services.

[0016] Preferably, based on the fault detection and recovery mechanism of the ESB, when both the local disaster recovery center and the production center fail, the replica e of the off-site disaster recovery center and the replica f of the off-site disaster recovery center will respectively undertake the data read-write tasks previously responsible by the replica a of the production center and the replica c of the local disaster recovery center, ensuring business continuity; at this time, the off-site disaster recovery center will serve as a new production center to provide external services, and after the production center and the local disaster recovery center are restored, data synchronization and recovery will be performed.

[0017] Preferably, the broadcast mechanism adopts a combination of incremental replication and full replication, and incremental replication is used for frequently changing data to reduce replication overhead, and periodic full replication is used for infrequently changing data to ensure data integrity.

[0018] Preferably, when the production center and the local disaster recovery center send change data respectively, and the production center and the local disaster recovery center send change data to the off-site disaster recovery center respectively, each data replica is accompanied by a version number for tracking the change history of the data; when a data conflict occurs, if two or more centers simultaneously attempt to modify the same data item, the system will automatically select the correct data version according to the pre-set conflict resolution strategy, ensuring data consistency and accuracy.

[0019] Preferably, data caches are deployed in the front ends of the production center, the local disaster recovery center and the off-site disaster recovery center, hot data is cached in the memory, the number of direct access to the database is reduced, the database load is reduced, and the response speed is improved, and at the same time, the cache layer will periodically synchronize with the backend database to ensure the timeliness of the data.

[0020] Preferably, a blockchain network is established between the production center, the local disaster recovery center and the off-site disaster recovery center, the hash value or summary information of the key data is chained, data tamper-proofing and traceability are realized; at the same time, the smart contract function of the blockchain is used to automatically perform data synchronization, verification and recovery operations, improving the automation level and reliability.

[0021] Preferably, by collecting and analyzing historical business traffic data, user behavior patterns and external event information, a prediction algorithm is used to predict the business traffic of the production center, the local disaster recovery center and the off-site disaster recovery center in the future period of time; based on the prediction result, the ESB can perform resource allocation and routing optimization in advance to ensure stable service provision during high traffic periods, while avoiding resource waste during low traffic periods.

[0022] Compared with the prior art, the present application has the following beneficial effects:

[0023] The application realizes a dual-active architecture by deploying two data replicas in the production center, the local disaster recovery center and the off-site disaster recovery center, and configuring complete business systems and data processing capabilities, thereby improving the availability and fault tolerance of the system. Meanwhile, by using the broadcast mechanism of the ESB, real-time synchronization of data changes is realized, the number of data replication is reduced, data overlap and resource waste are avoided, and the operation cost is reduced. In addition, based on the fault detection and recovery mechanism of the ESB, when any center fails, the standby replica can be quickly switched to, thereby ensuring the continuity and stability of the business. This architecture and mechanism not only solve the single-point failure and performance bottleneck problem in the traditional ESB architecture, but also meet the high availability and scalability requirements of the business, thereby providing a more efficient, flexible and reliable solution for large enterprises and Internet companies.

[0024] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 The ESB two-site three-center deployment diagram of the present application;

[0026] Figure 2 The specific flowchart of the method for realizing the ESB two-site three-center dual-active of the present application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the present application.

[0028] Please refer to Figure 1 and Figure 2 A method for realizing the ESB two-site three-center dual-active in the present application, the method comprises the following contents:

[0029] I. System architecture and deployment

[0030] Geographical location and center establishment:

[0031] The production center (denoted as PC) and the local disaster recovery center (denoted as DCC) are respectively established in different geographical locations (such as the east of the city and the west of the city) in the same city.

[0032] Establish a remote disaster recovery center (RCC) in a different location (e.g., another city or province).

[0033] Data replica deployment:

[0034] Deploy data replicas a and b in the PC.

[0035] Deploy data replicas c and d in the DCC.

[0036] Deploy data replicas e and f in the RCC.

[0037] Business system and data processing capabilities:

[0038] Ensure that the PC, DCC, and RCC are all equipped with complete business systems and data processing capabilities to achieve dual-active.

[0039] II. Data change and broadcast mechanism

[0040] Data change generation:

[0041] When replica a in the PC or replica c in the DCC detects a data change, an event is triggered, marking the data as "to be broadcast."

[0042] Broadcast mechanism implementation:

[0043] Step 1: Use the message middleware of ESB (such as Apache Kafka or RabbitMQ) to design an efficient data change broadcast system. This system includes a producer (Producer) and multiple consumers (Consumer).

[0044] Step 2: When the data change event is triggered, the producer will encapsulate the change data into a message and attach necessary metadata (such as data type, change timestamp, etc.).

[0045] Step 3: The message is asynchronously sent to all relevant consumers, i.e., replica d of the DCC and replica e of the RCC (for changes generated by replica a of the PC), replica b of the PC and replica f of the RCC (for changes generated by replica c of the DCC), through the network topology of ESB.

[0046] Step 4: After receiving the message, the consumer will decode and verify it to ensure data integrity and correctness. Then, according to the business logic, the change data is applied to the local data replica.

[0047] Incremental and full replication:

[0048] Incremental replication: For frequently changing data, periodically (e.g., every second or minute) check the data change log and replicate new changes in an incremental manner to other data centers.

[0049] Full replication: For data that does not change frequently, set a fixed time period (such as daily or weekly) and replicate all data to other data centers in a full manner within this period. During full replication, incremental replication will be stopped first to ensure data consistency.

[0050] III. Fault Detection and Recovery Mechanism

[0051] Fault detection:

[0052] Utilize ESB to monitor the load, health status, and geographical location factors of each data center in real time.

[0053] Recovery mechanism:

[0054] When DCC fails, replica b in PC automatically switches to active status, receives and continues to process the data read and write tasks previously handled by DCC; at the same time, replica a in PC sends these changed data to replica e in RCC, and replica b in PC sends these changed data to replica f in RCC.

[0055] When the PC fails, replica d in the DCC automatically switches to active status, receiving and continuing to process the data read and write tasks previously handled by the PC. Simultaneously, replica c in the DCC sends these changed data to replica e in the RCC, and replica d in the DCC sends these changed data to replica f in the RCC.

[0056] When both the PC and DCC fail, RCC replicas e and f will take over the data read / write tasks previously handled by the PC and DCC, respectively, ensuring business continuity. In this situation, the RCC will serve as a new production center providing external services.

[0057] IV. Data Consistency and Conflict Resolution

[0058] Version number management:

[0059] Each data copy is assigned a version number when it is stored, which is used to track the data's change history. The version number can be an incrementing integer or a timestamp.

[0060] Conflict resolution strategies (incorporating specific implementation details):

[0061] Strategy 1: Timestamp Priority. When multiple data replicas attempt to modify the same data item simultaneously, the system compares their timestamps. The data version with the latest timestamp is selected as the final version and updated in other data centers.

[0062] Strategy Two: Business Logic Judgment. In some complex scenarios, it may be necessary to combine business logic to resolve conflicts. For example, for some specific types of data (such as order status), the system may choose the correct data version according to business rules (such as orders submitted first).

[0063] In some advanced conflict resolution strategies, mathematical formulas are used to calculate the priority or consistency of data. For example, use weighted average method to calculate the weight of multiple data versions, and select the version with the highest weight as the final version; due to the complexity of data consistency, combined with business logic and algorithm for comprehensive judgment.

[0064] Five, data caching and performance optimization

[0065] Data caching deployment:

[0066] Deploy LRU (Least Recently Used) cache in the front end of PC, DCC and RCC, cache hot data to memory to reduce access pressure on the backend database.

[0067] Cache synchronization and update (integrate specific implementation details):

[0068] Step one: set the cache synchronization period, such as every minute or every hour. Within the synchronization period, the cache layer will periodically synchronize with the backend database to ensure data timeliness.

[0069] Step two: when the backend database changes, the system will trigger a cache update event. The event will notify the cache layer to perform the corresponding data update operation.

[0070] Step three: after receiving the update event, the cache layer will update the data item in the local cache according to the data change information in the event. If the cache space is insufficient, use LRU algorithm to remove the least recently used data item.

[0071] Six, blockchain network and data tamper-proofing

[0072] Blockchain network establishment:

[0073] Select a suitable blockchain platform (such as Ethereum, Hyperledger Fabric, etc.) and design a blockchain network topology structure that meets business needs.

[0074] Data on-chain and tamper-proofing:

[0075] Step one: hash or digest information of key data on-chain. This can be achieved by calling the API provided by the blockchain platform.

[0076] Step two: When data changes occur, the system recalculates the hash value or summary information of the data and compares it with the record on the blockchain. If they are inconsistent, it means that the data may have been tampered with.

[0077] Step three: Using the smart contract function of the blockchain, an automated script can be written to monitor data changes and trigger the corresponding anti-tampering mechanism. For example, when tampering with data is detected, the smart contract can automatically perform data recovery operations or send an alert to the administrator.

[0078] Smart contract application:

[0079] Write a smart contract to automate data synchronization, verification, and recovery operations. The smart contract can contain multiple functions and state variables to manage the status and results of data synchronization.

[0080] For example, write a function called syncData to trigger data synchronization operations; write a function called verifyData to verify the integrity and correctness of the data; and write a function called recoverData to perform recovery operations when data is tampered with.

[0081] Seven, intelligent DNS technology and healthy routing injection (RHI) mechanism

[0082] Intelligent DNS technology:

[0083] Intelligent DNS technology is a technology that can intelligently select the best server address for DNS resolution based on factors such as user's geographic location, network conditions, etc. In the ESB two-site three-center dual-active architecture, intelligent DNS technology can further optimize user access experience, improve system availability and stability.

[0084] Intelligent DNS technology collects and analyzes user network request information, including IP address, geographic location, operator, etc., and then intelligently selects the best data center for DNS resolution based on this information. For example, when the user is located in the telecom network, intelligent DNS technology can select the IP address of the telecom data center for resolution, thereby avoiding the delay and instability caused by cross-network access.

[0085] Implementation steps:

[0086] Step one: Deploy intelligent DNS server. In the ESB architecture, one or more intelligent DNS servers need to be deployed to handle user DNS query requests.

[0087] Step two: Configure intelligent DNS strategy. According to business needs and network environment, configure intelligent DNS strategy, including selecting the best data center for DNS resolution based on geographic location, network conditions, etc.

[0088] Step three: Testing and optimization. After the implementation of intelligent DNS technology, comprehensive testing and optimization are needed to ensure the accuracy and stability of DNS resolution. At the same time, according to user feedback and actual needs, intelligent DNS strategy needs to be continuously optimized to improve the usability and performance of the system.

[0089] By intelligently selecting the best data center for DNS resolution, user access latency can be reduced, and access speed can be improved. Intelligent DNS technology can intelligently allocate resources based on user actual needs and network environment, optimizing resource utilization. By avoiding cross-network access and selecting the best data center for DNS resolution, the stability and reliability of the system can be enhanced.

[0090] Healthy route injection (RHI) mechanism

[0091] The healthy route injection (RHI) mechanism is a technology that dynamically adjusts routing based on the health status of network paths. In the ESB two-site three-center dual-active architecture, the RHI mechanism can monitor the health status of network paths in real time and dynamically adjust routing based on monitoring results to ensure reliable data transmission and system stability.

[0092] The RHI mechanism deploys monitoring nodes on network paths to monitor key indicators such as delay and packet loss rate in real time. When a network path is found to have failed or performance has declined, the RHI mechanism will automatically adjust routing to divert data traffic to other healthy network paths to ensure reliable data transmission.

[0093] Implementation steps:

[0094] Step one: Deploy monitoring nodes. Deploy monitoring nodes at key locations on the network path to monitor the health status of the network path in real time.

[0095] Step two: Configure RHI strategy. Configure RHI strategy according to business needs and network environment, including selection of monitoring indicators, setting of threshold values, and rules for routing adjustment, etc.

[0096] Step three: Real-time monitoring and adjustment. The RHI mechanism will monitor the health status of the network path in real time and automatically adjust the routing based on the monitoring results. At the same time, the effect of the RHI mechanism needs to be analyzed and evaluated regularly, and necessary adjustments and optimizations need to be made based on the evaluation results.

[0097] By monitoring the health of network paths in real-time and dynamically adjusting routing, reliable data transmission can be ensured; the RHI mechanism can dynamically adjust routing based on the health of network paths, thereby optimizing network performance and improving data transmission speed and efficiency; by monitoring and dynamically adjusting routing in real-time, network failures or performance degradation problems can be discovered and solved in a timely manner, enhancing the stability and reliability of the system.

[0098] Eight, business traffic prediction

[0099] Historical data analysis:

[0100] Collect and analyze historical business traffic data, user behavior patterns, and external event information (such as holidays, major events, etc.). This can be achieved through data mining and machine learning techniques.

[0101] Prediction algorithm application:

[0102] Algorithm selection: Select appropriate prediction algorithms such as ARIMA (Autoregressive Integrated Moving Average Model) or LSTM (Long Short-Term Memory Network).

[0103] Step one: Preprocess and feature extraction of historical data. This includes data cleaning, missing value filling, outlier detection, etc.

[0104] Step two: Train ARIMA model or LSTM model using preprocessed data; this can be achieved by calling machine learning libraries such as Python's scikit-learn or TensorFlow.

[0105] Step three: Verify the accuracy of the model. Use cross-validation or leave-out methods to evaluate the prediction performance of the model.

[0106] Step four: Apply the trained model to predict future business traffic. Based on the prediction results, ESB can perform resource allocation and routing optimization in advance.

[0107] Resource allocation and routing optimization:

[0108] Based on the prediction results, ESB can use intelligent DNS technology and health route injection (RHI) mechanism

[0109] Dynamically adjust resource allocation and routing strategies. For example, increase server resources or optimize network routing to improve system throughput during high traffic periods; reduce resource usage to reduce operating costs during low traffic periods.

Claims

1. A method for realizing ESB two-site three-center dual-homing, characterized in that, The method includes: A production center and a local disaster recovery center are set up in two different geographical locations within the same city, and an off-site disaster recovery center is set up in another location. Enterprise Service Bus (ESB) for data exchange and integration is deployed in the production center, local disaster recovery center and off-site disaster recovery center. The production center, the same-city disaster recovery center, and the off-site disaster recovery center each deploy two data copies, denoted as copies a and b, c and d, and e and f, respectively. The production center, the same-city disaster recovery center, and the off-site disaster recovery center are all equipped with complete business systems and data processing capabilities, which can provide services to the outside world at the same time and achieve dual-active operation. When data changes occur in production center replica a and local disaster recovery center replica c, they are all broadcast based on the ESB mechanism. Production center replica a and local disaster recovery center replica c send these changed data to local disaster recovery center replica d, remote disaster recovery center replica e, production center replica b, and remote disaster recovery center replica f, respectively. ESB-based fault detection and recovery mechanism: When the local disaster recovery center fails, production center replica b automatically switches to active status, receives and continues to process the data read and write tasks previously handled by local disaster recovery center replica c. At the same time, production center replica a sends these changed data to off-site disaster recovery center replica e, and production center replica b sends these changed data to off-site disaster recovery center replica f. When a failure occurs at the production center, the local disaster recovery center replica d automatically switches to active status, receives and continues to process the data read and write tasks previously handled by the production center replica a. At the same time, the local disaster recovery center replica c sends these changed data to the remote disaster recovery center replica f, and the local disaster recovery center replica d sends these changed data to the remote disaster recovery center replica e.

2. The method for realizing ESB two-site three-center double-living according to claim 1, characterized in that, Using the ESB as the core of service routing, and through intelligent DNS technology and the Healthy Routing Injection (RHI) mechanism, user requests are dynamically distributed to either production center replica a or local disaster recovery center replica c for data diversion.

3. The method for realizing ESB two-site three-center double-living according to claim 1, characterized in that, The ESB monitors the load, health status, and geographical location of each data center in real time, and makes dynamic routing adjustments based on this information to ensure balanced distribution of business traffic and service availability.

4. The method for realizing ESB two-site three-center double-living according to claim 1, characterized in that, Based on the fault detection and recovery mechanism of the ESB, when both the local disaster recovery center and the production center fail, the replicas e and f of the off-site disaster recovery center will respectively take over the data read and write tasks previously handled by the production center replica a and the local disaster recovery center replica c, ensuring business continuity. At this time, the off-site disaster recovery center will serve as the new production center, providing external services, and will perform data synchronization and recovery after the production center and the local disaster recovery center are restored.

5. The method for realizing ESB two-site three-center double-living according to claim 1, characterized in that, The broadcast mechanism employs a combination of incremental and full replication. Incremental replication is used for frequently changing data to reduce replication overhead, while periodic full replication is used for infrequently changing data to ensure data integrity.

6. The method for realizing ESB two-site three-center double-living according to claim 1, characterized in that, When the production center and the local disaster recovery center send change data separately, and the production center and the local disaster recovery center and the remote disaster recovery center send change data separately, each data copy will be attached with a version number to track the change history of the data. When a data conflict occurs, if two or more centers attempt to modify the same data item at the same time, the system will automatically select the correct data version according to the preset conflict resolution strategy to ensure the consistency and accuracy of the data.

7. The method for realizing ESB two-site three-center double-living according to claim 1, characterized in that, Data caches are deployed in the front ends of the production center, the local disaster recovery center and the remote disaster recovery center to cache hot data in memory, reduce the number of direct accesses to the database, reduce the database load, improve the response speed, and at the same time, the cache layer will periodically synchronize with the backend database to ensure the timeliness of the data.

8. The method for realizing ESB two-site three-center double-living according to claim 1, characterized in that, A blockchain network is established between the production center, the local disaster recovery center and the remote disaster recovery center to chain the hash value or summary information of the key data, realize the anti-tampering and traceability of the data, and at the same time, use the smart contract function of the blockchain to automatically perform data synchronization, verification and recovery operations to improve the automation level and reliability.

9. The method for realizing ESB two-site three-center double-living according to claim 1, characterized in that, By collecting and analyzing historical business traffic data, user behavior patterns and external event information, the prediction algorithm is used to predict the business traffic of the production center, the local disaster recovery center and the remote disaster recovery center in the future period of time. Based on the prediction result, the ESB can perform resource allocation and routing optimization in advance to ensure stable service provision during high traffic periods, while avoiding resource waste during low traffic periods.

Images