Data synchronization system and data synchronization method

Abstract

The present application relates to the technical field of data storage, and proposes a data synchronization system and a data synchronization method. The data synchronization system includes: a real-time transmission device and a companion proxy service plug-in constructed for each data center. Construct a companion proxy service plug-in to decouple data transmission from the application cluster and Redis; establish multiple connection channels, and transmit synchronization data, management policies and heartbeat information through different connection channels respectively, isolate the mutual influence between different types of data, and improve data transmission efficiency; perform sharding and compression processing on the synchronization data to improve the transmission efficiency of data synchronization and ensure the timeliness of data synchronization; construct memory computing, complete restoration and other operations in the memory computing area, and further ensure the real-time capability of data synchronization; and monitor memory computing, transmission status and heartbeat information to ensure data synchronization efficiency in various scenarios such as poor network quality and data transmission accumulation.

Description

Technical Field

[0001] The present application relates to the field of data storage technology, and in particular to a data synchronization system and a data synchronization method. Background Art

[0002] Redis (Remote Dictionary Server), a remote dictionary service, supports periodic writing of updated data to disk or writing modification operations to appended log files. Furthermore, it provides data synchronization between master and slave nodes. With the acceleration of enterprise digital transformation, business continuity efforts are placing higher demands on existing Redis capabilities. Business continuity efforts often require building active-active or multi-active service capabilities for application systems across data centers. As a widely used technical component in application system development, real-time cross-data center synchronization of Redis is a critical consideration. To support the requirements of active-active in the same city or multi-active across different geospatial locations, building Redis multi-active service capabilities requires a real-time, efficient, and accurate multi-data center synchronization mechanism, whether used as a cache database or a key-value database.

[0003] The existing solution has the following technical problems:

[0004] First, existing solutions suffer from high data synchronization latency and insufficient real-time data synchronization capabilities. Existing solutions typically rely on two implementation modes: synchronization based on RDB / AOF data files, and synchronization based on key-value pair scanning. Both of these data synchronization operations have a clear sequential dependency: data must first be stored in Redis in the primary data center, then transferred to other data centers through RDB / AOF files or key-value pair scanning to form a data stream. Only then can data be written to Redis in the same or remote data centers. When synchronizing large amounts of data, network congestion and the need to store data before writing to it can lead to latency of several minutes to hours, making it impossible to meet the active-active or multi-active requirements of application systems.

[0005] Secondly, existing solutions lack flexibility during the service process. Existing solutions usually develop corresponding plug-ins on Redis components to build multi-data center synchronization capabilities, tightly binding the solution to the Redis components. If the Redis architecture deployment mode of the data center is different, such as synchronizing from Cluster mode to Standalone mode, the existing components need to be modified, and flexible online adjustment cannot be achieved. At the same time, the existing plug-in service capabilities are single, such as single reading of RDB files and single writing of key-value data streams. It is impossible for a component to flexibly switch between reading and writing, file data streams, etc. This results in the existing solution being able to only complete one-way synchronization and lacking two-way synchronization capabilities. When switching from a data center to the primary data center, reverse data synchronization cannot be supported, and flexibility is relatively lacking. In addition, when network failures, network congestion, etc. occur, the existing solution cannot flexibly respond to the above-mentioned abnormal transmission status.

[0006] Third, the existing data synchronization model lacks technical capabilities such as data classification, sharding, and compression to improve data transmission efficiency. For example, based on the RDB data file model, the relevant RDB files are transferred to the destination data center before the RDB files are parsed and written; in the key-value pair scanning mode, the scanned key-value pairs are directly transferred by the relevant components to the destination data center before data writing. Metadata information such as data structure, data volume, and data distribution are mixed with business data and transmitted together, resulting in data transmission delay rate and data transmission accuracy. When large data sets, network congestion, node anomalies, etc. occur, it is impossible to achieve data channel division, data sharding compression, data sequence restoration, data metadata verification, and real-time calculation of management data, resulting in obvious shortcomings in data synchronization stability. Summary of the Invention

[0007] The present application provides a data synchronization system and a data synchronization method, which construct a companion proxy service plug-in and a real-time transmission device, and connect the companion proxy service plug-in and the real-time transmission device through multiple connection channels to solve the above problems.

[0008] This application adopts the following technical solutions:

[0009] In a first aspect, the present application provides a data synchronization system, the data synchronization system comprising: a real-time transmission device and a companion proxy service plug-in constructed for each data center;

[0010] Each companion proxy service plug-in is started when an update action occurs in the application cluster of the main data center, and establishes a data transmission connection channel, a policy transmission connection channel and a heartbeat information connection channel with the real-time transmission device respectively;

[0011] The companion proxy service plug-in corresponding to the main data center reads the synchronization data from the application cluster of the main data center according to the reading rules, compresses the synchronization data into multiple sub-data blocks according to the segmentation rules, and transmits the multiple sub-data blocks to the real-time transmission device through the data transmission connection channel according to the transmission rules;

[0012] The real-time transmission device performs memory calculation on multiple sub-data blocks and restores them into synchronous data, and transmits the synchronous data to the companion proxy service plug-in corresponding to each slave data center through each data transmission connection channel, so that the companion proxy service plug-in corresponding to each slave data center writes the synchronous data into the Redis of the corresponding slave data center;

[0013] The real-time transmission device monitors the transmission status of the data transmission connection channel between the memory computing and the companion proxy service plug-in corresponding to the main data center, and transmits the monitoring result to the companion proxy service plug-in corresponding to the main data center through the policy transmission connection channel;

[0014] The companion proxy service plug-in corresponding to the main data center updates the reading rules and transmission rules based on the monitoring results;

[0015] Each companion proxy service plug-in transmits heartbeat information to the real-time transmission device through the corresponding heartbeat information connection channel;

[0016] The real-time transmission device generates a node handling strategy when determining that the corresponding companion proxy service plug-in is abnormal based on any heartbeat information, and transmits the node handling strategy to the corresponding companion proxy service plug-in through the strategy transmission connection channel.

[0017] Optionally, a data transmission connection channel, a policy transmission connection channel and a heartbeat information connection channel are established based on the WebSocket mechanism, and a long connection between the real-time transmission device and each connection channel is established based on a long connection bus.

[0018] Optionally, each companion proxy service plug-in includes a base layer and an application layer;

[0019] The basic layer includes: read service sub-plugin and write service sub-plugin;

[0020] When an update occurs to the application cluster in the master data center, the base layer of the companion proxy service plug-in corresponding to the master data center takes effect to read the service sub-plug-in, and the base layer of the companion proxy service plug-in corresponding to each slave data center takes effect to write the service sub-plug-in;

[0021] The application layer includes: reading rule sub-plug-in, flow control sub-plug-in, mode management sub-plug-in and node disposal sub-plug-in;

[0022] The reading rule sub-plug-in adjusts the reading rules according to the monitoring results of memory calculation, the flow control sub-plug-in adjusts the transmission flow according to the monitoring results of the transmission status, the mode management sub-plug-in switches the deployment mode, and the node disposal sub-plug-in executes the self-healing mechanism for the companion proxy service plug-in according to the node disposal strategy.

[0023] Optionally, by overriding the premain method in the Java Agent, the sub-plugin of the base layer can be enabled when an update action occurs in the application cluster of the primary data center; by overriding the agentmain method in the Java Agent, the sub-plugin of the application layer can be enabled during the synchronous data transmission process.

[0024] Optionally, the real-time transmission device includes: an incoming phase message queue; the incoming phase message queue includes: a metadata queue and a business data queue;

[0025] The companion proxy service plug-in corresponding to the main data center divides the synchronized data into multiple business data according to the segmentation rules, determines the metadata corresponding to the multiple business data, and identifies the ID information of the metadata corresponding to the multiple business data respectively;

[0026] Compress multiple business data into business data blocks respectively, and add corresponding metadata ID information to the multiple business data blocks respectively;

[0027] Multiple business data blocks and corresponding metadata are transmitted to the real-time transmission device through the data transmission connection channel according to the transmission rules of the companion proxy service plug-in corresponding to the main data center, wherein multiple business data blocks are transmitted to the business data queue and the corresponding metadata are transmitted to the metadata queue.

[0028] Optionally, the real-time transmission device includes: a memory computing unit connected to the incoming phase message queue; the memory computing unit includes: a sequential data restoration module, a metadata verification module and a synchronous data transmission module;

[0029] The sequential data restoration module receives a plurality of business data blocks and corresponding metadata, and restores the corresponding plurality of business data blocks into sequential data according to the metadata;

[0030] The metadata verification module performs format verification of the metadata corresponding to the sequential data in sequence, and transmits the format verification results to the corresponding companion proxy service plug-in of the main data center through the policy transmission connection channel;

[0031] The synchronous data outgoing module processes the sequential data into synchronous data.

[0032] Optionally, the sequential data restoration module includes: four memory data storage areas and a unified disposal area;

[0033] Divide the metadata and corresponding business data blocks into different memory data storage areas based on the hash result of the metadata ID information;

[0034] Restore multiple business data blocks into sequential data according to the method group loop mode;

[0035] The method group includes: planning a restoration computing task, broadcasting the restoration computing task to each memory data storage area, executing the restoration computing task in each memory data storage area respectively, sending the results of executing the restoration computing task in each memory data storage area to a unified processing area, and forming sequential data in the unified processing area.

[0036] Optionally, the real-time transmission device includes: an outgoing phase message queue, the outgoing phase message queue is connected to the memory computing unit; the outgoing phase message queue includes: an outgoing data queue corresponding to each slave data center;

[0037] The synchronous data outgoing module transmits the synchronous data to the outgoing data queue corresponding to each slave data center;

[0038] Each outgoing data queue transmits the synchronization data to the corresponding companion proxy service plug-in of each slave data center through each data transmission connection channel.

[0039] Optionally, the real-time transmission device includes: a management service unit; the management service unit includes: a data verification module, a capacity assessment module, a bandwidth monitoring module, a mode management module, a node monitoring module and a configuration center;

[0040] During the synchronous data transmission process, the metadata verification module sends the format verification result to the data verification module, and the data verification module transmits the format verification result to the corresponding companion proxy service plug-in of the main data center through the policy transmission connection channel;

[0041] The capacity assessment module obtains the data transmission volume of the data transmission connection channel of the companion proxy service plug-in corresponding to the primary data center, calculates the transmission capacity threshold based on the data transmission volume, and transmits the transmission capacity threshold to the companion proxy service plug-in corresponding to the primary data center through the policy transmission connection channel;

[0042] The bandwidth monitoring module obtains the data transmission bandwidth of the data transmission connection channel of the companion proxy service plug-in corresponding to the primary data center, calculates the bandwidth occupancy ratio based on the data transmission bandwidth, and transmits the bandwidth occupancy ratio to the companion proxy service plug-in corresponding to the primary data center through the policy transmission connection channel;

[0043] The mode management module determines the cluster deployment mode or the stand-alone deployment mode according to the mode of each data center;

[0044] The node monitoring module determines the node health status of the corresponding companion proxy service plug-in based on each heartbeat signal, generates a corresponding node handling strategy based on the abnormal node health status, and transmits the node handling strategy to the corresponding companion proxy service plug-in through the policy transmission connection channel;

[0045] The configuration center provides plug-in configuration information to each companion proxy service plug-in through each policy transmission connection channel.

[0046] In a second aspect, the present application provides a data synchronization method, which is applied to a data synchronization system. The data synchronization system includes: a real-time transmission device and a companion proxy service plug-in constructed for each data center; the data synchronization method includes:

[0047] Each companion proxy service plug-in is started when an update action occurs in the application cluster of the main data center, and establishes a data transmission connection channel, a policy transmission connection channel and a heartbeat information connection channel with the real-time transmission device respectively;

[0048] The companion proxy service plug-in corresponding to the main data center reads the synchronization data from the application cluster of the main data center according to the reading rules, compresses the synchronization data into multiple sub-data blocks according to the segmentation rules, and transmits the multiple sub-data blocks to the real-time transmission device through the data transmission connection channel according to the transmission rules;

[0049] The real-time transmission device performs memory calculation on multiple sub-data blocks and restores them into synchronous data, and transmits the synchronous data to the companion proxy service plug-in corresponding to each slave data center through each data transmission connection channel, so that the companion proxy service plug-in corresponding to each slave data center writes the synchronous data into the Redis of the corresponding slave data center;

[0050] The real-time transmission device monitors the transmission status of the data transmission connection channel between the memory computing and the companion proxy service plug-in corresponding to the main data center, and transmits the monitoring result to the companion proxy service plug-in corresponding to the main data center through the policy transmission connection channel;

[0051] The companion proxy service plug-in corresponding to the main data center updates the reading rules and transmission rules based on the monitoring results;

[0052] Each companion proxy service plug-in transmits heartbeat information to the real-time transmission device through the corresponding heartbeat information connection channel;

[0053] The real-time transmission device generates a corresponding node handling strategy when determining that the corresponding companion proxy service plug-in is abnormal based on any heartbeat information, and transmits the node handling strategy to the corresponding companion proxy service plug-in through the strategy transmission connection channel.

[0054] At least one of the above technical solutions adopted in this application can achieve the following beneficial effects:

[0055] Build a companion proxy service plug-in to decouple data transmission from application clusters and Redis.

[0056] The companion proxy service plug-in supports hot-swappable service switching, which can realize the synchronous switching of data transmission direction after the master and slave data center roles are switched.

[0057] Based on the capabilities of the companion proxy service plug-in, we build real-time awareness capabilities for business application clusters and Redis data updates, and achieve near-real-time synchronous writing between master and slave data centers.

[0058] Establish multiple connection channels and transmit synchronization data, management policies and heartbeat information through different connection channels respectively, isolate the mutual influence between different types of data, and improve data transmission efficiency.

[0059] During the transmission process, the synchronized data is first fragmented and compressed, and then the fragmented and compressed sub-data blocks are transmitted according to the transmission rules, which improves the transmission efficiency of data synchronization and ensures the timeliness of data synchronization.

[0060] Build in-memory computing to perform operations such as restoration in the in-memory computing area for synchronized data that has been fragmented and compressed during transmission. This improves data processing efficiency using in-memory computing and further ensures the real-time capability of data synchronization.

[0061] By monitoring memory calculation, transmission status and heartbeat information, data synchronization efficiency is guaranteed in various scenarios such as poor network quality and data transmission accumulation. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0063] Figure 1 A schematic diagram illustrating an implementation structure of a data synchronization system according to an embodiment of the present application is shown;

[0064] Figure 2 A schematic diagram illustrating an implementation logic of a data synchronization system according to an embodiment of the present application is shown;

[0065] Figure 3 A schematic diagram illustrating an implementation flow of a data synchronization method according to an embodiment of the present application is shown;

[0066] Figure 4 A schematic diagram showing the implementation structure of a data synchronization system according to another embodiment of the present application is shown;

[0067] Figure 5 A schematic diagram illustrating implementation logic of a data synchronization system according to another embodiment of the present application is shown. DETAILED DESCRIPTION

[0068] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0069] The idea of this application is that Redis, as a key technical component, plays an important role in the construction of enterprise application system architecture. With the advancement of enterprise digital transformation and business continuity, it is necessary to actively explore Redis cross-data center synchronization solutions to effectively deal with data synchronization tasks with different data volumes and different application requirements. In the current business scenario requirements, how to build real-time data synchronization capabilities, improve transmission quality and efficiency, strengthen process monitoring, and support flexible online mode adjustments have become technical problems that need to be solved urgently. In practice, the existing Redis cross-data center synchronization solution has obvious shortcomings in real-time capabilities, lacks flexibility in the service process, and cannot flexibly support deployment modes, synchronization directions, traffic control, etc. In addition, the existing data transmission mode has problems such as mixed data transmission, verification difficulties, and insufficient management and scheduling capabilities, which makes it impossible to effectively guarantee the timeliness and quality of transmission.

[0070] In response to the above-mentioned drawbacks, this application proposes a data synchronization system and a data synchronization method, constructs a companion proxy service plug-in and a real-time transmission device, and connects the companion proxy service plug-in and the real-time transmission device through multiple connection channels to solve the above-mentioned problems.

[0071] Figure 1 A schematic diagram of the implementation structure of a data synchronization system proposed in one embodiment of the present application is shown; Figure 2 A schematic diagram of the implementation logic of a data synchronization system proposed in one embodiment of the present application is shown; Figure 3 The following is a schematic diagram showing the implementation process of the data synchronization method proposed in one embodiment of the present application. Figure 1 and Figure 2 The data synchronization system shown is Figure 3 The data synchronization method shown is described.

[0072] Reference Figure 1 and Figure 2As shown, the data synchronization system includes: a real-time transmission device and a companion proxy service plug-in built for each data center.

[0073] exist Figure 1 and Figure 2 In this example, three data centers (Data Center A, Data Center B, and Data Center C) are used. Any of the three data centers can serve as the master data center, with the other two serving as slave data centers. The master data center can synchronize data to each slave data center using the data synchronization method proposed in this embodiment. To clarify the subsequent description of the data synchronization method, this embodiment uses Data Center A as the master data center, and Data Centers B and C as slave data centers.

[0074] In step S310, each companion proxy service plug-in is started when an update action occurs in the application cluster of the main data center, and establishes a data transmission connection channel, a policy transmission connection channel and a heartbeat information connection channel with the real-time transmission device respectively.

[0075] In existing technologies, it is usually necessary to complete the writing of synchronized data to Redis in the master data center before synchronizing the synchronized data to other slave data centers. However, the technical solution proposed in this embodiment enables data synchronization from the master data center to other slave data centers when an update occurs in the application cluster in the master data center. When data synchronization is enabled from the master data center to other slave data centers, Redis in the master data center continues to write synchronized data normally.

[0076] When an application cluster in the primary data center is updated, each RD TransferAgent plugin establishes three connection channels. These channels include: a DataUpload Channel for synchronous data transfer, a StrategyTransChannel for managing policy transfers, and a HeartBeat Channel for heartbeat information transmission. The channel ID is composed of the RD TransferAgent plugin ID and the channel type.

[0077] For example, in Figure 1 In the case shown, when the data of application cluster A is updated, the companion proxy service plug-in A establishes data transmission connection channel A, policy transmission connection channel A and heartbeat information connection channel A with the real-time transmission device; the companion proxy service plug-in B establishes data transmission connection channel B, policy transmission connection channel B and heartbeat information connection channel B with the real-time transmission device; the companion proxy service plug-in C establishes data transmission connection channel C, policy transmission connection channel C and heartbeat information connection channel C with the real-time transmission device.

[0078] In step S320, the companion proxy service plug-in corresponding to the main data center reads the synchronization data from the application cluster of the main data center according to the reading rules, compresses the synchronization data into multiple sub-data blocks according to the segmentation rules, and transmits the multiple sub-data blocks to the real-time transmission device through the data transmission connection channel according to the transmission rules.

[0079] The companion proxy service plug-in corresponding to the master data center takes effect as a read service role, and the companion proxy service plug-in corresponding to each slave data center takes effect as a write service role. The companion proxy service plug-in corresponding to the master data center takes effect based on the read service role, and reads synchronization data from the application cluster of the master data center according to the read rules. The read rules are updated based on the monitoring results of the memory calculation monitored by the real-time transmission device. The companion proxy service plug-in corresponding to the master data center segments and compresses the synchronization data into multiple sub-data blocks according to the segmentation rules. The segmentation rules are preset fixed rules. The companion proxy service plug-in corresponding to the master data center transmits the multiple sub-data blocks to the real-time transmission device through the data transmission connection channel according to the transmission rules. The transmission rules are updated based on the monitoring results of the transmission status monitored by the real-time transmission device.

[0080] For example, in Figure 2 In the illustrated scenario, Companion Proxy Service Plugin A assumes the Read service role, while Companion Proxy Service Plugin B and Companion Proxy Service Plugin C assume the Write service role. Companion Proxy Service Plugin A reads synchronized data from the application cluster in Data Center A according to the read rule, slices and compresses the synchronized data into multiple sub-data blocks according to the segmentation rule, and transmits the sub-data blocks to the real-time transmission device via data transmission connection channel A according to the transmission rule.

[0081] In step S330, the real-time transmission device performs memory calculation on multiple sub-data blocks and restores them into synchronous data, and transmits the synchronous data to the companion proxy service plug-in corresponding to each slave data center through each data transmission connection channel, so that the companion proxy service plug-in corresponding to each slave data center writes the synchronous data into the Redis of the corresponding slave data center.

[0082] The real-time transmission device performs in-memory computations on the multiple sub-data blocks within its in-memory computation area, converting the sub-data blocks into synchronized data. The real-time transmission device transmits the synchronized data to the associated proxy service plug-in corresponding to each slave data center via each data transmission connection channel. The associated proxy service plug-in corresponding to each slave data center activates its write service role and writes the synchronized data to the Redis server in the slave data center.

[0083] For example, in Figure 2In the illustrated scenario, the real-time transmission device performs in-memory computation on multiple sub-data blocks within its in-memory computation area, converting the sub-data blocks into synchronized data. The real-time transmission device transmits the synchronized data to companion proxy service plug-in B via data transmission connection channel B. Companion proxy service plug-in B writes the synchronized data to Redis-B in data center B. The real-time transmission device transmits the synchronized data to companion proxy service plug-in C via data transmission connection channel C. Companion proxy service plug-in C writes the synchronized data to Redis-C in data center C.

[0084] In step S340, the real-time transmission device monitors the transmission status of the data transmission connection channel of the memory computing and the companion proxy service plug-in corresponding to the main data center, and transmits the monitoring results to the companion proxy service plug-in corresponding to the main data center through the policy transmission connection channel.

[0085] The real-time transmission device monitors the memory calculation status while performing memory calculation, and also monitors the transmission status of the data transmission connection channel of the companion proxy service plug-in corresponding to the main data center, thereby transmitting the monitoring results of the memory calculation and the monitoring results of the transmission status to the companion proxy service plug-in corresponding to the main data center through the policy transmission connection channel.

[0086] For example, in Figure 2 In the case shown, the real-time transmission device monitors the transmission status of the memory calculation and data transmission connection channel A, and transmits the monitoring results to the companion proxy service plug-in A through the policy transmission connection channel A.

[0087] Step S350: The companion proxy service plug-in corresponding to the primary data center updates the reading rules and transmission rules according to the monitoring results.

[0088] The companion proxy service plug-in corresponding to the main data center updates the reading rules and transmission rules based on the monitoring results.

[0089] For example, in Figure 2 In the case shown, the companion proxy service plug-in A updates the reading rules and the transmission rules according to the monitoring results.

[0090] In step S360, each companion proxy service plug-in transmits heartbeat information to the real-time transmission device through the corresponding heartbeat information connection channel.

[0091] Each companion proxy service plug-in sends heartbeat information to the real-time transmission device through the corresponding heartbeat information connection channel.

[0092] For example, in Figure 2In the case shown, the companion proxy service plug-in A transmits the heartbeat packet to the real-time transmission device through the heartbeat information connection channel A, the companion proxy service plug-in B transmits the heartbeat packet to the real-time transmission device through the heartbeat information connection channel B, and the companion proxy service plug-in C transmits the heartbeat packet to the real-time transmission device through the heartbeat information connection channel C.

[0093] In step S370, the real-time transmission device generates a node handling strategy when determining that the corresponding companion proxy service plug-in is abnormal based on any heartbeat information, and transmits the node handling strategy to the corresponding companion proxy service plug-in through the policy transmission connection channel.

[0094] The real-time transmission device detects the node health status of each companion proxy service plug-in based on each heartbeat information, and promptly generates and issues node handling strategies such as restart for abnormal nodes. Each node handling strategy is transmitted to the corresponding companion proxy service plug-in through each strategy transmission connection channel.

[0095] For example, in Figure 2 In the illustrated scenario, the real-time transmission device detects and senses heartbeat packets from heartbeat information connection channel A, heartbeat information connection channel B, and heartbeat information connection channel C. If, based on the heartbeat packets from heartbeat information connection channel B, companion proxy service plug-in B is determined to be abnormal, a node handling policy is promptly generated and transmitted to companion proxy service plug-in B via policy transmission connection channel B, causing companion proxy service plug-in B to execute a self-healing mechanism based on the node handling policy.

[0096] from Figures 1 to 2 The system shown and Figure 3 As can be seen from the method shown, the data synchronization system and data synchronization method provided by this application have the following advantages:

[0097] 1. Regarding transmission efficiency and real-time performance:

[0098] 1. Leveraging the capabilities of the companion proxy service plug-in, we build real-time awareness of data updates between business application clusters and Redis, enabling near-real-time synchronous writes between master and slave data centers. When the application cluster in the master data center initiates a data update, each companion proxy service plug-in immediately detects it, initiates the data synchronization process, and writes data to Redis in each slave data center simultaneously.

[0099] 2. Establish multiple connection channels and transmit synchronization data, management policies and heartbeat information through different connection channels respectively, isolate the mutual influence between different types of data, and improve data transmission efficiency.

[0100] 3. During the transmission process, the synchronized data is first fragmented and compressed, and then the fragmented and compressed sub-data blocks are transmitted according to the transmission rules, which improves the transmission efficiency of data synchronization and ensures the timeliness of data synchronization.

[0101] 4. Build in-memory computing to restore synchronized data that has been fragmented and compressed during transmission in the in-memory computing area. This improves data processing efficiency and further ensures the real-time capability of data synchronization.

[0102] 5. By monitoring memory calculation, transmission status and heartbeat information, data synchronization efficiency is guaranteed in various scenarios such as poor network quality and data transmission accumulation.

[0103] 2. Regarding flexibility:

[0104] 1. Build a companion proxy service plug-in to decouple data transmission from application clusters and Redis.

[0105] 2. The companion proxy service plug-in supports hot-swappable service switching, which can realize the synchronous switching of data transmission direction after the master and slave data center roles are switched.

[0106] Figure 4 A schematic diagram of the implementation structure of a data synchronization system proposed in another embodiment of the present application is shown. Figure 5 A schematic diagram of the implementation logic of the data synchronization system proposed in another embodiment of the present application is shown. Figure 4 and Figure 5 As shown, the data synchronization method is further described.

[0107] Reference Figure 4 and Figure 5 As shown, the data synchronization system includes: various accompanying proxy service plug-ins and real-time transmission devices.

[0108] The companion proxy service plug-in includes: basic layer and application layer.

[0109] The base layer includes: a read service sub-plugin and a write service sub-plugin.

[0110] The application layer includes: reading rule sub-plug-in, flow control sub-plug-in, mode management sub-plug-in and node disposal sub-plug-in.

[0111] The real-time transmission device includes: an incoming phase message queue, a memory calculation unit, an outgoing phase message queue and a management service unit.

[0112] The message queues in the incoming phase include: metadata queue and business data queue.

[0113] The memory computing unit includes: a sequential data restoration module, a metadata verification module and a synchronous data transmission module; the sequential data restoration module includes: four memory data storage areas and a unified processing area.

[0114] The outgoing phase message queue includes: an outgoing data queue corresponding to each slave data center.

[0115] The management service unit includes: data verification module, capacity assessment module, bandwidth monitoring module, mode management module, node monitoring module and configuration center.

[0116] exist Figure 4 and Figure 5 In this example, three data centers (Data Center A, Data Center B, and Data Center C) are used. Any of the three data centers can serve as the master data center, with the other two serving as slave data centers. The master data center can synchronize data to each slave data center using the data synchronization method proposed in this embodiment. To clarify the subsequent description of the data synchronization method, this embodiment uses Data Center A as the master data center, and Data Centers B and C as slave data centers.

[0117] Based on Java Agent technology, various accompanying agent service plug-ins are built, which include a basic capability framework enhanced by bytecode and various data transmission capability components.

[0118] In the basic capability framework of bytecode enhancement, by overriding the addTransformer method and retransformClasses method in the Instrumentation API, interception is performed during class loading and reloading to modify the bytecode of the input class or dynamically inject jar files into the classpath of the ClassLoader.

[0119] In various data transmission capability components, a two-layer design and lazy loading mode are provided.

[0120] The basic layer includes a read service sub-plugin (data reader service) and a write service sub-plugin (data writer service), which are used to control the direction of synchronous data transmission.

[0121] The application layer includes reading rule sub-plug-in, flow control sub-plug-in, mode management sub-plug-in and node disposal sub-plug-in, etc.

[0122] Each companion proxy service plug-in overrides the premain method in the Java Agent to enable the base-layer sub-plug-in to take effect when an update occurs in the application cluster of the primary data center. Each companion proxy service plug-in overrides the agentmain method in the JavaAgent to enable the application-layer sub-plug-in to take effect during the synchronous data transmission process.

[0123] Each companion proxy service plug-in is started through premain, and establishes a data transmission connection channel, a policy transmission connection channel and a heartbeat information connection channel based on the WebSocket mechanism, and establishes a long connection between the real-time transmission device and each connection channel based on the long connection bus.

[0124] In the WebSocket message header, an extension field is enabled to store the connection channel identification field (ChannelID). During the connection channel establishment process, each companion proxy service plug-in calls the connection channel creation method to create a data transmission connection channel, a policy transmission connection channel, and a heartbeat information connection channel based on the extended and customized WebSocket protocol. This multi-channel isolation management mechanism ensures that data does not affect each other and enhances the real-time performance of data transmission.

[0125] The real-time transmission device establishes a long connection with each connection channel. According to different functional purposes, each unit / module of the real-time transmission device is connected to the corresponding connection channel.

[0126] For example: the node monitoring module senses each heartbeat signal from the heartbeat information connection channel, detects the node health status of each companion proxy service plug-in, generates corresponding node disposal strategies based on the abnormal node health status, and transmits the node disposal strategies to the corresponding companion proxy service plug-ins through the policy transmission connection channel. For example, for abnormal nodes, node disposal strategies such as restart are issued in a timely manner through the policy transmission connection channel.

[0127] For example: the capacity assessment module obtains the data transmission volume of the data transmission connection channel of the companion proxy service plug-in corresponding to the main data center, calculates the transmission capacity threshold based on the data transmission volume, and transmits the transmission capacity threshold to the companion proxy service plug-in corresponding to the main data center through the policy transmission connection channel; the bandwidth monitoring module obtains the data transmission bandwidth of the data transmission connection channel of the companion proxy service plug-in corresponding to the main data center, calculates the bandwidth occupancy ratio based on the data transmission bandwidth, and transmits the bandwidth occupancy ratio to the companion proxy service plug-in corresponding to the main data center through the policy transmission connection channel; the companion proxy service plug-in corresponding to the main data center adjusts the transmission traffic (such as flow control, pause, etc.) in a timely manner based on the monitoring results such as the transmission capacity threshold and the bandwidth occupancy ratio.

[0128] For example: the mode management module determines the cluster deployment mode (Cluster) or the standalone deployment mode (Standalone) according to the mode of each data center, and transmits the cluster deployment mode or the standalone deployment mode to the companion proxy service plug-in corresponding to each data center through each policy transmission connection channel; the mode management sub-plug-in of the application layer of the companion proxy service plug-in corresponding to each data center switches the deployment mode accordingly.

[0129] Based on the master / slave role of each data center and the transmission direction defined in premain, each companion proxy service plug-in activates the base layer's subplug-in. When the companion proxy service plug-in corresponds to the master data center, the base layer activates the read service subplug-in, effectively taking on the Data Reader service role. When the companion proxy service plug-in corresponds to the slave data center, the base layer activates the write service subplug-in, effectively taking on the Data Writer service role.

[0130] After the sub-plug-in of the base layer takes effect, the synchronization data of the master data center is transmitted through its data transmission connection channel, and the synchronization data of each slave data center is transmitted through the corresponding data transmission connection channel.

[0131] If the primary and secondary data centers swap roles, the management service unit's configuration center can distribute the effective sub-plugin configuration information for the base layer of each companion proxy service plug-in through each policy transmission connection channel. Each companion proxy service plug-in redefines the transmission direction in premain based on the effective sub-plugin configuration information of the base layer, and restarts to take effect on the corresponding base layer sub-plugin.

[0132] According to agentmain, each sub-plugin in the application layer is started and activated. Each companion agent service plug-in relies on the monitoring results and node handling policies issued by the management service unit through each policy transmission connection channel, and pulls the relevant plug-in configuration information from the configuration center of the management service unit.

[0133] For the companion proxy service plug-in corresponding to the main data center, it takes effect hot-swappable reading rule sub-plug-in, flow control sub-plug-in, mode management sub-plug-in and node disposal sub-plug-in.

[0134] The read rule sub-plug-in adjusts the read rules based on the in-memory computing monitoring results delivered by the real-time transmission device via policy transmission connection channel A. Specifically, the read rule sub-plug-in adjusts the read rules based on the format verification results delivered by the data verification module of the management service unit. The format verification results of the data verification module are derived from the metadata format verification results of the metadata verification module of the in-memory computing unit.

[0135] The flow control sub-plug-in adjusts the transmission flow rate based on the monitoring results of the transmission status of the data transmission connection channel A, which are sent by the real-time transmission device via the policy transmission connection channel A. Specifically, the flow control sub-plug-in adjusts the transmission flow rate based on the transmission capacity threshold sent by the capacity assessment module of the management service unit and the bandwidth occupancy ratio sent by the bandwidth monitoring module.

[0136] The Mode Management subplugin switches deployment modes. This subplugin provides differentiated management of Redis deployment modes. The proxy service plug-in associated with the read subplugin in the primary data center and the write subplugin in the secondary data center both provide the ability to switch between cluster and standalone deployment modes.

[0137] The node handling sub-plugin executes a self-healing mechanism on the companion proxy service plug-in based on the node handling policy delivered by the real-time transmission device via the corresponding policy transmission connection channel. Specifically, the node handling sub-plugin schedules a self-healing restart mechanism based on the node handling policy for heartbeat loss delivered by the node monitoring module of the management service unit, restarting the companion proxy service plug-in and resuming data transmission.

[0138] The companion proxy service plug-in corresponding to the main data center divides the synchronized data into multiple business data according to the segmentation rules, determines the metadata corresponding to the multiple business data, and identifies the ID information of the metadata corresponding to the multiple business data respectively.

[0139] The metadata part stores attribute information about business data, including business data category ID, business data segmentation order, business data structure, business data volume, etc.

[0140] The plurality of business data are respectively compressed into business data blocks, and corresponding metadata ID information is respectively added to the plurality of business data blocks.

[0141] Each piece of business data is compressed separately to form multiple business data blocks, and metadata ID information is added to each business data block.

[0142] Multiple business data blocks and corresponding metadata are transmitted to the real-time transmission device through the data transmission connection channel according to the transmission rules of the companion proxy service plug-in corresponding to the main data center, wherein multiple business data blocks are transmitted to the business data queue and the corresponding metadata are transmitted to the metadata queue.

[0143] The business data block and the corresponding metadata enter the data transmission connection channel of the companion proxy service plug-in corresponding to the main data center, and are transmitted to the incoming phase message queue of the real-time transmission device according to the transmission rules.

[0144] The incoming message queue supports multiple message queues, including Rocket MQ, Kafka, and Rabbit MQ. By abstracting away common queue transmission features, the incoming message queue builds a metadata queue and a business data queue based on the underlying message queue. The metadata queue and business data queue transmit data in parallel. This separation of the metadata queue and business data queue increases data transmission throughput and allows for verification of data transmission quality through metadata verification mode.

[0145] The incoming message queue receives processed synchronization data from the primary data center's companion proxy service plug-in's data transmission channel. It is categorized into business data blocks and metadata. Metadata is transferred to the metadata queue; business data blocks are transferred to the business data queue.

[0146] In addition, the business data queue supports horizontal capacity expansion, can timely perceive the congestion of the business data queue, and achieve flexible expansion and contraction.

[0147] Since the fragmentation order of business data blocks is stored in the corresponding metadata, the transmission of business data blocks in the business data queue is not restricted by sequence. The parallel transmission of each business data block can improve the timeliness of data transmission.

[0148] After receiving each business data block and the corresponding metadata, the incoming stage message queue transmits them to the memory computing unit connected to the incoming stage message queue.

[0149] The sequential data restoration module receives multiple business data blocks and corresponding metadata, and restores the corresponding multiple business data blocks into sequential data according to the metadata.

[0150] The sequential data restoration module plans four in-memory data storage areas and a unified processing area. Based on the hash of the metadata ID information, the metadata and corresponding business data blocks are divided into different in-memory data storage areas, achieving co-location of metadata and corresponding business data blocks.

[0151] As data increases and data processing capacity is calculated, the sequential data restoration module can also promptly receive the expansion strategy of the management service unit, promptly expand the memory data storage area by multiples of four, and simultaneously implement the new hash function. Newly received metadata and corresponding business data blocks are juxtaposed and divided into the new memory data storage area. When the storage capacity of the original memory data storage area drops below 50%, the hash function is automatically switched to juxtapose the newly received metadata and corresponding business data blocks into the original memory data storage area.

[0152] Calculations are initiated directly in each memory data storage area. During calculations, multiple business data blocks are converted into sequential data according to the method group loop pattern.

[0153] The method group includes: planning a restoration computing task, broadcasting the restoration computing task to each memory data storage area, executing the restoration computing task in each memory data storage area respectively, sending the results of executing the restoration computing task in each memory data storage area to a unified processing area, and forming sequential data in the unified processing area.

[0154] The compute() method is used to plan the restoration computation task, which is then broadcasted using the broadcast method. All computational logic for the restoration computation task is distributed to the corresponding in-memory data storage area. The call() method executes the specific computation logic within the in-memory data storage area, and the inform() method sends the results to a centralized processing area, where sequential data is generated. All computational data is directly retrieved from the in-memory data storage area, achieving a closed-loop memory loop through task planning, broadcast transmission, task execution, and result upload.

[0155] The metadata verification module performs format verification of the metadata corresponding to the sequential data in sequence, and transmits the format verification result to the companion proxy service plug-in corresponding to the main data center through the policy transmission connection channel.

[0156] The metadata verification module invokes metadata validation logic to perform format verification on the metadata corresponding to the sequential data. The metadata verification module can transmit the format verification results to the data verification module of the management service unit. The data verification module then transmits the format verification results to the read rule sub-plug-in of the application layer of the companion proxy service plug-in corresponding to the primary data center via the policy transmission connection channel of the companion proxy service plug-in corresponding to the primary data center. The read rule sub-plug-in of the application layer of the companion proxy service plug-in corresponding to the primary data center updates the read rules based on the format verification results.

[0157] The synchronous data outgoing module processes the sequential data into synchronous data.

[0158] The synchronous data outbound module reorganizes sequential data into synchronized data and transmits it to the outbound message queue connected to the in-memory computing unit. This in-memory computing, which performs a series of calculations such as metadata verification and synchronous data reorganization, improves data processing efficiency and further ensures real-time data synchronization.

[0159] The outbound message queue supports multiple message queues, including Rocket MQ, Kafka, and Rabbit MQ. By abstracting away the common features of queue transmission, the outbound message queue builds multiple outbound data queues corresponding to each slave data center based on the underlying message queue.

[0160] The synchronous data outgoing module transmits the synchronous data to the outgoing data queues corresponding to each slave data center respectively, and each outgoing data queue transmits the synchronous data to the companion proxy service plug-in corresponding to each slave data center through each data transmission connection channel.

[0161] In parallel with the outgoing data queues corresponding to each slave data center, parallel processing of synchronous data writing can be achieved, reducing the latency of cross-data center transmission and further improving data transmission efficiency.

[0162] The companion proxy service plug-in corresponding to each slave data center writes the synchronized data to the Redis of each slave data center.

[0163] The data synchronization system and data synchronization method proposed in this embodiment are as follows:

[0164] 1. The real-time transmission device uses persistent connections and message queues to continuously improve data transmission efficiency and quality. Persistent connections are used to transmit management policies, ensuring the quality and efficiency of management policy transmission. The incoming message queue includes a metadata queue and a business data queue. The metadata queue transmits data attributes, while the data queue transmits fragmented and compressed business data blocks. The outgoing message queue includes an outgoing data queue corresponding to each slave data center. Parallel transmission ensures transmission efficiency.

[0165] 2. In the face of the classified transmission of metadata and business data blocks, sequential data restoration, metadata verification, and synchronous data reorganization and transmission are achieved through the memory computing mechanism.

[0166] 3. Each module of the management service unit monitors the transmission status, network status, transmission accuracy, deployment mode, etc. in real time, and synchronizes management strategies such as capacity assessment, bandwidth monitoring, deployment mode, node monitoring, data quality, and transmission direction to the strategy transmission connection channel through a long connection, and transmits them to the master and slave data centers in real time, ensuring that the management strategy has real-time and online adjustment and effectiveness capabilities, thereby improving the level of flexibility.

[0167] It should be noted that the above-mentioned data synchronization system can implement the data synchronization method one by one, which will not be described in detail.

[0168] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A data synchronization system, characterized in that: The data synchronization system includes: a real-time transmission device and a companion proxy service plug-in built for each data center; Each companion proxy service plug-in is started when an update action occurs in the application cluster of the main data center, and establishes a data transmission connection channel, a policy transmission connection channel and a heartbeat information connection channel with the real-time transmission device respectively; The companion proxy service plug-in corresponding to the main data center reads the synchronization data from the application cluster of the main data center according to the reading rules, compresses the synchronization data into multiple sub-data blocks according to the segmentation rules, and transmits the multiple sub-data blocks to the real-time transmission device through the data transmission connection channel according to the transmission rules; The real-time transmission device performs memory calculation on multiple sub-data blocks and restores them into synchronous data, and transmits the synchronous data to the companion proxy service plug-in corresponding to each slave data center through each data transmission connection channel, so that the companion proxy service plug-in corresponding to each slave data center writes the synchronous data into the Redis of the corresponding slave data center; The real-time transmission device monitors the transmission status of the data transmission connection channel between the memory computing and the companion proxy service plug-in corresponding to the main data center, and transmits the monitoring result to the companion proxy service plug-in corresponding to the main data center through the policy transmission connection channel; The companion proxy service plug-in corresponding to the main data center updates the reading rules and transmission rules based on the monitoring results; Each companion proxy service plug-in transmits heartbeat information to the real-time transmission device through the corresponding heartbeat information connection channel; The real-time transmission device generates a node handling strategy when determining that the corresponding companion proxy service plug-in is abnormal based on any heartbeat information, and transmits the node handling strategy to the corresponding companion proxy service plug-in through the strategy transmission connection channel.

2. The data synchronization system according to claim 1, characterized in that: Based on the WebSocket mechanism, a data transmission connection channel, a policy transmission connection channel and a heartbeat information connection channel are established, and based on the long connection bus, a long connection is established between the real-time transmission device and each connection channel.

3. The data synchronization system according to claim 1, wherein: Each companion proxy service plug-in includes a basic layer and an application layer; The basic layer includes: read service sub-plugin and write service sub-plugin; When an update occurs to the application cluster in the master data center, the base layer of the companion proxy service plug-in corresponding to the master data center takes effect to read the service sub-plug-in, and the base layer of the companion proxy service plug-in corresponding to each slave data center takes effect to write the service sub-plug-in; The application layer includes: reading rule sub-plug-in, flow control sub-plug-in, mode management sub-plug-in and node disposal sub-plug-in; The reading rule sub-plug-in adjusts the reading rules according to the monitoring results of memory calculation, the flow control sub-plug-in adjusts the transmission flow according to the monitoring results of the transmission status, the mode management sub-plug-in switches the deployment mode, and the node disposal sub-plug-in executes the self-healing mechanism for the companion proxy service plug-in according to the node disposal strategy.

4. The data synchronization system according to claim 3, wherein: By rewriting the premain method in the Java Agent, the sub-plugins of the base layer can be activated when the application cluster in the main data center is updated; by rewriting the agentmain method in the Java Agent, the sub-plugins of the application layer can be activated during the synchronous data transmission process.

5. The data synchronization system according to claim 1, characterized in that: The real-time transmission device includes: an incoming phase message queue; the incoming phase message queue includes: a metadata queue and a business data queue; The companion proxy service plug-in corresponding to the main data center divides the synchronized data into multiple business data according to the segmentation rules, determines the metadata corresponding to the multiple business data, and identifies the ID information of the metadata corresponding to the multiple business data respectively; Compress multiple business data into business data blocks respectively, and add corresponding metadata ID information to the multiple business data blocks respectively; Multiple business data blocks and corresponding metadata are transmitted to the real-time transmission device through the data transmission connection channel according to the transmission rules of the companion proxy service plug-in corresponding to the main data center, wherein multiple business data blocks are transmitted to the business data queue and the corresponding metadata are transmitted to the metadata queue.

6. The data synchronization system according to claim 5, characterized in that: The real-time transmission device includes: a memory computing unit, the memory computing unit is connected to the incoming stage message queue; the memory computing unit includes: a sequential data restoration module, a metadata verification module and a synchronous data transmission module; The sequential data restoration module receives a plurality of business data blocks and corresponding metadata, and restores the corresponding plurality of business data blocks into sequential data according to the metadata; The metadata verification module performs format verification of the metadata corresponding to the sequential data in sequence, and transmits the format verification results to the corresponding companion proxy service plug-in of the main data center through the policy transmission connection channel; The synchronous data outgoing module processes the sequential data into synchronous data.

7. The data synchronization system according to claim 6, characterized in that: The sequential data restoration module includes: four memory data storage areas and a unified disposal area; Divide the metadata and corresponding business data blocks into different memory data storage areas based on the hash result of the metadata ID information; Restore multiple business data blocks into sequential data according to the method group loop mode; The method group includes: planning a restoration computing task, broadcasting the restoration computing task to each memory data storage area, executing the restoration computing task in each memory data storage area respectively, sending the results of executing the restoration computing task in each memory data storage area to a unified processing area, and forming sequential data in the unified processing area.

8. The data synchronization system according to claim 6, characterized in that: The real-time transmission device includes: an outgoing phase message queue, the outgoing phase message queue is connected to the memory computing unit; the outgoing phase message queue includes: an outgoing data queue corresponding to each slave data center; The synchronous data outgoing module transmits the synchronous data to the outgoing data queue corresponding to each slave data center; Each outgoing data queue transmits the synchronization data to the corresponding companion proxy service plug-in of each slave data center through each data transmission connection channel.

9. The data synchronization system according to claim 6, characterized in that: The real-time transmission device includes: a management service unit; the management service unit includes: a data verification module, a capacity assessment module, a bandwidth monitoring module, a mode management module, a node monitoring module and a configuration center; During the synchronous data transmission process, the metadata verification module sends the format verification result to the data verification module, and the data verification module transmits the format verification result to the corresponding companion proxy service plug-in of the main data center through the policy transmission connection channel; The capacity assessment module obtains the data transmission volume of the data transmission connection channel of the companion proxy service plug-in corresponding to the primary data center, calculates the transmission capacity threshold based on the data transmission volume, and transmits the transmission capacity threshold to the companion proxy service plug-in corresponding to the primary data center through the policy transmission connection channel; The bandwidth monitoring module obtains the data transmission bandwidth of the data transmission connection channel of the companion proxy service plug-in corresponding to the primary data center, calculates the bandwidth occupancy ratio based on the data transmission bandwidth, and transmits the bandwidth occupancy ratio to the companion proxy service plug-in corresponding to the primary data center through the policy transmission connection channel; The mode management module determines the cluster deployment mode or the stand-alone deployment mode according to the mode of each data center; The node monitoring module determines the node health status of the corresponding companion proxy service plug-in based on each heartbeat signal, generates a corresponding node handling strategy based on the abnormal node health status, and transmits the node handling strategy to the corresponding companion proxy service plug-in through the policy transmission connection channel; The configuration center provides plug-in configuration information to each companion proxy service plug-in through each policy transmission connection channel.

10. A data synchronization method, characterized in that: The data synchronization method is applied to a data synchronization system, which includes: a real-time transmission device and a companion proxy service plug-in built for each data center; the data synchronization method includes: Each companion proxy service plug-in is started when an update action occurs in the application cluster of the main data center, and establishes a data transmission connection channel, a policy transmission connection channel and a heartbeat information connection channel with the real-time transmission device respectively; The companion proxy service plug-in corresponding to the main data center reads the synchronization data from the application cluster of the main data center according to the reading rules, compresses the synchronization data into multiple sub-data blocks according to the segmentation rules, and transmits the multiple sub-data blocks to the real-time transmission device through the data transmission connection channel according to the transmission rules; The real-time transmission device performs memory calculation on multiple sub-data blocks and restores them into synchronous data, and transmits the synchronous data to the companion proxy service plug-in corresponding to each slave data center through each data transmission connection channel, so that the companion proxy service plug-in corresponding to each slave data center writes the synchronous data into the Redis of the corresponding slave data center; The real-time transmission device monitors the transmission status of the data transmission connection channel between the memory computing and the companion proxy service plug-in corresponding to the main data center, and transmits the monitoring result to the companion proxy service plug-in corresponding to the main data center through the policy transmission connection channel; The companion proxy service plug-in corresponding to the main data center updates the reading rules and transmission rules based on the monitoring results; Each companion proxy service plug-in transmits heartbeat information to the real-time transmission device through the corresponding heartbeat information connection channel; The real-time transmission device generates a node handling strategy when determining that the corresponding companion proxy service plug-in is abnormal based on any heartbeat information, and transmits the node handling strategy to the corresponding companion proxy service plug-in through the strategy transmission connection channel.

Images