Cross-regional information transmission system

Abstract

The invention discloses a cross-regional information transmission system, which comprises a core node, a plurality of regional nodes, a cluster connection module, a cross-regional load balancing module and a disaster recovery module, and is characterized in that the core node is in communication connection with the plurality of regional nodes and is used for synchronizing a global theme to each of the plurality of regions in real time through a configured global theme mapping table, synchronizing the asynchronous messages to each area regularly in batches; each region node in the plurality of region nodes is used for processing a local message of a corresponding region and synchronizing subscription information to the core node through an asynchronous mechanism; the cluster connection module is used for managing connection and communication of MQTT clusters of a plurality of regions; the cross-regional load balancing module is in communication connection with the cluster connection module and is used for dynamically distributing loads for the plurality of regions based on connection and communication of the MQTT clusters of the plurality of regions; the disaster recovery module is used for monitoring the communication states of the MQTT clusters of the multiple areas and switching to a standby path when the communication states are abnormal. According to the invention, the cross-regional information transmission efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of information transmission technology, and in particular to a cross-regional information transmission system. Background Art

[0002] With the development of the Internet of Things (IoT) and intelligent transportation systems, the connection between vehicles and the cloud has become an important foundation for promoting smart cars and related services. MQTT (Message Queuing Telemetry Transport), as a lightweight, low-bandwidth publish-subscribe messaging protocol, is widely used in vehicle communications and the Internet of Things.

[0003] At present, cross-regional information transmission is mainly achieved between MQTT clusters in different regions through MQTT bridging technology.

[0004] However, the application of bridging technology in cross-regional information transmission not only consumes a large amount of traffic, but also leads to poor reliability of information transmission when the network is not smooth. Summary of the invention

[0005] The main purpose of this application is to provide a cross-regional information transmission system to solve the problem that the existing cross-regional information transmission technology consumes a large amount of traffic and causes poor information transmission reliability when the network is not smooth.

[0006] In order to achieve the above objectives, in a first aspect, the present application provides a cross-regional information transmission system, including:

[0007] A core node, multiple regional nodes, a cluster connection module, a cross-region load balancing module, and a disaster recovery module, wherein the multiple regional nodes are configured in multiple regions, and each of the multiple regional nodes corresponds to one region among the multiple regions;

[0008] The core node is connected to multiple regional nodes for real-time synchronization of global topics to each of the multiple regions through the configured global topic mapping table, and periodically batch synchronization of asynchronous messages to each region;

[0009] Each of the multiple regional nodes is used to process local messages of the corresponding region and synchronize subscription information to the core node through an asynchronous mechanism;

[0010] The cluster connection module is used to manage the connection and communication of MQTT clusters in multiple regions, and feed back the connection and communication status of MQTT clusters in multiple regions to the core node;

[0011] The cross-region load balancing module is communicated with the cluster connection module to dynamically distribute loads to multiple regions based on the connection and communication of MQTT clusters in multiple regions;

[0012] The disaster recovery module is used to monitor the communication status of MQTT clusters in multiple regions and switch to the backup communication path when an abnormal communication status is detected.

[0013] In a possible implementation, the subscription information is synchronized to the core node through an asynchronous mechanism, including:

[0014] For each of the multiple regional nodes, a corresponding asynchronous message queue is configured for each regional node, wherein the asynchronous message queue caches subscription information to be synchronized arranged in sequence;

[0015] Within the set time window or information quantity threshold, when each regional node triggers batch synchronization, multiple subscription information to be synchronized arranged in sequence in the asynchronous message queue are packaged and synchronized to the core node.

[0016] In a possible implementation, the system further includes: a retransmission module;

[0017] The retransmission module is used to mark the subscription information as pending retransmission in the event of network interruption or transmission failure, and re-add the subscription message to be retransmitted to the asynchronous message queue, waiting for the next transmission.

[0018] In a possible implementation, the cluster connection module includes a path calculation submodule;

[0019] The path calculation submodule is used to calculate the optimal message transmission path across regions based on network conditions, regional node loads, and the Dijkstra algorithm.

[0020] In a possible implementation, the cluster connection module includes a data transmission submodule;

[0021] The data transmission submodule is used to obtain historical status messages and new status messages when a new status message is released;

[0022] Calculate the difference between the historical status message and the new status message using a differential algorithm to obtain differential data;

[0023] The difference data is compressed using a compression algorithm and then transmitted.

[0024] In one possible implementation, the cross-region load balancing module includes a message distribution submodule;

[0025] The message distribution submodule is used to perform hash calculation on the message subject to obtain the hash value corresponding to the message subject;

[0026] Sort multiple regional nodes according to hash values ​​to form hash rings corresponding to the multiple regional nodes;

[0027] The corresponding regional node is matched on the hash ring according to the hash value corresponding to the message subject, and the message content corresponding to the message subject is distributed to the corresponding regional node on the hash ring.

[0028] In a possible implementation, the cross-region load balancing module includes a node expansion submodule;

[0029] The node expansion submodule is used to monitor the load of multiple regional nodes in real time;

[0030] If the load of any regional node among multiple regional nodes exceeds a preset threshold, a new regional node is automatically expanded and added to the hash ring.

[0031] In one possible implementation, the cross-region load balancing module includes a load migration submodule;

[0032] The load migration submodule is used to migrate the load exceeding the preset threshold to a regional node adjacent to any regional node when the load of any regional node among the multiple regional nodes exceeds a preset threshold.

[0033] In a possible implementation, the disaster recovery module includes a switching submodule;

[0034] The switching submodule is used to periodically monitor the health status of multiple regional nodes through a fault detection algorithm;

[0035] When any of the multiple regional nodes fails, the communication is switched to a healthy standby node or path according to the network topology and node health status.

[0036] In a possible implementation, the disaster recovery module includes a fault recovery module;

[0037] The fault recovery module is used to complete the missing data through the data synchronization mechanism when any regional node among multiple regional nodes recovers from the fault.

[0038] The embodiment of the present invention provides a cross-region information transmission system, including: a core node, multiple regional nodes, a cluster connection module, a cross-region load balancing module and a disaster recovery module, wherein the multiple regional nodes are configured in multiple regions, and each of the multiple regional nodes corresponds to one of the multiple regions; the core node is connected in communication with the multiple regional nodes, and is used to synchronize the global topic to each of the multiple regions in real time through the configured global topic mapping table, and periodically synchronize asynchronous messages to each region in batches; each of the multiple regional nodes is used to process the local message of the corresponding region and synchronize the subscription information to the core node through an asynchronous mechanism; the cluster connection module is used to manage the connection and communication of the MQTT clusters in multiple regions; the cross-region load balancing module is connected in communication with the cluster connection module, and is used to dynamically allocate loads to multiple regions based on the connection and communication of the MQTT clusters in multiple regions; the disaster recovery module is used to monitor the communication status of the MQTT clusters in multiple regions, and switch to a backup communication path when an abnormal communication status is detected. The present invention configures an MQTT cluster in each region, and realizes cross-region information transmission through the MQTT cluster connection technology, which can not only reduce traffic consumption, but also improve the stability and reliability of information transmission, thereby improving the efficiency of cross-region information transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings constituting a part of this application are used to provide a further understanding of this application, so that other features, purposes and advantages of this application become more obvious. The schematic embodiment drawings and their descriptions of this application are used to explain this application and do not constitute an improper limitation on this application. In the drawings:

[0040] Figure 1 It is a structural diagram of a cross-regional information transmission system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in sequences other than those illustrated or described herein.

[0043] It should be understood that in various embodiments of the present invention, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0044] It should be understood that in the present invention, "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0045] It should be understood that in the present invention, "plurality" refers to two or more than two. "And / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "Contains A, B and C", "Contains A, B, C" means that A, B, and C are all included, "Contains A, B or C" means that one of A, B, and C is included, and "Contains A, B and / or C" means that any one, any two, or any three of A, B, and C are included.

[0046] It should be understood that in the present invention, "B corresponding to A", "B corresponding to A", "A corresponds to B" or "B corresponds to A" means that B is associated with A and B can be determined based on A. Determining B based on A does not mean determining B based only on A, but B can also be determined based on A and / or other information. A and B match when the similarity between A and B is greater than or equal to a preset threshold.

[0047] Depending on the context, "if" as used herein may be interpreted as "when" or "when" or "in response to determining" or "in response to detecting."

[0048] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0049] In order to make the purpose, technical solutions and advantages of the present invention more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0050] In one embodiment, a cross-region information transmission system is provided, comprising:

[0051] A core node, multiple regional nodes, a cluster connection module, a cross-region load balancing module, and a disaster recovery module, wherein the multiple regional nodes are configured in multiple regions, and each of the multiple regional nodes corresponds to one region among the multiple regions;

[0052] The core node is connected to multiple regional nodes for real-time synchronization of global topics to each of the multiple regions through the configured global topic mapping table, and periodically batch synchronization of asynchronous messages to each region;

[0053] Each of the multiple regional nodes is used to process local messages of the corresponding region and synchronize subscription information to the core node through an asynchronous mechanism;

[0054] The cluster connection module is connected to the core node for managing the connection and communication of MQTT clusters in multiple regions, and feeding back the connection and communication status of MQTT clusters in multiple regions to the core node;

[0055] The cross-region load balancing module is communicated with the cluster connection module to dynamically distribute loads to multiple regions based on the connection and communication of MQTT clusters in multiple regions;

[0056] The disaster recovery module is used to monitor the communication status of MQTT clusters in multiple regions and switch to the backup communication path when an abnormal communication status is detected.

[0057] like Figure 1 As shown, it is assumed that the regional nodes include regional node 1 and regional node 2, each regional node is configured with a corresponding MQTT cluster, regional node 1 is configured with MQTT cluster 1, and regional node 2 is configured with MQTT cluster 2. The core node is connected in communication with regional node 1 and regional node 2, and is used to synchronize global topics to regional node 1 and regional node 2 in real time through the configured global topic mapping table, and periodically synchronize asynchronous messages to regional node 1 and regional node 2 in batches; each regional node in regional node 1 and regional node 2 is used to process local messages of the corresponding region and synchronize subscription information to the core node through an asynchronous mechanism; the cluster connection module is connected in communication with the core node, and is used to manage the connection and communication of MQTT cluster 1 and MQTT cluster 2, and feed back the connection and communication status of MQTT cluster 1 and MQTT cluster 2 to the core node; the cross-region load balancing module is connected in communication with the cluster connection module, and is used to dynamically distribute loads to multiple regions based on the connection and communication of MQTT cluster 1 and MQTT cluster 2; the disaster recovery module is used to monitor the communication status of MQTT cluster 1 and MQTT cluster 2, and switch to the backup communication path when an abnormal communication status is detected.

[0058] In one embodiment, the core node serves as the control center of the system and optimizes cross-region communication efficiency by managing the global topic namespace.

[0059] Specifically, the global topic mapping table: The core node maintains the global topic mapping table and synchronizes the topic information of each region in real time. For example, when a vehicle subscribes to the topic / vehicle / location in the European region, the core node updates the topic to the global mapping table and synchronizes it to other regions to ensure that vehicles around the world can access the topic. The introduction of the global topic namespace eliminates data redundancy in cross-regional communication and improves communication efficiency in global deployment.

[0060] Traffic compression technology: The system uses a differential compression algorithm to transmit only differential data to reduce bandwidth consumption. For example, when multiple regions share the same vehicle status update, the system only transmits the differential part to optimize communication performance. By transmitting only differential data, the traffic of cross-regional communication is reduced, and the bandwidth utilization of the system is optimized.

[0061] The present invention introduces a global topic namespace and cluster connection mechanism to achieve unified topic management in all regions. Messages only need to be published once and can be transmitted to multiple regions simultaneously, reducing cross-region communication traffic.

[0062] In one embodiment, the regional node is responsible for processing local messages and synchronizing subscription information to the core node through an asynchronous mechanism.

[0063] Specifically, the subscription information is synchronized to the core node through an asynchronous mechanism, including: for each regional node in multiple regional nodes, a corresponding asynchronous message queue is configured for each regional node, wherein the asynchronous message queue caches the subscription information to be synchronized arranged in sequence; within a set time window or information quantity threshold, when each regional node triggers batch synchronization, multiple subscription information to be synchronized arranged in sequence in the asynchronous message queue are packaged and synchronized to the core node.

[0064] In the event of a network outage or transmission failure, the subscription information will be marked as pending retransmission, and the subscription message to be retransmitted will be added to the asynchronous message queue, waiting for the next transmission.

[0065] The present invention maintains an asynchronous message queue in each regional node to cache messages that need to be synchronized. The replication node synchronizes the messages to the core node in batches to reduce the network load. In addition, the regional node dynamically adjusts the synchronization frequency of the message according to the network bandwidth and delay to ensure the consistency of the message in different regions, while giving priority to emergency messages.

[0066] In one embodiment, the cluster connection module is responsible for managing cluster connections in different regions and optimizing cross-region data transmission.

[0067] The cluster connection module includes a path calculation submodule, which is used to calculate the optimal message transmission path across regions based on network conditions, loads of regional nodes and the Dijkstra algorithm.

[0068] The cluster connection module also includes a data transmission submodule, which is used to obtain historical status messages and new status messages when a new status message is released; use a differential algorithm to calculate the difference between the historical status message and the new status message to obtain differential data; and use a compression algorithm to compress the differential data before transmission.

[0069] The present invention reduces the delay and bandwidth consumption of cross-regional communications and improves the system transmission efficiency through optimal path calculation and differential algorithm.

[0070] In one embodiment, the cross-region load balancing module dynamically distributes the global system load to ensure maximum resource utilization.

[0071] The cross-region load balancing module includes a message distribution submodule, which is used to perform hash calculation on the message subject to obtain the hash value corresponding to the message subject; sort multiple regional nodes according to the hash value to form a hash ring corresponding to multiple regional nodes; match the corresponding regional node on the hash ring according to the hash value corresponding to the message subject, and distribute the message content corresponding to the message subject to the matching regional node on the hash ring. By performing hash calculation on the message subject, the load is evenly distributed to nodes in different regions. Changes in the hash value will trigger node reallocation to ensure node load balancing.

[0072] The cross-region load balancing module also includes a node expansion submodule, which is used to monitor the load of multiple regional nodes in real time; if the load of any regional node among multiple regional nodes exceeds the preset threshold, a new regional node is automatically expanded and added to the hash ring. The health status of each cluster is regularly monitored and automatically switched to a healthy node to ensure stable system operation.

[0073] The cross-region load balancing module also includes a load migration submodule, which is used to migrate the load exceeding the preset threshold to a regional node adjacent to any regional node when the load of any regional node among the multiple regional nodes exceeds a preset threshold.

[0074] In one embodiment, the disaster recovery module monitors the health of the network and regional nodes to ensure that it can quickly switch to a backup path or node when the network is interrupted or the regional node fails.

[0075] The disaster recovery module includes a switching submodule, which is used to regularly monitor the health status of multiple regional nodes through a fault detection algorithm; when any of the multiple regional nodes fails, the communication is switched to a healthy backup node or path based on the network topology and node health status.

[0076] Specifically, the system regularly checks the status of nodes in each region, monitoring indicators such as CPU usage, memory usage, and network bandwidth. If a node exceeds the set threshold or loses response, the system marks it as "failed". Multiple tests are performed to verify whether nodes in each region are faulty. If the fault persists, an alarm is immediately issued and the regional node is marked as "unavailable".

[0077] When a node or network failure is detected in a certain area, it will automatically switch to an alternative path or node based on the network topology and node health status to continue processing business. Messages with higher priority will be processed first to ensure that important business is not interrupted.

[0078] The health status of nodes in each region is regularly monitored through fault detection algorithms. When a regional node fails, the system automatically switches to a backup node or path to ensure business continuity.

[0079] The disaster recovery module also includes a fault recovery module, which is used to complete the missing data through the data synchronization mechanism when any regional node in multiple regional nodes recovers from a fault. The system resynchronizes the missing data to ensure data consistency.

[0080] The high availability of the present invention on a global scale enhances the disaster recovery capability in the event of network interruption or node failure.

[0081] The embodiment of the present invention provides a cross-region information transmission system, including: a core node, multiple regional nodes, a cluster connection module, a cross-region load balancing module and a disaster recovery module, wherein the multiple regional nodes are configured in multiple regions, and each of the multiple regional nodes corresponds to one of the multiple regions; the core node is connected in communication with the multiple regional nodes, and is used to synchronize the global topic to each of the multiple regions in real time through the configured global topic mapping table, and periodically synchronize asynchronous messages to each region in batches; each of the multiple regional nodes is used to process the local message of the corresponding region and synchronize the subscription information to the core node through an asynchronous mechanism; the cluster connection module is used to manage the connection and communication of the MQTT clusters in multiple regions; the cross-region load balancing module is connected in communication with the cluster connection module, and is used to dynamically allocate loads to multiple regions based on the connection and communication of the MQTT clusters in multiple regions; the disaster recovery module is used to monitor the communication status of the MQTT clusters in multiple regions, and switch to a backup communication path when an abnormal communication status is detected. The present invention configures an MQTT cluster in each region, and realizes cross-region information transmission through the MQTT cluster connection technology, which can not only reduce traffic consumption, but also improve the stability and reliability of information transmission, thereby improving the efficiency of cross-region information transmission.

[0082] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0083] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A cross-regional information transmission system, characterized in that: include: A core node, a plurality of regional nodes, a cluster connection module, a cross-region load balancing module and a disaster recovery module, wherein the plurality of regional nodes are configured in a plurality of regions, and each of the plurality of regional nodes corresponds to one of the plurality of regions; The core node is in communication connection with the multiple regional nodes, and is used to synchronize the global topic to each of the multiple regions in real time through the configured global topic mapping table, and periodically synchronize asynchronous messages in batches to each of the regions; Each of the plurality of regional nodes is used to process local messages of a corresponding region and synchronize subscription information to the core node through an asynchronous mechanism; The cluster connection module is used to manage the connection and communication of the MQTT clusters in the multiple regions, and feed back the connection and communication status of the MQTT clusters in the multiple regions to the core node; The cross-region load balancing module is communicatively connected to the cluster connection module, and is used to dynamically distribute loads to the multiple regions based on the connection and communication of the MQTT clusters of the multiple regions; The disaster recovery module is used to monitor the communication status of the MQTT clusters in the multiple areas and switch to a backup communication path when an abnormality in the communication status is detected.

2. A cross-regional information transmission system according to claim 1, characterized in that: The step of synchronizing the subscription information to the core node through an asynchronous mechanism includes: For each of the multiple regional nodes, a corresponding asynchronous message queue is configured for each of the regional nodes, wherein the asynchronous message queue caches the subscription information to be synchronized arranged in sequence; Within a set time window or information quantity threshold, when each regional node triggers batch synchronization, multiple subscription information to be synchronized arranged in sequence in the asynchronous message queue are packaged and synchronized to the core node.

3. A cross-regional information transmission system according to claim 2, characterized in that: The system further comprises: a retransmission module; The retransmission module is used to mark the subscription information as to be retransmitted in case of network interruption or transmission failure, and re-add the subscription message to be retransmitted to the asynchronous message queue to wait for the next transmission.

4. The inter-regional information transmission system according to claim 1, characterized in that: The cluster connection module includes a path calculation submodule; The path calculation submodule is used to calculate the optimal message transmission path across regions according to the network status, the load of the regional nodes and the Dijkstra algorithm.

5. The inter-regional information transmission system according to claim 1, characterized in that: The cluster connection module includes a data transmission submodule; The data transmission submodule is used to obtain historical status messages and new status messages when a new status message is released; Calculating the difference between the historical status message and the new status message using a differential algorithm to obtain difference data; The difference data is compressed using a compression algorithm and then transmitted.

6. The inter-regional information transmission system according to claim 1, characterized in that: The cross-region load balancing module includes a message distribution submodule; The message distribution submodule is used to perform hash calculation on the message subject to obtain the hash value corresponding to the message subject; Sort the multiple regional nodes according to the hash values ​​to form a hash ring corresponding to the multiple regional nodes; The corresponding regional node is matched on the hash ring according to the hash value corresponding to the message subject, and the message content corresponding to the message subject is distributed to the corresponding regional node on the hash ring.

7. A cross-region information transmission system according to claim 6, characterized in that: The cross-region load balancing module includes a node expansion submodule; The node expansion submodule is used to monitor the load conditions of the multiple regional nodes in real time; If the load of any regional node among the multiple regional nodes exceeds a preset threshold, a new regional node is automatically expanded and added to the hash ring.

8. The inter-regional information transmission system according to claim 6, characterized in that: The cross-region load balancing module includes a load migration submodule; The load migration submodule is used for migrating the load exceeding the preset threshold to a regional node adjacent to the regional node when the load of any regional node among the multiple regional nodes exceeds the preset threshold.

9. The inter-regional information transmission system according to claim 1, characterized in that: The disaster recovery module includes a switching submodule; The switching submodule is used to regularly monitor the health status of the multiple regional nodes through a fault detection algorithm; When any of the multiple regional nodes fails, the communication is switched to a healthy standby node or path according to the network topology diagram and the health status of the nodes.

10. The inter-regional information transmission system according to claim 1, characterized in that: The disaster recovery module includes a fault repair module; The fault recovery module is used to complete the missing data through a data synchronization mechanism when any regional node among the multiple regional nodes recovers from a fault.

Images