MQTT server failover method, framework, equipment and medium

Abstract

The invention discloses an MQTT server failover method, architecture, equipment and medium, a master node and a slave node communicate through a virtual routing redundancy (VRRP) protocol, and the master node periodically sends a VRRP heartbeat message to the slave node; when a host-level fault occurs and the slave node does not receive a VRRP heartbeat message of the master node within a preset timeout interval, the slave node takes over a virtual IP address and is upgraded to a master role; when a service-level fault occurs, the detection program running on the main node detects that the specified service is abnormal, and the main node actively reduces the priority of the main node and cancels the virtual IP address bound with the main node; and when detecting that the priority of the master node is lower than that of the slave node, the slave node immediately initiates election and becomes a new master node, and binds the virtual IP address to complete fault switching. By detecting the host-level fault and the service-level fault respectively, compared with single-point deployment, when the server fails, services can be continuously provided to the outside, and the continuity of services is ensured.

Description

Technical Field

[0001] This invention relates to the field of network communication technology, and specifically to an MQTT server failover method, architecture, device, and medium. Background Technology

[0002] MQTT (Message Queuing Telemetry Transport) is a lightweight, publish / subscribe-based messaging protocol widely used in the Internet of Things (IoT), mobile applications, and low-bandwidth network environments. The MQTT server (also known as the Broker) is responsible for receiving, routing, and distributing messages, and its availability directly impacts the stability and reliability of the entire communication system.

[0003] In practical applications, the availability of the MQTT Broker directly affects the stability and reliability of the entire IoT system. A Broker failure will result in message transmission failure, impacting continuity and potentially causing serious economic losses or security risks. Therefore, ensuring the high availability of MQTT services is a critical requirement in the construction of IoT systems.

[0004] Traditional high-availability solutions typically employ multi-node clusters, such as using multiple MQTT Brokers to form a cluster, and distributing traffic through load balancers or DNS round-robin. These traditional solutions usually rely on expensive hardware load balancing equipment or complex software load balancing clusters (such as NLB or HAProxy), and require complex built-in cluster configurations of the MQTT Broker. The entire solution depends on the complex built-in cluster configuration of the MQTT Broker and the coordinated operation of the load balancing infrastructure, resulting in extremely high deployment and maintenance costs, and the overall cost and risk remain significant. It is worth noting that single-node deployments present a significant single point of failure risk; if the server crashes or the network is interrupted, the entire MQTT service will be unavailable, failing to meet the high availability requirements of production environments, and resulting in high deployment and maintenance costs. Summary of the Invention

[0005] The technical problem this invention aims to solve is that traditional methods can only detect and optimize faults based on single-point deployment. The goal is to provide an MQTT server failover method, architecture, device, and medium. By detecting both host-level and service-level faults separately, compared to a single-point deployed MQTT Broker, it can continue to provide services in the event of server hardware failure, network interruption, operating system crash, or abnormal service process. For host-level faults, it switches between primary and backup nodes; for service-level faults, it reorders and selects a primary node, preventing the entire IoT connectivity platform from paralyzing, leading to device disconnection, message loss, and business interruption. This ensures the reliability, continuity, and maintainability of the business.

[0006] This invention is achieved through the following technical solution:

[0007] The first aspect of this invention provides an MQTT server failover method, comprising the following specific steps:

[0008] The master node and slave node communicate through the Virtual Router Redundancy Protocol (VRRP), with the master node periodically sending VRRP heartbeat messages to the slave node.

[0009] When a host-level failure occurs, if the slave node does not receive a VRRP heartbeat message from the master node within a preset timeout interval, the slave node takes over the virtual IP address and becomes the master.

[0010] When a service-level failure occurs, if the detection program running on the master node detects that the specified service is abnormal, the master node will proactively lower its own priority and revoke the virtual IP address it is bound to.

[0011] When a slave node detects that the priority of the master node is lower than its own, it immediately initiates an election and becomes the new master node, binds the virtual IP address, and completes the failover.

[0012] Furthermore, the service-level fault detection process includes:

[0013] Establish a TCP connection with the target IoT communication protocol MQTT broker service port;

[0014] After the TCP connection is successfully established, an MQTT CONNECT message is sent to the MQTT broker, and it is verified whether a valid CONNACK message is received within a preset time.

[0015] After successful verification of the CONNACK message, a SUBSCRIBE message is sent to the MQTT broker to subscribe to the specified topic, and the corresponding subscription success response is verified.

[0016] After the subscription success response is verified, a PUBLISH message is sent to the MQTT broker to publish a test message to the specified topic and verify whether the publishing operation was successful.

[0017] If any of the above verification steps fails, the process will be terminated immediately and the corresponding error code will be returned to indicate that the MQTT broker service is in an abnormal state.

[0018] Furthermore, the service-level fault detection process also includes:

[0019] When sending a DISCONNECT message to the MQTT broker, the TCP connection is closed;

[0020] When verifying whether a valid CONNACK message has been received, it is determined whether the return code in the received CONNACK message is a value indicating success.

[0021] The second aspect of this invention provides a deployment architecture applied to an MQTT server failover method, comprising: a virtual IP, a master node, a backup node, an MQTT broker instance deployed on the master node and the backup node, and a KeepAlived instance;

[0022] The KeepAlived instance of the master node is initially set to MASTER and bound to a virtual IP address;

[0023] The initial state of the KeepAlived instance of the slave node is BACKUP;

[0024] The master node and slave node communicate via the VRRP protocol;

[0025] The KeepAlived instance of the master node is configured with a first service detection module, which is used to periodically check the service status of the local MQTT broker, and when a service anomaly is detected, it actively lowers the local priority and cancels the binding of the virtual IP address.

[0026] Furthermore, the configuration of the KeepAlived instance includes:

[0027] Create KeepAlived configuration files on both the master and slave nodes;

[0028] The configuration file must include at least the following parameters:

[0029] Virtual IP address;

[0030] The initial states and priorities of the master and slave nodes, wherein the priority of the master node is configured to be greater than that of the slave node;

[0031] VRRP heartbeat detection interval;

[0032] VRRP authentication information;

[0033] Configure KeepAlived to work in non-preemptive mode;

[0034] Define a monitoring script block, wherein the script block specifies a vrrp_script block for checking;

[0035] Specifies the notification script to be triggered when the node state changes to MASTER or BACKUP.

[0036] Furthermore, the KeepAlived instance of the slave node is configured to take over the virtual IP address and transition to MASTER state when any of the following conditions are met:

[0037] No VRRP announcement message was received from the master node within the preset timeout interval;

[0038] The VRRP protocol reveals that the master node has a lower priority than the slave node.

[0039] Furthermore, the vrrp_script block is configured with the path of the detection program, the execution interval, the execution timeout, and the weight value for priority adjustment when the check fails.

[0040] Furthermore, the KeepAlived instances of the master node and slave node are each configured with a notification module. The notification module is configured to trigger the execution of the corresponding NOTIFY_MASTER script or NOTIFY_BACKUP script and issue an alarm when the state of the local KeepAlived instance changes to MASTER or BACKUP.

[0041] A third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement an MQTT server failover method.

[0042] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an MQTT server failover method.

[0043] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0044] By detecting both host-level and service-level failures, compared to a single-point deployed MQTT Broker, it can continue to provide services in the event of server hardware failure, network interruption, operating system crash, or abnormal service process. Host-level failures are resolved by switching between primary and backup nodes, while service-level failures are resolved by reordering and selecting a primary node. This prevents the entire IoT connectivity platform from becoming paralyzed, which could lead to device disconnection, message loss, or business interruption, thus ensuring the reliability, continuity, and maintainability of the business.

[0045] By configuring MQTT broker instances and KeepAlived instances on both primary and backup nodes, and combining Keepalived with a custom MQTT monitoring program, MQTT clients connect to the virtual IP to access the MQTT service. With the virtual IP migration mechanism, clients will automatically reconnect to the new primary server if the TCP connection is interrupted, subscribe to topics, and thus restore the service. This automates fault detection, VIP transfer, and service takeover, improving the reliability of the MQTT service. Attached Figure Description

[0046] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0047] Figure 1 This is an overall architecture diagram of the MQTT server failover method in this embodiment of the invention;

[0048] Figure 2 This is the service-level fault detection process in this embodiment of the invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0050] Example 1

[0051] As one possible implementation method, such as Figure 1 As shown, this embodiment provides an MQTT server failover method, including the following specific steps:

[0052] The master node and slave node communicate through the Virtual Router Redundancy Protocol (VRRP), with the master node periodically sending VRRP heartbeat messages to the slave node.

[0053] When a host-level failure occurs, if the slave node does not receive a VRRP heartbeat message from the master node within a preset timeout interval, the slave node takes over the virtual IP address and becomes the master. During normal operation, the KeepAlived host will periodically send heartbeat packets to the slave nodes via the VRRP protocol. When a host-level failure occurs, the slave node cannot receive a VRRP heartbeat message from the master node. After the timeout (master_down_interval), the slave node immediately takes over the VIP virtual IP address and becomes the master node.

[0054] When a service-level failure occurs, if the detection program running on the master node detects an anomaly in the specified service, the master node will proactively lower its own priority and revoke its bound virtual IP address. KeepAlived uses the CHECK_MQTT program to detect whether the MQTT service is abnormal. If an anomaly occurs, the master will proactively lower its own priority and revoke the VIP. When a slave node detects that the master node's priority is lower than its own, it will immediately initiate an election and become the new master node, binding a virtual IP address and completing the failover.

[0055] This embodiment detects both host-level and service-level faults separately. Compared to a single-point deployed MQTTBroker, it can continue to provide services in the event of server hardware failure, network interruption, operating system crash, or abnormal service process. Host-level faults are handled by switching between primary and backup nodes, while service-level faults are handled by reordering and selecting a primary node. This prevents the entire IoT connectivity platform from becoming paralyzed, which could lead to device disconnection, message loss, and business interruption. It ensures the reliability, continuity, and maintainability of the business.

[0056] like Figure 2 As shown, the service-level fault detection process includes:

[0057] Establish a TCP connection with the target IoT communication protocol MQTT broker service port;

[0058] After the TCP connection is successfully established, an MQTT CONNECT message is sent to the MQTT broker, and it is verified whether a valid CONNACK message is received within a preset time. The TCP connection is closed when a DISCONNECT message is sent to the MQTT broker.

[0059] After the CONNACK message is successfully verified, a SUBSCRIBE message is sent to the MQTT broker to subscribe to the specified topic, and the corresponding subscription success response is verified. When verifying whether a valid CONNACK message has been received, it is also necessary to determine whether the return code in the received CONNACK message is a value indicating success.

[0060] After the subscription is successfully verified, a PUBLISH message is sent to the MQTT broker to publish a test message to the specified topic and verify whether the publishing operation was successful.

[0061] If any of the above verification steps fails, the process will be terminated immediately and the corresponding error code will be returned to indicate that the MQTT broker service is in an abnormal state.

[0062] CHECK_MQTT is not simply a check of whether the MQTT service is running and whether the service port (1883) is accessible. Rather, it is a health check of the protocol. Specifically, the MQTT_CHECK program simulates an MQTT client, sends CONNECT, SUBSCRIBE, and PULISH packets, and verifies the correctness of the returned packets, thereby more accurately determining the true service status of the Broker node.

[0063] Example 2

[0064] As one possible implementation method, such as Figure 1 As shown, this embodiment provides a deployment architecture applied to an MQTT server failover method. By configuring MQTT broker instances and KeepAlived instances on both the primary and backup nodes, and combining KeepAlived with a custom MQTT monitoring program, MQTT clients connect to the virtual IP to access the MQTT service. With the virtual IP migration mechanism, the client will automatically reconnect to the new primary server if the TCP connection is interrupted, subscribe to topics, and thus restore the service. This automates fault detection, VIP transfer, and service takeover, improving the reliability of the MQTT service.

[0065] The specific architecture includes:

[0066] Host: Runs the main MQTT Broker, Keepalived main process, and CHECK_MQTT monitoring program;

[0067] Standby machine: running the standby MQTT Broker, the Keepalived standby process, and the CHECK_MQTT monitoring program;

[0068] Virtual IP (VIP): Clients access the MQTT service through this IP, which actually points to the current primary server;

[0069] MQTT broker instances and KeepAlived instances are deployed on the master and slave nodes;

[0070] The KeepAlived instance on the master node is initially set to MASTER and bound to a virtual IP address;

[0071] The KeepAlived instance on the node is initially in the BACKUP state;

[0072] The master node and slave node communicate via the VRRP protocol;

[0073] The KeepAlived instance on the master node is configured with a first service detection module, which is used to periodically check the service status of the local MQTT broker and, when a service anomaly is detected, proactively lower the local priority and revoke the binding of the virtual IP address.

[0074] The configuration of KeepAlived instances deployed on master and slave nodes includes: creating KeepAlived configuration files on the master and slave nodes; configuring at least the following parameters in the configuration files: virtual IP address; initial state and priority of master and slave nodes, where the priority of the master node is configured to be greater than that of the slave node; VRRP heartbeat detection interval; VRRP authentication information; configuring KeepAlived's working mode to non-preemptive mode; defining a monitoring script block, which specifies a vrrp_script block for checking, where the vrrp_script block is configured with the path of the detection program, execution interval, execution timeout, and priority adjustment weight value when the check fails; specifying a notification script to be triggered when the node state changes to MASTER or BACKUP;

[0075] The KeepAlived instance of the slave node is configured to take over the virtual IP address and transition to the MASTER state when either of the following conditions is met: no VRRP advertisement message is received from the master node within a preset timeout interval; or the master node is found to have a lower priority than the slave node via the VRRP protocol.

[0076] The KeepAlived instances on both the master and slave nodes are also configured with notification modules. These modules are configured to trigger the execution of the corresponding NOTIFY_MASTER or NOTIFY_BACKUP scripts and issue an alert when the status of the local KeepAlived instance changes to MASTER or BACKUP.

[0077] Example 3

[0078] As one possible implementation, this embodiment provides a deployment architecture example for an MQTT server failover method, the specific steps of which include:

[0079] Install EMQX (MQTT Broker), Keepalived service, and mailx client on the primary and backup nodes. The mailx client is used to send emails.

[0080] Create a keepalived.conf file on both the primary and backup nodes to configure the virtual IP, heartbeat detection time, primary / backup node identifiers and failover weight (primary node weight greater than backup node weight), authentication, MQTT monitoring execution program MQTT_CHECK, and the execution of notify_master and notify_backup scripts. Notify_master and notify_backup call mailx to send emails to operations personnel. The keepalived working mode is configured to non-preemptive mode.

[0081] Deploy the `check_mqtt` monitoring test on both the primary and standby nodes. This test is used by keepalived to monitor the MQTT service status. The `check_mqtt` monitoring test is compiled into an executable file in C. The program's input parameter is the MQTT instance IP address, and the default port is 1883. The specific checks performed are as follows: Figure 2 The inspection process returns a non-zero value when an exception is found in the MQTT service, and a zero value when no exception is found in the MQTT service.

[0082] Configure the vrrp_script block in the Keepalived configuration file to define the script for monitoring the MQTT service:

[0083] vrrp_script chk_mqtt {

[0084] script " / etc / keepalived / check_mqtt" # Call monitoring test

[0085] interval 2 # Check every 2 seconds

[0086] weight -50 # Priority reduced by 50 when check fails

[0087] timeout 3 # Script execution timeout

[0088] fall 2 # Fault is only diagnosed after two consecutive failed checks.

[0089] rise 1 # Service recovery is determined upon successful check 1

[0090] }

[0091] vrrp_instance VI_1 { ...

[0093] state BACKUP # Set both machines to BACKUP

[0094] virtual_router_id 51

[0095] priority 90 # The standby node has a lower priority.

[0096] nopreempt # Non-preemptive mode

[0097] track_script {

[0098] chk_mqtt # Use the above script to check

[0099] } ...

[0101] }

[0102] Log in to physical machines A and B, save the keepalived.conf file, and restart the keepalived instance service.

[0103] Normally, the client accesses the service by forwarding the keepalived virtual IP to the backend MQTT real IP, and the configuration is complete;

[0104] When a host fails, fault detection and failover will be automatically implemented according to the fault detection and failover process described in the architecture.

[0105] Example 4

[0106] As one possible implementation, this embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements an MQTT server failover method.

[0107] Example 5

[0108] As one possible implementation, this embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements an MQTT server failover method.

[0109] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for failover of an MQTT server, characterized in that, The specific steps include the following: The master node and slave node communicate through the Virtual Router Redundancy Protocol (VRRP), with the master node periodically sending VRRP heartbeat messages to the slave node. When a host-level failure occurs, if the slave node does not receive a VRRP heartbeat message from the master node within a preset timeout interval, the slave node takes over the virtual IP address and becomes the master. When a service-level failure occurs, if the detection program running on the master node detects that the specified service is abnormal, the master node will proactively lower its own priority and revoke the virtual IP address it is bound to. When a slave node detects that the priority of the master node is lower than its own, it immediately initiates an election and becomes the new master node, binds to the virtual IP address, and completes the failover.

2. The MQTT server failover method according to claim 1, characterized in that, The service-level fault detection process includes: Establish a TCP connection with the target IoT communication protocol MQTT broker service port; After the TCP connection is successfully established, an MQTT CONNECT message is sent to the MQTT broker, and it is verified whether a valid CONNACK message is received within a preset time. After successful verification of the CONNACK message, a SUBSCRIBE message is sent to the MQTT broker to subscribe to the specified topic, and the corresponding subscription success response is verified. After the subscription success response is verified, a PUBLISH message is sent to the MQTT broker to publish a test message to the specified topic and verify whether the publishing operation was successful. If any of the above verification steps fails, the process will be terminated immediately and the corresponding error code will be returned to indicate that the MQTT broker service is in an abnormal state.

3. The MQTT server failover method according to claim 2, characterized in that, The service-level fault detection process also includes: When sending a DISCONNECT message to the MQTT broker, the TCP connection is closed; When verifying whether a valid CONNACK message has been received, it is determined whether the return code in the received CONNACK message is a value indicating success.

4. A deployment architecture, characterized in that, The method for failover of an MQTT server as described in any one of claims 1-3 includes: a virtual IP, a master node, a backup node, an MQTT broker instance deployed on the master node and the backup node, and a KeepAlived instance; The KeepAlived instance of the master node is initially set to MASTER and bound to a virtual IP address; The initial state of the KeepAlived instance of the slave node is BACKUP; The master node and slave node communicate via the VRRP protocol; The KeepAlived instance of the master node is configured with a first service detection module, which is used to periodically check the service status of the local MQTT broker, and when a service anomaly is detected, it actively lowers the local priority and cancels the binding of the virtual IP address.

5. The MQTT server failover deployment architecture according to claim 4, characterized in that, The configuration of the KeepAlived instance includes: Create KeepAlived configuration files on both the master and slave nodes; The configuration file must include at least the following parameters: Virtual IP address; The initial states and priorities of the master and slave nodes, wherein the priority of the master node is configured to be greater than that of the slave node; VRRP heartbeat detection interval; VRRP authentication information; Configure KeepAlived to work in non-preemptive mode; Define a monitoring script block, wherein the script block specifies a vrrp_script block for checking; Specifies the notification script to be triggered when the node state changes to MASTER or BACKUP.

6. The MQTT server failover deployment architecture according to claim 5, characterized in that, The KeepAlived instance of the slave node is configured to take over the virtual IP address and transition to MASTER state when any of the following conditions are met: No VRRP announcement message was received from the master node within the preset timeout interval; The VRRP protocol reveals that the master node has a lower priority than the slave node.

7. The MQTT server failover deployment architecture according to claim 5, characterized in that, The vrrp_script block is configured with the path of the detection program, the execution interval, the execution timeout, and the weight value for priority adjustment when the check fails.

8. The MQTT server failover deployment architecture according to claim 4, characterized in that, The KeepAlived instances of the master and slave nodes are also configured with notification modules. The notification modules are configured to trigger the execution of the corresponding NOTIFY_MASTER script or NOTIFY_BACKUP script and issue an alarm when the state of the local KeepAlived instance changes to MASTER or BACKUP.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the MQTT server failover method as described in any one of claims 1 to 3.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the MQTT server failover method as described in any one of claims 1 to 3.

Images