A method for implementing MQTT Broker server based on Netty framework using Raft algorithm

Abstract

The present invention provides a method for implementing an MQTT Broker server based on a Netty framework by applying a Raft algorithm. The method for implementing an MQTT Broker server based on a Netty framework by applying a Raft algorithm comprises the following steps: S1. Netty: The Netty is an asynchronous event-driven network application framework that supports rapid development of maintainable, high-performance, protocol-oriented servers and clients; S2. Reactor model: The Netty architecture is designed and implemented according to the Reactor model, and the Reactor concurrency model of Netty: Netty implements and extends the Reactor model. The method for implementing an MQTT Broker server based on a Netty framework by applying a Raft algorithm provided by the present invention can avoid the negative impact of commercial authorization, continuously iterate and upgrade, provide data support for Broker operation and decision-making, ensure mandatory consistency of data within a cluster, and even if a node fails, the node can still provide data normally after recovery, while also solving the shortcomings of the prior art.

Description

Technical Field

[0001] The present invention belongs to the technical field of Internet of Things, and in particular relates to a method for realizing an MQTT Broker server based on a Netty framework and applying a Raft algorithm. Background Art

[0002] Message queue telemetry transmission is a message protocol based on the publish / subscribe paradigm under the ISO standard (ISO / IEC PRF 20922), which can be regarded as a "bridge for data transmission". It works on the TCP / IP protocol family and is a publish / subscribe message protocol designed for remote devices with low hardware performance and poor network conditions. For this reason, it requires a message middleware to solve the current heavy data transmission protocol.

[0003] The MQTT protocol defines two network entities: message broker and client. The message broker is used to receive messages from the client and forward them to the target client. The MQTT client can be any device that runs the MQTT library and is connected to the message broker through the network, such as a microcontroller or a large server.

[0004] The transmission of information is managed by topics. When a publisher has data to distribute, it sends a control message carrying the data to the connected message broker. The broker distributes the data to the clients that subscribe to this topic. The publisher does not need to know the data and specific location of the subscriber; similarly, the subscriber does not need to configure the publisher's relevant information. If the message broker receives a message on a topic and there is no subscription to this topic, the broker will discard it unless the publisher marks it as a retained message.

[0005] The existing MQTT broker:

[0006] 1). Mosquitto

[0007] Mosquitto is an open source message agent software launched by the Eclipse open source community that implements the message push protocol MQTT v3.1. It provides a lightweight, publish / subscribe message push mode to facilitate short message communication between devices. It is developed in C language and is suitable for lightweight MQTT access application scenarios.

[0008] 2).EMQX

[0009] EMQX is a distributed IoT MQTT server launched by Hangzhou Yingyun Technology Co., Ltd. It provides open source and commercial versions, is developed using the Erlang language, and supports the connection of massive IoT devices.

[0010] However, the prior art has the following disadvantages:

[0011] 1). Mosquitto

[0012] (1) Security authentication is not supported;

[0013] (2) Cluster mode is not supported;

[0014] (3) Data persistence is not supported;

[0015] (4) No business management interface is open;

[0016] 2).EMQX

[0017] (1) The open source version has limited functions and requires a front-end load to implement pseudo-cluster mode;

[0018] (2) The commercial version is fully functional, charged by traffic or number of connected devices, and can only be privately deployed after authorization;

[0019] (3) Only the commercial version supports data persistence. If the open source version crashes, data will be lost.

[0020] (4) Developed using Erlang, there are many extension methods and the threshold for extension implementation is high.

[0021] Therefore, it is necessary to provide a new method for implementing the MQTT Broker server based on the Netty framework and applying the Raft algorithm to solve the above technical problems. Summary of the invention

[0022] The technical problem solved by the present invention is to provide a method for realizing an MQTT Broker server based on a Netty framework and applying a Raft algorithm, which can avoid the negative impact of commercial authorization, continuously iterate and upgrade, provide data support for Broker operation and decision-making, ensure mandatory consistency of data within a cluster, and even if a node fails, still provide data normally after the node is restored. At the same time, the method can also solve the shortcomings of the prior art.

[0023] To solve the above technical problems, the method for implementing the MQTT Broker server based on the Netty framework and applying the Raft algorithm provided by the present invention comprises the following steps: S1. Netty

[0024] Netty is an asynchronous event-driven network application framework that supports rapid development of maintainable, high-performance protocol-oriented servers and clients;

[0025] S2. Reactor Model

[0026] The Netty architecture is designed and implemented according to the Reactor mode. The Reactor concurrency model of Netty:

[0027] Netty implements and extends the Reactor model as follows:

[0028] 1). The Acceptor main thread pool is an independent NIO thread pool, which plays the role of Acceptor in the network link in the form of NIO, also known as AcceptorReactor;

[0029] 2). After the program starts, a thread will be randomly selected from the Acceptor main thread pool as the Acceptor thread to bind the port and receive client connections;

[0030] 3). The Acceptor thread receives the connection request and hands it over to other threads of the AcceptorReactor to handle the client's login, handshake, and security authentication;

[0031] 4). After the business layer link is formally established, the request will be transferred to a thread in the IO sub-thread pool, which will handle the IO reading, writing and encoding operations;

[0032] 5). Finally, according to the different request types, the request is transferred to a specific type of WorkThreadsPools for specific business processing. In the Netty concurrency model, concurrent access of clients, reading and writing of underlying Sockets, and processing of business requests are all multi-threaded asynchronous operations. Therefore, Netty gives the application a core capability to respond to network events generated at any time point in any order, and to continuously reuse the computer's thread resources in certain resources to adapt to higher throughput and scalability requirements;

[0033] S3.Raft

[0034] The Raft cluster consists of several nodes, each of which has three states: Leader, Follower, and Candidate. Raft will first elect a Leader, and only one node will be the Leader, while the other nodes are Followers. The Leader node processes all client requests and copies the Replicated Log to the Follower nodes. If the Leader fails, the Leader heartbeat detection is still abnormal after the election countdown expires, and the Follower will re-elect a new Leader.

[0035] S4. Implement MQTT Broker based on Netty and Raft:

[0036] The MQTT Broker is a server that directly accepts concurrent client network connections. It uses Netty to process the network communication part, which can maximize the use of server resources to handle more client connections and more services. Netty comes with an MQTT encoding and decoding module, which can be used directly to complete the encoding and decoding of MQTT protocol communication.

[0037] S5.Broker service maintains data consistency through Raft cluster:

[0038] The MQTT Broker service implements the Raft algorithm to give the MQTT Brokers data replication features, forming a cluster based on strong consistency data synchronization, thereby reducing the risk of single point failure in the system;

[0039] S6. The Raft algorithm gives the system the following characteristics:

[0040] 1) Persistent session: When the session identifier "Clean Session" is set to 0, it means that a persistent session is created. When the client is disconnected, the session still maintains and saves offline messages until the session times out. After the persistent session information and offline MQTT information are saved locally, they are copied to other Brokers in the cluster through the Raft algorithm. After the client comes back online, it can obtain all offline information in time, reducing waiting time.

[0041] 2) Subscription data: After the client successfully subscribes, the subscription relationship data is copied to other Brokers. If broadcasting or instructions are required, or the client changes Broker access, all the client's current subscription topic data can be obtained in time. When canceling the subscription, the data is deleted locally and then deleted in the cluster.

[0042] 3) Message retransmission with guaranteed quality of service (QoS): The MQTT protocol stipulates that the messages sent by the server and the client to the other party should meet the quality of service requirements. That is, after the MQTT message is sent, if the other party does not receive a response within the specified time, the client will resend the message. At the same time, if the client reconnects while maintaining the session, the Broker will automatically resend the response message to the reconnected client to ensure the correctness of the QoS process. For the above scenario, the message sent by the client but not confirmed is saved locally, and then copied and synchronized to other Brokers. Whether it is a retry message or a publisher reconnects, the message message can still be found in other Brokers, and the corresponding message message is maintained according to the semantic requirements of QoS.

[0043] As a further solution of the present invention, the MQTT Broker service includes the following:

[0044] (1) The Broker service consists of a network service layer, an MQTT protocol service layer, a business service layer, a storage module, and a bridge module, which separates network communication from business processing. The storage module consists of Raft nodes and local storage, and the storage module defines several storage devices.

[0045] (2) The network service layer processes the client connection and forwards the client reported data to the protocol layer for decoding. At the same time, the service data encoded by the protocol layer is sent to the corresponding client;

[0046] (3) The protocol layer uses the MQTT protocol encoding and decoding module implemented by Netty to process the MQTT protocol messages required for communication between the client and the Broker, define the processing flow of the corresponding messages, and implement the triggering processing of the protocol services, such as login, authentication, and message distribution;

[0047] (4) Business services consist of a message delivery module and a data distribution module;

[0048] (5) The Raft service implements network communication, provides inter-node communication functions for the Raft cluster, and supports the operation of the Raft cluster: election, log replication, security check, and heartbeat detection;

[0049] (6) Raft nodes are composed of storage devices that implement the Raft protocol and Raft communication services. The process data and result data in the business link can submit data changes to the Raft Leader as needed, and will be stored in the local specific storage engine as Raft Logs. The log replication and strong consistency check of the Raft algorithm are used to complete data replication between nodes in the cluster.

[0050] (7) Local storage is used to store data including but not limited to the following: client session records, unconfirmed messages (QoS1, QoS2), indicator measurement values, and configuration data required for program operation;

[0051] (8) The stateless bridge service exposes the REST API to the outside world and provides business function management operations including but not limited to the following in a low-intrusive manner: obtaining Broker status information, publishing messages, and viewing client lists.

[0052] As a further solution of the present invention, the message sending is responsible for capturing user operations, abstracting them into MQTT message bodies, and then transferring them to the protocol layer for processing. The data distribution is responsible for capturing the message bodies decoded by the protocol layer. The data distribution is responsible for processing including but not limited to: message persistence, communication between devices, message replay, shared subscription, etc., and completes the distribution of reported data by docking with persistence middleware and specific service interfaces.

[0053] As a further solution of the present invention, the data replication process is specifically as follows:

[0054] (1) The leader is responsible for receiving requests from clients. The leader stores the requests as Raft log entries in local storage. After the storage is successful, it initiates a replication action to other servers in parallel to replicate the log entries.

[0055] (2) When the log is replicated from the leader to most followers, the leader publishes the log entry to its state machine and returns the execution result to the client, i.e., globally commits the replication result;

[0056] (3) Block is the smallest unit of data management and movement in the Raft cluster. It corresponds to an actual data interval in the local storage. Each Block contains multiple pieces of data. Block has multiple copies, each of which is located in a different Node. Multiple Blocks on different nodes constitute an abstract Raft cluster and are copies of each other.

[0057] (4) Based on the log consistency check of the Raft algorithm, some Followers may not successfully complete log replication. The Leader will then retry replication indefinitely until all Followers finally store all log entries.

[0058] As a further solution of the present invention, the MQTT runtime is a single instance, exposing the TCP endpoint to the outside. According to the characteristics of Raft, at least three instances need to be run to form a Raft cluster, as follows:

[0059] (1) The Netty service can only obtain access from clients that implement the MQTT protocol and only processes requests that meet the MQTT protocol specification. When the Broker service is started, it exposes a fixed {ip:port} and the communication endpoint {ip:port} of the Raft node;

[0060] (2) The client uses the username and password to request access to the Broker service according to the predefined configuration. After approval, the two ends establish a long communication link. During the "CONNECT" message monitoring process, the network service will store the link in the client session list and client link list with the ClientID as the key, and update the routing table;

[0061] (3) When the Broker starts, the Raft node starts with the node list in the configuration and creates a cluster. At the beginning, all nodes are followers, so a new Leader election is initiated;

[0062] (4) The Netty service starts processing MQTT communication messages with the client. Message types include but are not limited to: subscribing to topics, sending messages, and heartbeat detection. The client's subscription to topics and sending messages are subject to the authorization restrictions of the administrator and need to be edited and maintained in the management background.

[0063] (5) If the client allows subscription to topics, it can initiate a "SUBSCRIBE" message after establishing a long link. Before returning the "SUBACK" message, the Broker service decodes and stores the relationship between the client and multiple topics in the local storage as {ClientID, Subscription}, and at the same time, uses the Raft algorithm to replicate it to other nodes;

[0064] (6) If the client allows sending messages, after establishing a long link, it can initiate a "PUBLISH" message. After receiving the message, the Broker processes the information according to the message type in the message processor. If it is general information, it is directly forwarded; if it is retained information, it is stored in the local storage of the retained information and waits to be resent to the client. If it is a QoS 1 or QoS 2 message, multiple message transmission processes are initiated in sequence according to the MQTT specification requirements. During this period, unconfirmed messages are stored in the corresponding local storage. After receiving the "PUBCOMP" message, the unconfirmed messages are deleted;

[0065] (7) No matter the message is published by the client or through the bridge service, it must be decoded by the protocol layer message processor of the Broker service. After being decoded by the processor, the message is transferred to the message distribution module. The destination of the message is determined according to the data characteristics of the message, such as MessageType, MessageHeader, QoS, etc. (as shown in Figure 4). If the subject is known, it can be subscribed through the subject. After the client range is reversely determined, the message can be forwarded. According to the data characteristics of the message, such as MessageType, MessageHeader, QoS, etc., different encoders are used to encode the message, and different message sending processes are enabled. If delayed sending is required, the message is stored in the delay storage; if QoS 2 semantics is required, the message is stored in the pending confirmation message storage, and data processing is carried out in the subsequent QoS2 process node. If the message needs to be persisted, it is stored in the corresponding storage engine according to the current persistence strategy through the bridge service;

[0066] (8) Managers can directly send requests through the web platform. The web platform connects to the bridge service. External applications complete the communication with the Broker service by adapting the bridge service, such as adjusting settings, obtaining Broker status information, publishing messages, proxy subscription and unsubscription, disconnecting specified clients, and viewing client lists.

[0067] As a further solution of the present invention, the Raft algorithm decomposes the consistency problem into three sub-problems, namely election, log replication and security, as follows:

[0068] (1) Election: When the leader fails or the cluster is newly created, followers will elect a new leader;

[0069] (2) Log replication: The leader processes all client requests in a unified manner. After receiving data from a client, the leader will copy the data to other nodes in the cluster in the form of log entries and force the logs of other nodes to be consistent with its own.

[0070] (3) Security: Each node has a clear log stored in the state machine. The log index position of the node cannot be inconsistent with the log index of the Leader. The Leader will force the node's index position to be consistent with its own.

[0071] As a further solution of the present invention, the Raft algorithm adds two restrictions to enhance cluster availability and ensure strong data consistency: election restriction and log consistency check, as follows:

[0072] (1) Election restrictions: After initiating an election, a candidate node will visit most nodes and compare its own log with the log of each node. By comparing the index value and term number of the last log in the two logs, it determines which node's log is more recent. If the term numbers of the last entries in the two logs are different, the log with the larger term number is more recent. If the term numbers of the last entries in the two logs are the same, the log with the longer term number is more recent. Only the node with the largest log index value and the latest term number can win the election.

[0073] (2) Log consistency check: Under normal circumstances, the logs of the leader and followers will be consistent. In the production operation environment, various abnormal situations may occur in the leader and followers, resulting in inconsistent logs of each node. The Raft algorithm stipulates that the leader must save the nextIndex of each follower. The newly elected leader will initialize the nextIndex to the log index value of the current leader + 1. If the log of a follower is inconsistent with the leader, the consistency check will fail when the leader synchronizes the log in the next heartbeat. When the nextIndex does not match the currentIndex of the target follower, the leader reduces the value of nextIndex and retries. The nextIndex and the currentIndex of the follower will eventually be consistent at a certain position. Due to election restrictions, the leader saves the latest and most complete log data of the cluster. After determining the nextIndex, the leader will synchronize the data after nextIndex to the followers.

[0074] Compared with the related art, the method for implementing the MQTT Broker server based on the Netty framework and applying the Raft algorithm provided by the present invention has the following beneficial effects:

[0075] 1. The present invention is designed and implemented by using Java industrial-grade computer language, which is highly versatile and reduces the threshold for secondary development;

[0076] 2. The present invention can avoid the negative impact of commercial authorization and continuously iterate and upgrade;

[0077] 3. The present invention realizes network access by using Netty, and a single machine with 4CoreCPU and 16GRAM can support 500,000 links;

[0078] 4. The present invention can access the role of MQTT Broker. In addition to the classic client form, it can also be in the form of application service, and it is no longer mandatory to access as a Client;

[0079] 5. The system of the present invention is associated with external services through bridging services, and can provide data support for Broker operation and decision-making by connecting to existing configuration systems or custom configuration systems;

[0080] 6. The present invention implements a lightweight Broker cluster by using the Raft algorithm, which can ensure the mandatory consistency of data in the cluster. Even if a node fails, the node can still provide data normally after recovery;

[0081] 7. The present invention can theoretically meet the characteristics required for distributed cluster operation by continuously upgrading and expanding the bridge service, protocol layer, and Raft node. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0083] Figure 1 It is a schematic diagram of the Netty concurrency model in the present invention;

[0084] Figure 2 A schematic diagram of Raft node state transition in the present invention;

[0085] Figure 3 It is a schematic diagram of the overall architecture of the MQTT Broker service in the present invention;

[0086] Figure 4 It is a schematic diagram of the message body structure in the present invention;

[0087] Figure 5 It is a schematic diagram of the Raft cluster structure in the present invention. DETAILED DESCRIPTION

[0088] Please refer to Figure 1 , Figure 2 , Figure 3 , Figure 4 and Figure 5 ,in, Figure 1 It is a schematic diagram of the Netty concurrency model in the present invention; Figure 2 A schematic diagram of Raft node state transition in the present invention; Figure 3 It is a schematic diagram of the overall architecture of the MQTT Broker service in the present invention; Figure 4 It is a schematic diagram of the message body structure in the present invention; Figure 5 The method for implementing the MQTT Broker server based on the Netty framework and applying the Raft algorithm comprises the following steps: S1.

[0089] Netty is an asynchronous event-driven network application framework that supports rapid development of maintainable, high-performance protocol-oriented servers and clients;

[0090] S2. Reactor Model

[0091] The Netty architecture is designed and implemented according to the Reactor mode. The Reactor concurrency model of Netty:

[0092] like Figure 1As shown in the figure, Netty implements and extends the Reactor model as follows:

[0093] 1). The Acceptor main thread pool is an independent NIO thread pool, which plays the role of Acceptor in the network link in the form of NIO, also known as AcceptorReactor;

[0094] 2). After the program starts, a thread will be randomly selected from the Acceptor main thread pool as the Acceptor thread to bind the port and receive client connections;

[0095] 3). The Acceptor thread receives the connection request and hands it over to other threads of the AcceptorReactor to handle the client's login, handshake, and security authentication;

[0096] 4). After the business layer link is formally established, the request will be transferred to a thread in the IO sub-thread pool, which will handle the IO reading, writing and encoding operations;

[0097] 5). Finally, according to the different request types, the request is transferred to a specific type of WorkThreadsPools for specific business processing. In the Netty concurrency model, concurrent access of clients, reading and writing of underlying Sockets, and processing of business requests are all multi-threaded asynchronous operations. Therefore, Netty gives the application a core capability to respond to network events generated at any time point in any order, and to continuously reuse the computer's thread resources in certain resources to adapt to higher throughput and scalability requirements;

[0098] S3.Raft

[0099] The Raft cluster consists of several nodes, each of which has three states: Leader, Follower, and Candidate (e.g. Figure 2 As shown in the figure, Raft will first elect a Leader, and only one node will be the Leader, and the other nodes are Followers. The Leader node processes all client requests and copies the Replicated Log to the Follower nodes. If the Leader fails, the Leader heartbeat detection is still abnormal after the election countdown expires, and the Follower will re-elect a new Leader;

[0100] S4. Implement MQTT Broker based on Netty and Raft:

[0101] like Figure 3As shown, the MQTT Broker is a server that directly accepts concurrent client network connections. Using Netty to process the network communication part can maximize the use of server resources to handle more client connections and more services. Netty comes with an MQTT encoding and decoding module, which can be used directly to complete the encoding and decoding of MQTT protocol communication.

[0102] S5.Broker service maintains data consistency through Raft cluster:

[0103] like Figure 5 As shown in the figure, the MQTT Broker service implements the Raft algorithm to give the MQTT Brokers data replication features, and forms a cluster based on strong consistency data synchronization, thereby reducing the risk of single point failure in the system;

[0104] S6. The Raft algorithm gives the system the following characteristics:

[0105] 1) Persistent session: When the session identifier "Clean Session" is set to 0, it means that a persistent session is created. When the client is disconnected, the session still maintains and saves offline messages until the session times out. After the persistent session information and offline MQTT information are saved locally, they are copied to other Brokers in the cluster through the Raft algorithm. After the client comes back online, it can obtain all offline information in time, reducing waiting time.

[0106] 2) Subscription data: After the client successfully subscribes, the subscription relationship data is copied to other Brokers. If broadcasting or instructions are required, or the client changes Broker access, all the client's current subscription topic data can be obtained in time. When canceling the subscription, the data is deleted locally and then deleted in the cluster.

[0107] 3) Message retransmission with guaranteed quality of service (QoS): The MQTT protocol stipulates that the messages sent by the server and the client to the other party should meet the quality of service requirements. That is, after the MQTT message is sent, if the other party does not receive a response within the specified time, the client will resend the message. At the same time, if the client reconnects while maintaining the session, the Broker will automatically resend the response message to the reconnected client to ensure the correctness of the QoS process. For the above scenario, the message sent by the client but not confirmed is saved locally, and then copied and synchronized to other Brokers. Whether it is a retry message or a publisher reconnects, the message message can still be found in other Brokers, and the corresponding message message is maintained according to the semantic requirements of QoS.

[0108] The MQTT Broker services include:

[0109] (1) The Broker service consists of a network service layer, an MQTT protocol service layer, a business service layer, a storage module, and a bridge module, which separates network communication from business processing. The storage module consists of Raft nodes and local storage, and the storage module defines several storage devices.

[0110] (2) The network service layer processes the client connection and forwards the client reported data to the protocol layer for decoding. At the same time, the service data encoded by the protocol layer is sent to the corresponding client;

[0111] (3) The protocol layer uses the MQTT protocol encoding and decoding module implemented by Netty to process the MQTT protocol messages required for communication between the client and the Broker, define the processing flow of the corresponding messages, and implement the triggering processing of the protocol services, such as login, authentication, and message distribution;

[0112] (4) Business services consist of a message delivery module and a data distribution module;

[0113] (5) The Raft service implements network communication, provides inter-node communication functions for the Raft cluster, and supports the operation of the Raft cluster: election, log replication, security check, and heartbeat detection;

[0114] (6) Raft nodes are composed of storage devices that implement the Raft protocol and Raft communication services. The process data and result data in the business link can submit data changes to the Raft Leader as needed, and will be stored in the local specific storage engine as Raft Logs. The log replication and strong consistency check of the Raft algorithm are used to complete data replication between nodes in the cluster.

[0115] (7) Local storage is used to store data including but not limited to the following: client session records, unconfirmed messages (QoS1, QoS2), indicator measurement values, and configuration data required for program operation;

[0116] (8) The stateless bridge service exposes the REST API to the outside world and provides business function management operations including but not limited to the following in a low-intrusive manner: obtaining Broker status information, publishing messages, and viewing client lists.

[0117] The message sending is responsible for capturing user operations, abstracting them into MQTT message bodies, and then transferring them to the protocol layer for processing. The data distribution is responsible for capturing the message bodies that have been decoded by the protocol layer. The data distribution is responsible for processing including but not limited to: message persistence, communication between devices, message replay, shared subscription, etc., and completes the distribution of reported data by docking with persistence middleware and specific service interfaces.

[0118] The data replication process is as follows:

[0119] (1) The leader is responsible for receiving requests from clients. The leader stores the requests as Raft log entries in local storage. After the storage is successful, it initiates a replication action to other servers in parallel to replicate the log entries.

[0120] (2) When the log is replicated from the leader to most followers, the leader publishes the log entry to its state machine and returns the execution result to the client, i.e., globally commits the replication result;

[0121] (3) Block is the smallest unit of data management and movement in the Raft cluster. It corresponds to an actual data interval in the local storage. Each Block contains multiple pieces of data. Block has multiple copies, each of which is located in a different Node. Multiple Blocks on different nodes constitute an abstract Raft cluster and are copies of each other.

[0122] (4) Based on the log consistency check of the Raft algorithm, some Followers may not successfully complete log replication. The Leader will then retry replication indefinitely until all Followers finally store all log entries.

[0123] like Figure 3 As shown, the MQTT runtime is a single instance, exposing the TCP endpoint. According to the Raft characteristics, at least three instances need to be run to form a Raft cluster, as follows:

[0124] (1) The Netty service can only obtain access from clients that implement the MQTT protocol and only processes requests that meet the MQTT protocol specification. When the Broker service is started, it exposes a fixed {ip:port} and the communication endpoint {ip:port} of the Raft node;

[0125] (2) The client uses the username and password to request access to the Broker service according to the predefined configuration. After approval, the two ends establish a long communication link. During the "CONNECT" message monitoring process, the network service will store the link in the client session list and client link list with the ClientID as the key, and update the routing table;

[0126] (3) When the Broker starts, the Raft node starts with the node list in the configuration and creates a cluster. At the beginning, all nodes are followers, so a new Leader election is initiated;

[0127] (4) The Netty service starts processing MQTT communication messages with the client. Message types include but are not limited to: subscribing to topics, sending messages, and heartbeat detection. The client's subscription to topics and sending messages are subject to the authorization restrictions of the administrator and need to be edited and maintained in the management background.

[0128] (5) If the client allows subscription to topics, it can initiate a "SUBSCRIBE" message after establishing a long link. Before returning the "SUBACK" message, the Broker service decodes and stores the relationship between the client and multiple topics in the local storage as {ClientID, Subscription}, and at the same time, uses the Raft algorithm to replicate it to other nodes;

[0129] (6) If the client allows sending messages, after establishing a long link, it can initiate a "PUBLISH" message. After receiving the message, the Broker processes the information according to the message type in the message processor. If it is general information, it is directly forwarded; if it is retained information, it is stored in the local storage of the retained information and waits to be resent to the client. If it is a QoS 1 or QoS 2 message, multiple message transmission processes are initiated in sequence according to the MQTT specification requirements. During this period, unconfirmed messages are stored in the corresponding local storage. After receiving the "PUBCOMP" message, the unconfirmed messages are deleted;

[0130] (7) No matter the message is published by the client or through the bridge service, it must be decoded by the protocol layer message processor of the Broker service. After being decoded by the processor, the message is transferred to the message distribution module. The destination of the message is determined according to the data characteristics of the message, such as MessageType, MessageHeader, QoS, etc. (as shown in Figure 4). If the subject is known, it can be subscribed through the subject. After the client range is reversely determined, the message can be forwarded. According to the data characteristics of the message, such as MessageType, MessageHeader, QoS, etc., different encoders are used to encode the message, and different message sending processes are enabled. If delayed sending is required, the message is stored in the delay storage; if QoS 2 semantics is required, the message is stored in the pending confirmation message storage, and data processing is carried out in the subsequent QoS2 process node. If the message needs to be persisted, it is stored in the corresponding storage engine according to the current persistence strategy through the bridge service;

[0131] (8) Managers can directly send requests through the web platform. The web platform connects to the bridge service. External applications complete the communication with the Broker service by adapting the bridge service, such as adjusting settings, obtaining Broker status information, publishing messages, proxy subscription and unsubscription, disconnecting specified clients, and viewing client lists.

[0132] The Raft algorithm decomposes the consistency problem into three sub-problems: election, log replication, and security. The details are as follows:

[0133] (1) Election: When the leader fails or the cluster is newly created, followers will elect a new leader;

[0134] (2) Log replication: The leader processes all client requests in a unified manner. After receiving data from a client, the leader will copy the data to other nodes in the cluster in the form of log entries and force the logs of other nodes to be consistent with its own.

[0135] (3) Security: Each node has a clear log stored in the state machine. The log index position of the node cannot be inconsistent with the log index of the Leader. The Leader will force the node's index position to be consistent with its own.

[0136] To enhance cluster availability and ensure strong data consistency, the Raft algorithm adds two restrictions: election restriction and log consistency check. The details are as follows:

[0137] (1) Election restrictions: After initiating an election, a candidate node will visit most nodes and compare its own log with the log of each node. By comparing the index value and term number of the last log in the two logs, it determines which node's log is more recent. If the term numbers of the last entries in the two logs are different, the log with the larger term number is more recent. If the term numbers of the last entries in the two logs are the same, the log with the longer term number is more recent. Only the node with the largest log index value and the latest term number can win the election.

[0138] (2) Log consistency check: Under normal circumstances, the logs of the leader and followers will be consistent. In the production operation environment, various abnormal situations may occur in the leader and followers, resulting in inconsistent logs of each node. The Raft algorithm stipulates that the leader must save the nextIndex of each follower. The newly elected leader will initialize the nextIndex to the log index value of the current leader + 1. If the log of a follower is inconsistent with the leader, the consistency check will fail when the leader synchronizes the log in the next heartbeat. When the nextIndex does not match the currentIndex of the target follower, the leader reduces the value of nextIndex and retries. The nextIndex and the currentIndex of the follower will eventually be consistent at a certain position. Due to election restrictions, the leader saves the latest and most complete log data of the cluster. After determining the nextIndex, the leader will synchronize the data after nextIndex to the followers.

[0139] Compared with the prior art, the advantages and effects of the present invention are as follows:

[0140] (1) Designed and implemented using Java industrial-grade computer language, it is highly versatile and has a lower threshold for secondary development;

[0141] (2) It can avoid the negative impact of commercial authorization and continuously iterate and upgrade;

[0142] (3). Using Netty to achieve network access, a single machine with 4CoreCPU and 16GRAM can support 500,000 connections;

[0143] (4) In addition to the classic client form, the role of accessing MQTT Broker can also be in the form of application service, and it is no longer mandatory to access as a client;

[0144] (5) The system is connected to external services through bridge services. By connecting to existing configuration systems or custom configuration systems, it can provide data support for Broker operation and decision-making;

[0145] (6) Using the Raft algorithm to implement a lightweight Broker cluster can ensure mandatory consistency of data within the cluster. Even if a node fails, the node can still provide data normally after recovery;

[0146] (7) By continuously upgrading and expanding the bridge service, protocol layer, and Raft node, it is theoretically possible to meet the characteristics required for distributed cluster operation.

[0147] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for implementing an MQTT Broker server based on a Netty framework using the Raft algorithm, characterized in that: The following steps are involved: S1.Netty Netty is an asynchronous event-driven network application framework that supports rapid development of maintainable, high-performance protocol-oriented servers and clients; S2. Reactor Model The Netty architecture is designed and implemented according to the Reactor mode. The Reactor concurrency model of Netty: Netty implements and extends the Reactor model as follows: 1). The Acceptor main thread pool is an independent NIO thread pool, which plays the role of Acceptor in the network link in the form of NIO, also known as AcceptorReactor; 2). After the program starts, a thread will be randomly selected from the Acceptor main thread pool as the Acceptor thread to bind the port and receive client connections; 3). The Acceptor thread receives the connection request and hands it over to other threads of the AcceptorReactor to handle the client's login, handshake, and security authentication; 4). After the business layer link is formally established, the request will be transferred to a thread in the IO sub-thread pool, which will handle the IO reading, writing and encoding operations; 5). Finally, according to the different request types, the request is transferred to a specific type of WorkThreadsPools for specific business processing. In the Netty concurrency model, concurrent access of clients, reading and writing of underlying Sockets, and processing of business requests are all multi-threaded asynchronous operations. Therefore, Netty gives the application a core capability to respond to network events generated at any time point in any order, and to continuously reuse the computer's thread resources in certain resources to adapt to higher throughput and scalability requirements; S3.Raft The Raft cluster consists of several nodes, each of which has three states: Leader, Follower, and Candidate. Raft will first elect a Leader, and only one node will be the Leader, while the other nodes are Followers. The Leader node processes all client requests and copies the Replicated Log to the Follower nodes. If the Leader fails, the Leader heartbeat detection is still abnormal after the election countdown expires, and the Follower will re-elect a new Leader. S4. Implement MQTT Broker based on Netty and Raft: The MQTT Broker is a server that directly accepts concurrent client network connections. It uses Netty to process the network communication part, which can maximize the use of server resources to handle more client connections and more services. Netty comes with an MQTT encoding and decoding module, which can be used directly to complete the encoding and decoding of MQTT protocol communication. S5.Broker service maintains data consistency through Raft cluster: The MQTT Broker service implements the Raft algorithm to give the MQTT Brokers data replication features, forming a cluster based on strong consistency data synchronization, thereby reducing the risk of single point failure in the system; S6. The Raft algorithm gives the system the following characteristics: 1). Persistent session: When the session identifier "Clean Session" is set to 0, it means that a persistent session is created. When the client is disconnected, the session still maintains and saves offline messages until the session times out. After the persistent session information and offline MQTT information are saved locally, they are copied to other Brokers in the cluster through the Raft algorithm. After the client comes back online, it can obtain all offline information in time, reducing waiting time; 2) Subscription data: After the client successfully subscribes, the subscription relationship data is copied to other Brokers. If broadcasting or instructions are required, or the client changes Broker access, all the client's current subscription topic data can be obtained in time. When canceling the subscription, the data is deleted locally and then deleted in the cluster. 3) Message retransmission with guaranteed quality of service (QoS): The MQTT protocol stipulates that the messages sent by the server and the client to the other party should meet the quality of service requirements. That is, after the MQTT message is sent, if the other party does not receive a response within the specified time, the client will resend the message. At the same time, if the client reconnects while maintaining the session, the Broker will automatically resend the response message to the reconnected client to ensure the correctness of the QoS process. For the above scenario, the message sent by the client but not confirmed is saved locally, and then copied and synchronized to other Brokers. Whether it is a retry message or a publisher reconnects, the message message can still be found in other Brokers, and the corresponding message message is maintained according to the semantic requirements of QoS.

2. The method for implementing an MQTT Broker server based on the Netty framework and applying the Raft algorithm according to claim 1 is characterized in that: The MQTT Broker services include: (1) The Broker service consists of a network service layer, an MQTT protocol service layer, a business service layer, a storage module, and a bridge module, which separates network communication from business processing. The storage module consists of Raft nodes and local storage, and the storage module defines several storage devices. (2) The network service layer processes the client connection and forwards the client reported data to the protocol layer for decoding. At the same time, the service data encoded by the protocol layer is sent to the corresponding client; (3) The protocol layer applies the MQTT protocol encoding and decoding module implemented by Netty to process the MQTT protocol messages required for communication between the client and the Broker, define the processing flow of the corresponding messages, and implement the triggering processing of the protocol business, including login, authentication and message distribution; (4) Business services consist of a message delivery module and a data distribution module; (5) The Raft service implements network communication, provides inter-node communication functions for the Raft cluster, and supports the operation of the Raft cluster: election, log replication, security check, and heartbeat detection; (6) Raft nodes are composed of storage devices that implement the Raft protocol and Raft communication services. The process data and result data in the business link can submit data changes to the Raft Leader as needed, and will be stored in the local specific storage engine as Raft Logs. The log replication and strong consistency check of the Raft algorithm are used to complete data replication between nodes in the cluster. (7) Local storage is used to store data including but not limited to the following: client session records, unconfirmed message QoS1 and QoS2, indicator measurement values, and configuration data required for program operation; (8) The stateless bridge service exposes the REST API to the outside world and provides business function management operations including but not limited to the following in a low-intrusive manner: obtaining Broker status information, publishing messages, and viewing client lists.

3. The method for implementing an MQTT Broker server based on the Netty framework and applying the Raft algorithm according to claim 2 is characterized in that: The message sending is responsible for capturing user operations, abstracting them into MQTT message bodies, and then transferring them to the protocol layer for processing. The data distribution is responsible for capturing the message bodies that have been decoded by the protocol layer. The data distribution is responsible for processing including but not limited to: message persistence, communication between devices, message replay, shared subscription, and completing the distribution of reported data by docking with persistence middleware and specific service interfaces.

4. The method for implementing an MQTT Broker server based on the Netty framework and applying the Raft algorithm according to claim 1 is characterized in that: The data replication process is as follows: (1) The leader is responsible for receiving requests from clients. The leader stores the requests as Raft log entries in local storage. After the storage is successful, it initiates a replication action to other servers in parallel to replicate the log entries. (2) When the log is copied from the leader to most of the Fol lowers, the leader publishes this log entry to its state machine and returns the execution result to the client, i.e., globally commits the replication result; (3) Block is the smallest unit of data management and movement in the Raft cluster. It corresponds to an actual data interval in the local storage. Each Block contains multiple pieces of data. Block has multiple copies, each of which is located in a different Node. Multiple Blocks on different nodes constitute an abstract Raft cluster and are copies of each other. (4) Based on the log consistency check of the Raft algorithm, some Fol lowers may not successfully complete log replication. The Leader will retry replication indefinitely until all Fol lowers finally store all log entries.

5. The method for implementing an MQTT Broker server based on the Netty framework and applying the Raft algorithm according to claim 2 is characterized in that: The MQTT runtime is a single instance, exposing the TCP endpoint. According to the characteristics of Raft, at least three instances need to be run to form a Raft cluster, as follows: (1) The Netty service can only obtain access from clients that implement the MQTT protocol and only processes requests that meet the MQTT protocol specification. When the Broker service is started, it exposes a fixed {ip:port} and the communication endpoint {ip:port} of the Raft node; (2) The client uses the username and password to request access to the Broker service according to the predefined configuration. After approval, the two ends establish a long communication link. During the "CONNECT" message monitoring process, the network service will store the link in the client session list and client link list with the ClientID as the key, and update the routing table; (3) When the Broker starts, the Raft node starts with the node list in the configuration and creates the cluster. When all nodes are in Fol lower at the beginning, a new Leader election is initiated; (4) The Netty service starts processing MQTT communication messages with the client. Message types include but are not limited to: subscribing to topics, sending messages, and heartbeat detection. The client's subscription to topics and sending messages are subject to the authorization restrictions of the administrator and need to be edited and maintained in the management background. (5) If the client allows subscription to topics, it can initiate a "SUBSCRIBE" message after establishing a long link. Before returning the "SUBACK" message, the Broker service decodes and stores the relationship between the client and multiple topics in the local storage as {ClientID, Subscription}, and at the same time, uses the Raft algorithm to replicate it to other nodes; (6) If the client allows sending messages, after establishing a long link, it can initiate a "PUBLISH" message. After receiving the message, the Broker processes the information according to the message type in the message processor. If it is general information, it is directly forwarded; if it is retained information, it is stored in the local storage of the retained information and waits to be resent to the client. If it is a QoS1 or QoS2 message, multiple message transmission processes are initiated in sequence according to the MQTT specification requirements. During this period, unconfirmed messages are stored in the corresponding local storage. After receiving the "PUBCOMP" message, the unconfirmed messages are deleted; (7) No matter it is a message published by the client or through the bridge service, it must be decoded by the protocol layer message processor of the Broker service. After being decoded by the processor, the message is transferred to the message distribution module. The destination of the message is determined based on the message MessageType, MessageHeader, and QoS data characteristics. If the topic is known, it can be subscribed through the topic. After the client range is reversely determined, the message can be forwarded. According to the message MessageType, MessageHeader, and QoS data characteristics, different encoders are used to encode the message, and different message sending processes are enabled. If delayed sending is required, the message is stored in the delay memory; If QoS2 semantics need to be achieved, the message is stored in the pending message storage and data processing is carried out in the subsequent QoS2 process nodes. If the message needs to be persisted, it is stored in the corresponding storage engine through the bridge service according to the current persistence strategy. (8) Managers can directly send requests through the web platform. The web platform connects to the bridge service. External applications complete the communication with the Broker service by adapting the bridge service, including adjusting settings, obtaining Broker status information, publishing messages, proxy subscription and unsubscription, disconnecting specified clients, and viewing client lists.

6. The method for implementing an MQTT Broker server based on the Netty framework and applying the Raft algorithm according to claim 4 is characterized in that: The Raft algorithm decomposes the consistency problem into three sub-problems: election, log replication, and security. The details are as follows: (1) Election: When the leader fails or the cluster is newly created, followers will elect a new leader; (2) Log replication: The leader processes all client requests in a unified manner. After receiving data from a client, the leader will copy the data to other nodes in the cluster in the form of log entries and force the logs of other nodes to be consistent with its own. (3) Security: Each node has a clear log stored in the state machine. The log index position of the node cannot be inconsistent with the log index of the Leader. The Leader will force the node's index position to be consistent with its own.

7. The method for implementing an MQTT Broker server based on the Netty framework and applying the Raft algorithm according to claim 4 is characterized in that: To enhance cluster availability and ensure strong data consistency, the Raft algorithm adds two restrictions: election restriction and log consistency check. The details are as follows: (1) Election restrictions: After initiating an election, a candidate node will visit most nodes and compare its own log with the log of each node. By comparing the index value and term number of the last log in the two logs, it determines which node's log is more recent. If the term numbers of the last entries in the two logs are different, the log with the larger term number is more recent. If the term numbers of the last entries in the two logs are the same, the log with the longer term number is more recent. Only the node with the largest log index value and the latest term number can win the election. (2) Log consistency check: Under normal circumstances, the logs of the leader and followers will be consistent. In the production operation environment, various abnormal situations may occur in the leader and followers, resulting in inconsistent logs of each node. The Raft algorithm stipulates that the leader must save the nextIndex of each follower. The newly elected leader will initialize the nextIndex to the log index value of the current leader + 1. If the log of a follower is inconsistent with the leader, the consistency check will fail when the leader synchronizes the log in the next heartbeat. When the nextIndex does not match the currentIndex of the target follower, the leader reduces the value of nextIndex and retries. The nextIndex and the currentIndex of the follower will eventually be consistent at a certain position. Due to election restrictions, the leader saves the latest and most complete log data of the cluster. After determining the nextIndex, the leader will synchronize the data after nextIndex to the followers.

Images