Mqtt automatic subscription system and subscription method based on redis

Abstract

The invention discloses a redis-based mqtt automatic subscription system and subscription method. The system comprises an MQTT Broker, a Redis database and an automatic subscription module, the MQTT client communicates with the Redis database through the MQTT Broker, and the MQTT Broker is integrated with the Redis database and is responsible for processing connection, disconnection, message publishing and subscription requests of the MQTT client; the Redis database is connected with the automatic subscription module, is used for storing configuration information, real-time data, subscription theme information and a subscription state of the system, and adopts a distributed architecture; and the automatic subscription module is responsible for automatically managing subscription and updating of the MQTT client theme according to predefined rules and data in the Redis database. By combining the advantages of the MQTT protocol and the Redis database, the subscription management problem in large-scale Internet of Things application is solved, and it is ensured that the latest theme is subscribed.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things, in particular to a Redis-based MQTT automatic subscription system and subscription method. Background Art

[0002] The rapid development of Internet of Things (IoT) technology has enabled various devices to exchange data and be remotely controlled through the Internet. In order to achieve efficient communication between these devices, the IoT system requires a protocol that can stably transmit data in a low-bandwidth, high-latency network environment. The MQTT (Message Queuing Telemetry Transport) protocol is a lightweight publish-subscribe message transmission protocol. Due to its low bandwidth usage, high reliability and flexible message distribution mechanism, it has been widely used in the field of Internet of Things (IoT).

[0003] The traditional MQTT subscription mechanism requires the client to explicitly specify the topic to be subscribed. For each topic subscription, manual intervention and manual configuration are usually required. This method has the following limitations:

[0004] Manual configuration is cumbersome: In a large-scale IoT environment, there are a large number of devices, and manually configuring the subscription topics of each device is inefficient and prone to errors.

[0005] Difficult to adjust dynamically: Faced with dynamically changing network environments and application requirements, manual adjustment of subscription topics has poor flexibility and is difficult to respond to new subscription requirements in a timely manner.

[0006] Complex management: In complex IoT applications, managing the subscription information of a large number of devices and ensuring that they subscribe to the latest topics is a challenging task. Summary of the invention

[0007] In order to solve the above technical problems, the purpose of the present invention is to provide a Redis-based MQTT automatic subscription system and subscription method, which can realize automatic subscription and update of MQTT topics. This Redis-based MQTT automatic subscription system and subscription method significantly improve the efficiency and flexibility of subscription management in the Internet of Things system, adapt to the dynamically changing needs in the Internet of Things environment, and have broad application prospects.

[0008] The present invention solves the above problems through the following technical solutions:

[0009] Redis-based MQTT automatic subscription system, including MQTT Broker, Redis database and automatic subscription module;

[0010] The MQTT client communicates with the Redis database through the MQTT Broker. The MQTT Broker is integrated with the Redis database and is responsible for handling the connection, disconnection, message publishing and subscription requests of the MQTT client.

[0011] The Redis database is connected to the automatic subscription module and is used to store system configuration information, real-time data, subscription topic information and subscription status, and adopts a distributed architecture;

[0012] The automatic subscription module is responsible for automatically managing the subscription and update of MQTT client topics based on predefined rules and data in the Redis database.

[0013] As a further improvement, the automatic subscription module includes: a rule engine, a subscription manager and a data monitor; wherein,

[0014] The rule engine decides which topics need to be subscribed or unsubscribed based on predefined rules;

[0015] The subscription manager is responsible for interacting with the MQTT client and performing actual subscription and unsubscription operations;

[0016] The data monitor monitors the subject data and real-time data changes in the Redis database in real time. When it monitors data changes that meet the rule conditions, it triggers the rule engine to perform subscription management.

[0017] As a further improvement, the rule engine is set according to the predefined rules based on the conditions of time, events or data changes.

[0018] As a further improvement, the automatic subscription module further includes:

[0019] Rule description language parser, used to support rule definition of complex condition combinations;

[0020] Rule priority management and timing triggering mechanism based on Redis lists and sorted sets, used to trigger the rule engine;

[0021] Dynamic loading and hot update functions of rules allow you to update rules without restarting the system.

[0022] As a further improvement, the automatic subscription module uses Redis bitmap and HyperLogLog structure to achieve large-scale subscription statistics; a multi-level cache strategy is adopted to optimize the subscription query performance of hot topics.

[0023] As a further improvement, the Redis database uses the following data structure to store subscription topic information and status:

[0024] Use Hash structure to store MQTT client subscription topic information and subscription status, where key is MQTT client ID, field is topic, and value is subscription status;

[0025] Use the Set structure to map topics to MQTT clients.

[0026] As a further improvement, the system also includes: a management and monitoring module, which is connected to the Redis database, used to monitor and manage the system's operating status, provide system configuration, status monitoring, log recording and alarm notification functions, and ensure stable system operation and timely response.

[0027] As a further improvement, the management and monitoring module includes:

[0028] Configuration manager, used to manage the configuration parameters and rules of the system; the configuration parameters include: the configuration of the MQTT server connected to the MQTT client, the connection configuration of the Redis database and the subscription rule configuration parameters;

[0029] The status monitor is used to monitor the operating status of each module of the system in real time and display it on the monitoring interface; the operating status includes: connection status, message transmission status and data storage status;

[0030] Logger, used to record the system's operation log and event log, so as to record the system's operation events and abnormal conditions in the log file, providing a basis for troubleshooting and system optimization;

[0031] The alarm system is used to trigger the alarm notification function when an abnormal situation is monitored, and automatically send alarm information to the system administrator.

[0032] As a further improvement, the management and monitoring module is also used for:

[0033] Through the distributed subscription state synchronization mechanism, the publish / subscribe function of the Redis database is used to achieve real-time synchronization between multiple nodes;

[0034] Atomic operation processing based on Redis transactions ensures the consistency of subscription operations.

[0035] At the same time, the present invention solves the above problems through the following technical solutions:

[0036] A subscription method implemented by the above-mentioned MQTT automatic subscription system based on redis includes the following steps:

[0037] A. The redis database receives the subscription request from the MQTT client;

[0038] B. The redis database parses the subscription topic and MQTT client information, and the MQTT client connects to the MQTT server;

[0039] C. Store the subscription topic information in the Redis cluster, which is integrated by multiple Redis databases;

[0040] D. The automatic subscription module evaluates and automatically executes the subscription rules, synchronously updates the subscription status to the Redis cluster, and sends a subscription confirmation to the MQTT client.

[0041] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0042] The present invention utilizes the high performance characteristics of the Redis database to increase the subscription query speed by 100 times and achieve millisecond-level response. Through the optimized data structure, the storage efficiency of subscription information is increased by 50%, effectively reducing system resource consumption. The execution efficiency of automatic subscription rules is high, supporting millisecond-level rule evaluation and subscription adjustment. It also supports the horizontal expansion of MQTT Broker and Redis cluster, and theoretically can expand the system capacity infinitely. It provides rich monitoring indicators and visual interfaces to make the system operation status clear at a glance. It supports seamless docking with the existing MQTT system, and the migration cost is extremely low. The stability and reliability of the system are improved, and user satisfaction is significantly improved.

[0043] In summary, the present invention comprehensively improves the performance, reliability, flexibility and scalability of the IoT messaging system by innovatively combining the advantages of the MQTT protocol and the Redis database. These improvements not only solve the subscription management problem in large-scale IoT applications, but also provide a solid technical foundation for the further development and application of the IoT platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a structural block diagram of the MQTT automatic subscription system based on redis of the present invention;

[0045] Figure 2 It is a flow chart of the interaction between the MQTT client and the Redis database of the present invention;

[0046] Figure 3 This is a workflow diagram of the automatic subscription module of the present invention;

[0047] Figure 4 This is a flow chart of direct interaction between an MQTT client and an MQTT server in the prior art of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described in more detail below in conjunction with the drawings in the preferred embodiments of the present application. In the drawings, the same or similar reference numerals throughout represent the same or similar parts or parts with the same or similar functions. The described embodiments are part of the embodiments of the present application, not all of the embodiments. The embodiments described below with reference to the drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limitations on the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0049] The following is combined with Figure 1-4 The embodiments of the present application are described in detail.

[0050] Redis is an open source in-memory database known for its high performance and rich data structures. It supports multiple data types such as strings, lists, sets, and hashes, and can quickly store and query data. The main advantages of Redis include:

[0051] Efficient data storage and query: Memory operations are fast and can respond to data storage and query requests in milliseconds.

[0052] Support publish / subscribe mode: The built-in publish / subscribe function makes it very suitable for message notification and real-time data update.

[0053] Persistence support: Although Redis mainly operates in memory, it still supports persisting data to disk to ensure data security.

[0054] Embodiment 1:

[0055] Combined with Figure 1-3 As shown, a redis-based MQTT automatic subscription system includes an MQTT Broker, a Redis database, and an automatic subscription module;

[0056] The MQTT client communicates with the Redis database through the MQTT Broker. The MQTT Broker is responsible for processing the connection, message publishing and subscription requests of the MQTT client. By using the MQTT Broker to implement publish / subscribe communication, the efficiency and scalability of the MQTT protocol can be significantly improved.

[0057] Furthermore, the MQTT client is connected to the Redis database and the MQTT database respectively. The interaction process between the MQTT client and the Redis database is as follows: Figure 2As shown, the message topic of the MQTT client is stored in the Redis database, and the published data will refer to the configuration information stored in the configuration data storage. Then, the topic is passed to the message publisher of the MQTT client through the Redis database for message publishing.

[0058] The Redis database is connected to the automatic subscription module to store system configuration information, real-time data, subscription topic information, and subscription status; and adopts a distributed architecture to ensure high availability and scalability;

[0059] The automatic subscription module is responsible for automatically managing the subscription and update of MQTT client topics according to predefined rules and data in the Redis database. It is the core module of the system and determines the flexibility and automation of the system.

[0060] Specifically, the automatic subscription module includes: a rule engine, a subscription manager, and a data monitor; wherein the rule engine determines which topics need to be subscribed or unsubscribed based on predefined rules; preferably, the rules can be set based on conditions such as time, events, or data changes. The subscription manager is responsible for interacting with the MQTT client to perform actual subscription and unsubscription operations. The data monitor monitors the topic data and real-time data changes in the Redis database in real time, triggering the rule engine to perform subscription management.

[0061] In this embodiment, first, the system administrator or user can set subscription rules and conditions in the Redis database. The workflow of the automatic subscription module is as follows: Figure 3 As shown, the data monitor in the automatic subscription module continuously monitors the topic data and real-time data in the Redis database. When the data changes that meet the rule conditions are monitored, the rule engine is notified; the rule engine starts to update the subscription information in the Redis database, determines which subscription or cancellation operations need to be performed, and issues subscription or cancellation instructions to the subscription manager. The subscription manager interacts with the mqtt client and sends the subscription information to the mqtt client for actual operation to perform actual topic subscription and cancellation.

[0062] In one possible embodiment, the automatic subscription module further comprises:

[0063] Rule description language parser, used to support rule definition of complex condition combinations;

[0064] Rule priority management and timing triggering mechanism based on Redis lists and sorted sets, used to trigger the rule engine;

[0065] Dynamic loading and hot update functions of rules allow you to update rules without restarting the system.

[0066] Furthermore, the automatic subscription module also uses Redis bitmap and HyperLogLog structure to achieve large-scale subscription statistics; and adopts a multi-level cache strategy to optimize the subscription query performance of hot topics.

[0067] In another possible embodiment, the Redis database uses the following data structure to store subscription information and status:

[0068] Use Hash structure to store MQTT client subscription information, where key is MQTT client ID, field is topic, and value is subscription status;

[0069] Use the Set structure to implement the mapping from topics to MQTT clients to support efficient message push.

[0070] In another possible embodiment, a redis-based MQTT automatic subscription system further includes: a management and monitoring module connected to the Redis database, used to monitor and manage the operating status of the system, provide system configuration, status monitoring, log recording and alarm notification functions, and ensure the stable operation and timely response of the system. The management and monitoring module can be extended to support automatic subscription rules based on multiple dimensions such as time and device type, further improving the flexibility of the system.

[0071] Specifically, the management and monitoring module includes a configuration manager, a status monitor, a log recorder and an alarm system, wherein the configuration manager is used to manage the configuration parameters of the system, and the configuration parameters include: the configuration of the MQTT server connected to the MQTT client, the connection configuration of the Redis database and the subscription rule configuration, etc.; the status monitor is used to monitor the operating status of each module of the system in real time, and the operating status includes the connection status, message transmission status and data storage status, etc.; the log recorder is used to record the system's operating log and event log, and provide a basis for troubleshooting and system optimization; the alarm system is used to trigger the alarm notification function when an abnormal situation is monitored, and send an alarm message to the system administrator.

[0072] The system administrator sets and adjusts various parameters and rules of the system through the configuration manager. The status monitor obtains the operating status of each module of the system in real time and displays it on the monitoring interface. The logger records the operating events and abnormal conditions of the system in the log file. When the system detects an abnormal situation (such as connection interruption, message loss, etc.), the alarm system will automatically send a notification to remind the administrator to handle it.

[0073] Preferably, the management and monitoring module further:

[0074] Through the distributed subscription state synchronization mechanism, the publish / subscribe function of Redis is used to achieve real-time synchronization between multiple nodes;

[0075] Atomic operation processing based on Redis transactions ensures the consistency of subscription operations.

[0076] The MQTT automatic subscription based on Redis in this embodiment also dynamically adjusts the number of subscription topics according to the system load to optimize system performance.

[0077] Embodiment 2:

[0078] See attached Figure 2 , 3 , a redis-based mqtt automatic subscription method, implemented by a redis-based mqtt automatic subscription system, including:

[0079] A. The redis database receives the subscription request from the MQTT client;

[0080] B. The redis database parses the subscription topic and MQTT client information, and the MQTT client connects to the MQTT server;

[0081] C. Store the subscription topic information in the Redis cluster, which is integrated by multiple Redis databases;

[0082] D. The automatic subscription module evaluates and automatically executes the subscription rules, synchronously updates the subscription status to the Redis cluster, and sends a subscription confirmation to the MQTT client.

[0083] And by regularly backing up the data in the Redis database, the persistence and security of the data are ensured.

[0084] Although the present invention is described herein with reference to the illustrative embodiments of the present invention, the above embodiments are only preferred embodiments of the present invention, and the embodiments of the present invention are not limited to the above embodiments. It should be understood that those skilled in the art can design many other modifications and embodiments, which will fall within the scope and spirit of the principles disclosed in this application.

Claims

1. The MQTT automatic subscription system based on redis is characterized by: Includes MQTT Broker, Redis database and automatic subscription module; The MQTT client communicates with the Redis database through the MQTT Broker. The MQTT Broker is integrated with the Redis database and is responsible for handling the connection, disconnection, message publishing and subscription requests of the MQTT client. The Redis database is connected to the automatic subscription module and is used to store system configuration information, real-time data, subscription topic information and subscription status, and adopts a distributed architecture; The automatic subscription module is responsible for automatically managing the subscription and update of MQTT client topics based on predefined rules and data in the Redis database.

2. According to the redis-based MQTT automatic subscription system of claim 1, it is characterized in that: The automatic subscription module includes: a rule engine, a subscription manager and a data monitor; wherein, The rule engine determines which topics need to be subscribed or unsubscribed based on predefined rules; The subscription manager is responsible for interacting with the MQTT client and performing actual subscription and unsubscription operations; The data monitor monitors the subject data and real-time data changes in the Redis database in real time. When it monitors data changes that meet the rule conditions, it triggers the rule engine to perform subscription management.

3. According to the redis-based MQTT automatic subscription system of claim 2, it is characterized in that: The rule engine is set according to the predefined rules based on the conditions of time, events or data changes.

4. According to the redis-based MQTT automatic subscription system of claim 2, it is characterized in that: The automatic subscription module also includes: Rule description language parser, used to support rule definition of complex condition combinations; Rule priority management and timing triggering mechanism based on Redis lists and sorted sets are used to trigger the rule engine; Dynamic loading and hot update functions of rules allow you to update rules without restarting the system.

5. According to the redis-based MQTT automatic subscription system of claim 1, it is characterized in that: The automatic subscription module uses the bitmap and HyperLogLog structure of Redis to achieve large-scale subscription statistics; and adopts a multi-level cache strategy to optimize the subscription query performance of hot topics.

6. According to the redis-based MQTT automatic subscription system of claim 1, it is characterized in that: The Redis database uses the following data structure to store subscription topic information and status: Use Hash structure to store MQTT client subscription topic information and subscription status, where key is MQTT client ID, field is topic, and value is subscription status; Use the Set structure to map topics to MQTT clients.

7. According to the redis-based MQTT automatic subscription system of claim 1, it is characterized in that: The system also includes: a management and monitoring module, which is connected to the Redis database and is used to monitor and manage the operating status of the system, provide system configuration, status monitoring, log recording and alarm notification functions, and ensure stable operation and timely response of the system.

8. According to the redis-based MQTT automatic subscription system of claim 7, it is characterized in that: The management and monitoring module includes: Configuration manager, used to manage the configuration parameters and rules of the system; the configuration parameters include: the configuration of the MQTT server connected to the MQTT client, the connection configuration of the Redis database and the subscription rule configuration parameters; The status monitor is used to monitor the operating status of each module of the system in real time and display it on the monitoring interface; the operating status includes: connection status, message transmission status and data storage status; Logger, used to record the system's operation log and event log, so as to record the system's operation events and abnormal conditions in the log file, providing a basis for troubleshooting and system optimization; The alarm system is used to trigger the alarm notification function when an abnormal situation is monitored, and automatically send alarm information to the system administrator.

9. According to the redis-based MQTT automatic subscription system of claim 7, it is characterized in that: The management and monitoring module is also used for: Through the distributed subscription state synchronization mechanism, the publish / subscribe function of the Redis database is used to achieve real-time synchronization between multiple nodes; Atomic operation processing based on Redis transactions ensures the consistency of subscription operations.

10. A subscription method implemented by using the MQTT automatic subscription system based on redis as described in any one of claims 1 to 9, characterized in that: The following steps are involved: A. The redis database receives the subscription request from the MQTT client; B. The redis database parses the subscription topic and MQTT client information, and the MQTT client connects to the MQTT server; C. Store the subscription topic information in the Redis cluster, which is integrated by multiple Redis databases; D. The automatic subscription module evaluates and automatically executes the subscription rules, synchronously updates the subscription status to the Redis cluster, and sends a subscription confirmation to the MQTT client.

Images