An internet of things multi-protocol gas sensing data collection system

Abstract

The application belongs to the technical field of Internet of Things system architecture, and particularly relates to a multi-protocol gas sensing data acquisition system of Internet of Things, which comprises a singleton MQTT bridging module, a port forced release module, a registration center heartbeat fine adjustment module, a cold and hot dual storage writing module, a multi-protocol message distribution module, a Spring Boot Admin client disabling module and a multi-protocol unified access processing bottom module; the singleton MQTT bridging module is arranged in the application, only a single node is specified to run the MQTT Bridge service in the device access cluster, a globally unique and fixed MQTT client identifier is configured for each data acquisition channel, a connection is established with the MQTT Broker through an exclusive binding mode, the MQTT protocol specification requirements are strictly met, and the mutual kicking disconnection problem caused by the multi-service instance sharing of the client identifier is completely avoided.

Description

Technical Field

[0001] This invention relates to the field of Internet of Things (IoT) system architecture technology, specifically to an IoT multi-protocol gas sensing data acquisition system. Background Technology

[0002] With the continuous development of the Internet of Things, industrial safety and environmental governance, the accurate sensing, real-time acquisition and efficient transmission of gas concentration have become core requirements in scenarios such as industrial production, urban environmental protection, smart mines, healthcare, and chemical industrial parks. As a key carrier connecting the physical sensing layer and the cloud application layer, the performance of the gas sensing data acquisition system directly determines the accuracy, real-time performance and reliability of gas monitoring, and plays an irreplaceable role in ensuring personnel safety, preventing and controlling environmental risks and optimizing production management.

[0003] From the development history of gas monitoring technology, it has gradually evolved from early passive biological early warning and single-point instrumental monitoring to the distributed and intelligent monitoring stage of the Internet of Things (IoT) era. Currently, IoT gas sensor data acquisition systems are widely used in various scenarios. By deploying multiple types of gas sensors, they can simultaneously monitor various gas components such as CO, NO2, SO2, VOCs, CO2, and CH2O, as well as environmental parameters such as temperature and humidity, providing basic data support for subsequent data processing, anomaly early warning, and decision analysis. However, with the increasing complexity of monitoring scenarios, the diversification of sensor types, and the expansion of network scale, existing IoT gas sensor data acquisition systems still have many technical shortcomings in practical applications:

[0004] In large-scale deployments of IoT gas sensing devices, data from these devices is transmitted to the cloud via various protocols such as MQTT and HTTP. To achieve high availability at the access layer, existing technologies typically deploy the same MQTT access service on multiple nodes. However, the MQTT protocol (section 3.1.3.1 of the MQTT 3.1.1 specification) stipulates that the client-id on the same Broker must be globally unique. When two instances connect using the same client-id, the Broker forcibly disconnects the first instance; the first instance automatically reconnects and then disconnects the second instance, creating a "mutual disconnect storm" with a message loss rate approaching 100%. Furthermore, existing technical literature lacks effective solutions for this scenario.

[0005] In addition, failure to clean up zombie processes on the same port during service restart caused port binding failure, preventing the service from starting normally; the default heartbeat timeout of 30 seconds in the registry center caused faulty nodes to persist for too long, resulting in a large number of requests being routed to unavailable instances; real-time queries and historical analysis used the same database, causing high-concurrency real-time requests and complex SQL analysis queries to interfere with each other, resulting in a serious decline in overall service quality.

[0006] Gas monitoring demands extremely high real-time performance and stability in data transmission, but existing systems have significant shortcomings in complex scenarios. In environments with strong signal interference and long transmission distances, such as underground mines and large chemical industrial parks, the system is susceptible to environmental factors, leading to problems such as data packet loss, transmission delays, and communication interruptions.

[0007] To address these issues, an IoT multi-protocol gas sensing data acquisition system is proposed. Summary of the Invention

[0008] The purpose of this invention is to provide an Internet of Things (IoT) multi-protocol gas sensing data acquisition system to solve the problems mentioned in the background art.

[0009] To achieve the above objectives, the present invention provides the following technical solution: an IoT multi-protocol gas sensor data acquisition system, comprising a single-instance MQTT bridging module, a port forced release module, a registry center heartbeat fine-tuning module, a cold and hot dual storage writing module, a multi-protocol message distribution module, a Spring Boot Admin client disabling module, and a multi-protocol unified access processing underlying module;

[0010] The multi-protocol unified access processing underlying module establishes data interaction connections with the single-instance MQTT bridging module and the multi-protocol message distribution module, respectively. The port forced release module serves as a global startup pre-support module. The registry center heartbeat fine-tuning module establishes service registration and health heartbeat connections with the single-instance MQTT bridging module, the multi-protocol message distribution module, the cold and hot dual-storage write module, and the multi-protocol unified access processing underlying module. The startup of the single-instance MQTT bridging module is supported by the port resources of the port forced release module, and its service registration and health status are managed by the registry center heartbeat fine-tuning module. Simultaneously, it receives MQTT protocol sensor data uploaded by the multi-protocol unified access processing underlying module, binds it with a unique client identifier, maintains the automatic reconnection link, and establishes a data forwarding connection with the multi-protocol message distribution module, normalizing the MQTT data. Uplink messages are pushed to the message distribution module; the multi-protocol message distribution module establishes connections with the single-instance MQTT bridge module, the multi-protocol unified access processing underlying module, and the cold and hot dual-storage write module respectively: it receives MQTT messages forwarded by the single-instance MQTT bridge module, receives other protocol messages gathered by the HTTP access layer and the underlying module, completes routing and message deduplication according to the device channel identifier, and pushes the valid collected data to the cold and hot dual-storage write module; the front end of the cold and hot dual-storage write module connects to the multi-protocol message distribution module to obtain compliant gas sensor collection data, and performs dual-path writing of Redis hot storage and relational database cold storage; at the same time, its own microservice connects to the governance system of the registry center heartbeat fine-tuning module, and relies on cluster deployment to achieve layered storage of real-time data and historical data and isolation of read and write requests.

[0011] Preferably, the single-instance MQTT bridge module: A single node is designated in the device access service cluster to run the MQTT Bridge service. The MQTT Bridge service maintains a globally unique and fixed MQTT client identifier (client-id) for each data acquisition channel, establishing a connection with the MQTT Broker through exclusive binding, fundamentally eliminating the mutual disconnection problem caused by multiple instance client-id conflicts; the MQTT Bridge service has a built-in automatic reconnection mechanism, initiating a reconnection attempt within 3 seconds after a disconnection, with a maximum backoff time of no more than 60 seconds.

[0012] Before the MQTT Bridge and other microservices start, the port forced release module performs the operation of forcibly releasing the target TCP port, cleans up all residual processes occupying the port, and ensures that the service process can successfully bind to the port;

[0013] The registry center heartbeat fine-tuning module fine-tunes the combination of three parameters: heartbeat interval, heartbeat timeout, and instance deletion timeout, reducing the duration of faulty instances in the registry center from the default value to less than 15 seconds.

[0014] When processing each piece of collected data, the hot and cold dual-storage write module performs dual-path write operations simultaneously: writing recent window data for real-time querying to the Redis cluster (hot storage); and writing full data for historical analysis to the relational database (cold storage); the real-time interface reads from the hot storage, and the historical interface reads from the cold storage, with the two types of requests completely isolated.

[0015] The multi-protocol message distribution module receives uplink messages from the MQTT Bridge and the HTTP access layer, and routes them to the corresponding message processing service according to the device channel identifier, thereby realizing unified access and distribution for MQTT protocol devices and HTTP protocol devices.

[0016] The Spring Boot Admin client disables the microservice client's active registration function, avoiding issues such as dual registration of microservices, duplicate instance display, and chaotic cluster management, and ensuring the stability of the visualized management of microservice cluster operation and maintenance.

[0017] The multi-protocol unified access processing underlying module is compatible with multiple communication protocols of industrial gas sensors, adapts to complex interference scenarios in chemical industrial parks and mines, solves defects such as heterogeneous access of multi-protocol devices, data packet loss, transmission delay, and communication interruption, and ensures continuous, stable, and real-time acquisition and transmission of gas sensing data.

[0018] The multi-protocol unified access processing underlying module establishes data interaction connections with the single-instance MQTT bridging module and the multi-protocol message distribution module, respectively. The port forced release module serves as a global startup pre-support module. The registry center heartbeat fine-tuning module establishes service registration and health heartbeat connections with the single-instance MQTT bridging module, the multi-protocol message distribution module, the cold and hot dual-storage write module, and the multi-protocol unified access processing underlying module. The startup of the single-instance MQTT bridging module is supported by the port resources of the port forced release module, and its service registration and health status are managed by the registry center heartbeat fine-tuning module. Simultaneously, it receives MQTT protocol sensor data uploaded by the multi-protocol unified access processing underlying module, binds it with a unique client identifier, maintains the automatic reconnection link, and establishes a data forwarding connection with the multi-protocol message distribution module, normalizing the MQTT data. Uplink messages are pushed to the message distribution module; the multi-protocol message distribution module establishes connections with the single-instance MQTT bridge module, the multi-protocol unified access processing underlying module, and the cold and hot dual-storage write module respectively: it receives MQTT messages forwarded by the single-instance MQTT bridge module, receives other protocol messages gathered by the HTTP access layer and the underlying module, completes routing and message deduplication according to the device channel identifier, and pushes the valid collected data to the cold and hot dual-storage write module; the front end of the cold and hot dual-storage write module connects to the multi-protocol message distribution module to obtain compliant gas sensor collection data, and performs dual-path writing of Redis hot storage and relational database cold storage; at the same time, its own microservice connects to the governance system of the registry center heartbeat fine-tuning module, and relies on cluster deployment to achieve layered storage of real-time data and historical data and isolation of read and write requests.

[0019] Preferably, the singleton MQTT bridge module includes a three-level process self-healing mechanism:

[0020] In the first stage, the watchdog daemon checks the MQTT Bridge process's liveness at fixed intervals (≤5 minutes) and performs a restart operation if an abnormality occurs.

[0021] At the second level, the operating system service manager (such as systemd) is configured to automatically restart within 10 seconds after a process exits;

[0022] The third level sets a limit on the number of times the device can be started. When the threshold is exceeded, an alarm is triggered to notify the operation and maintenance personnel. The three-level self-healing mechanism ensures that the MQTT Bridge service can run stably for a long time without human intervention, and the mean time to recovery (MTTR) is less than 15 seconds.

[0023] Preferably, the registry center heartbeat fine-tuning module sets heart-beat-interval to 3000ms, heart-beat-timeout to 9000ms, and ip-delete-timeout to 15000ms; at the same time, it configures the registry center gRPC communication timeout parameter to 3000ms, so that when any 1-2 registry center nodes fail, the healthy node can take over within 3 seconds, and the system unavailability time window is shortened from the default 30 seconds to 3 seconds.

[0024] Preferably, in the cold and hot dual storage writing module, the hot storage is deployed in a Redis cluster mode, containing no less than 7 nodes, supporting data sharding and automatic failover; the cold storage adopts a managed relational database cluster, containing read-write separated master nodes and read-only nodes; the message processing service writes to the cold storage synchronously and to the hot storage asynchronously. When the hot storage write fails, a failure log is recorded but the main process is not affected, ensuring the integrity of historical data.

[0025] Preferably, the multi-protocol message distribution module includes a message deduplication mechanism. Each uplink message carries a sequence number (seq) generated by the device. The message processing service checks whether the seq already exists before writing to storage, and duplicate messages are discarded directly. The deduplication mechanism is based on a deduplication window (time range ≤ 5 minutes) in hot storage. Historical messages that exceed the window time are not included in the deduplication judgment.

[0026] Preferably, the Spring Boot Admin client disable module is forcibly set to spring.boot.admin.client.enabled=false in the JVM startup parameters of all microservices to prevent services from registering both through the registry center and directly with Spring Boot Admin, thus avoiding the problem of duplicate instance display caused by double registration.

[0027] Preferably, the single-instance MQTT bridging module includes a single-instance deployment and scheduling unit, a unique client identifier binding unit, an automatic reconnection unit after disconnection, and a three-level process self-healing unit;

[0028] The port forced release module includes a port detection unit, a residual process retrieval unit, a process forced cleanup unit, and a port status verification unit;

[0029] The registration center heartbeat fine-tuning module includes a heartbeat parameter configuration unit, a timeout threshold control unit, a gRPC communication timeliness optimization unit, and a fault node rapid removal and cluster takeover unit.

[0030] The cold and hot dual storage write module includes an asynchronous hot storage write unit, a synchronous cold storage write unit, a Redis cluster deployment unit, a relational database read-write separation unit, and a write exception log recording and process fault tolerance unit.

[0031] The multi-protocol message distribution module includes a multi-protocol unified access receiving unit, a channel identifier routing distribution unit, a device serial number verification unit, a short-time window message deduplication unit, and a duplicate data filtering and discarding unit;

[0032] The Spring Boot Admin client disable module includes a global JVM startup parameter unified configuration unit, a client registration switch forced disable unit, and a microservice registration source unified management unit;

[0033] The underlying module for unified access processing of multiple protocols includes a multi-protocol compatibility and adaptation unit, a complex working condition signal anti-interference unit, a data transmission stability control unit, and a disconnection resumption and transmission delay suppression unit.

[0034] Compared with the prior art, the beneficial effects of the present invention are:

[0035] 1) This application sets up a single-instance MQTT bridge module, specifying only a single node to run the MQTT Bridge service in the device access cluster. It configures a globally unique and fixed MQTT client identifier for each data acquisition channel and establishes a connection with the MQTT Broker through an exclusive binding method, strictly complying with the requirements of the MQTT protocol specification and completely avoiding the mutual kick-out disconnection problem caused by multiple service instances sharing the client identifier. At the same time, it is equipped with a 3-second fast automatic reconnection, a maximum 60-second backoff limit, and a three-level process self-healing mechanism. Even if the link is temporarily disconnected or the process is abnormal, the service can be quickly restored, preventing data disconnection and message loss caused by mutual kick-out storms, and filling the gap in the existing technology that does not specifically solve the defects of this type of cluster conflict.

[0036] 2) This application configures a port forced release module, which automatically detects and forcibly releases target TCP ports before the MQTT bridging service and various microservices start, and batch cleans up residual zombie processes and invalid background processes occupying ports, ensuring that service processes can normally occupy and bind communication ports. This solves the problems of service startup failure and system deployment anomalies caused by port resource occupation from the source, and ensures the stable start and stop of the overall system. A registry center heartbeat fine-tuning module is added to accurately adapt and optimize core parameters such as heartbeat interval, heartbeat timeout, and instance deletion timeout, compressing the lifespan of faulty instances to within 15 seconds. At the same time, the gRPC communication timeout parameter is limited. When one or two nodes of the registry center fail, healthy nodes can quickly take over the cluster service within 3 seconds, greatly shortening the system downtime, avoiding a large number of business requests being continuously distributed to failed instances, reducing invalid request losses, and improving the overall fault tolerance and scheduling rationality of the microservice cluster.

[0037] 3) This application adopts a hot and cold dual storage write module to implement a dual-path separate storage architecture for gas acquisition data. It uses a Redis cluster for hot storage to cache recent real-time monitoring data and relies on a read-write separated relational database for cold storage to archive all historical data. The system divides the interface call logic, with real-time business requests reading from hot storage and historical statistics and data analysis requests reading from cold storage. The two types of data access requests are completely physically isolated. Simultaneously, by combining differentiated write strategies for hot and cold storage, cluster deployment, and failover mechanisms, the system avoids resource contention between high-concurrency real-time read / write operations and complex analysis SQL queries, effectively improving system response speed and concurrency capacity, and ensuring data integrity without loss. A multi-protocol message distribution module uniformly handles uplink data from the MQTT bridging service and the HTTP access layer, relying on device channel identifiers to achieve precise routing and distribution, enabling unified access and scheduling of gas sensing devices with different communication protocols. Coupled with a sequence number deduplication mechanism based on short-time windows, duplicate reported data is automatically identified and discarded, reducing redundant data storage and computational pressure, and adapting to large-scale application scenarios with mixed deployments of multiple types of sensing devices. Through the Spring Boot Admin client's disable module, microservice JVM startup parameters are configured globally and uniformly, forcibly disabling the client's active registration function, unifying the microservice registration entry point, eliminating dual service registration, ensuring accurate instance display on the operation and maintenance monitoring interface, maintaining microservice cluster management order, and assisting in the long-term stable operation and maintenance of the system.

[0038] 4) This application is equipped with a multi-protocol unified access processing underlying module, which is compatible with various industrial-grade gas sensor communication protocols. It performs underlying adaptation and optimization for the working conditions of strong electromagnetic interference and complex transmission environment in underground mines and large chemical industrial parks, strengthens the data anti-interference transmission capability, effectively improves defects such as data packet loss, transmission delay and short-term communication interruption, and ensures continuous, stable and low-latency acquisition and uploading of gas sensing data in harsh industrial environments, meeting the high standard requirements of safety production monitoring. Attached Figure Description

[0039] Figure 1 This is a diagram showing the module composition of the system of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] Example:

[0042] Please see Figure 1 The present invention provides a technical solution:

[0043] An IoT multi-protocol gas sensor data acquisition system includes a single-instance MQTT bridging module, a port forced release module, a registry center heartbeat fine-tuning module, a cold and hot dual storage writing module, a multi-protocol message distribution module, a Spring BootAdmin client disabling module, and a multi-protocol unified access processing underlying module.

[0044] The multi-protocol unified access processing underlying module establishes data interaction connections with the single-instance MQTT bridging module and the multi-protocol message distribution module, respectively. The port forced release module serves as a global startup pre-support module. The registry center heartbeat fine-tuning module establishes service registration and health heartbeat connections with the single-instance MQTT bridging module, the multi-protocol message distribution module, the cold and hot dual-storage write module, and the multi-protocol unified access processing underlying module. The startup of the single-instance MQTT bridging module is supported by the port resources of the port forced release module, and its service registration and health status are managed by the registry center heartbeat fine-tuning module. Simultaneously, it receives MQTT protocol sensor data uploaded by the multi-protocol unified access processing underlying module, binds it with a unique client identifier, maintains the automatic reconnection link, and establishes a data forwarding connection with the multi-protocol message distribution module, normalizing the MQTT data. Uplink messages are pushed to the message distribution module; the multi-protocol message distribution module establishes connections with the single-instance MQTT bridge module, the multi-protocol unified access processing underlying module, and the cold and hot dual-storage write module respectively: it receives MQTT messages forwarded by the single-instance MQTT bridge module, receives other protocol messages gathered by the HTTP access layer and the underlying module, completes routing and message deduplication according to the device channel identifier, and pushes the valid collected data to the cold and hot dual-storage write module; the front end of the cold and hot dual-storage write module connects to the multi-protocol message distribution module to obtain compliant gas sensor collection data, and performs dual-path writing of Redis hot storage and relational database cold storage; at the same time, its own microservice connects to the governance system of the registry center heartbeat fine-tuning module, and relies on cluster deployment to achieve layered storage of real-time data and historical data and isolation of read and write requests.

[0045] Singleton MQTT Bridge Module: The MQTT Bridge service runs on a single designated node within the device access service cluster. Each data acquisition channel maintains a globally unique and fixed MQTT client identifier (client-id), establishing a connection with the MQTT Broker through exclusive binding, fundamentally eliminating the mutual disconnection problem caused by client-id conflicts across multiple instances. The MQTT Bridge service has a built-in automatic reconnection mechanism, initiating a reconnection attempt within 3 seconds of a disconnection, with a maximum backoff time of no more than 60 seconds.

[0046] Before the MQTT Bridge and other microservices start, the port forced release module performs the operation of forcibly releasing the target TCP port, cleans up all residual processes occupying the port, and ensures that the service process can successfully bind to the port;

[0047] The registry center heartbeat fine-tuning module fine-tunes the combination of three parameters: heartbeat interval, heartbeat timeout, and instance deletion timeout, reducing the duration of faulty instances in the registry center from the default value to less than 15 seconds.

[0048] When processing each piece of collected data, the hot and cold dual-storage write module performs dual-path write operations simultaneously: writing recent window data for real-time querying to the Redis cluster (hot storage); and writing full data for historical analysis to the relational database (cold storage); the real-time interface reads from the hot storage, and the historical interface reads from the cold storage, with the two types of requests completely isolated.

[0049] The multi-protocol message distribution module receives uplink messages from the MQTT Bridge and the HTTP access layer, and routes them to the corresponding message processing service according to the device channel identifier, thereby realizing unified access and distribution for MQTT protocol devices and HTTP protocol devices.

[0050] The Spring Boot Admin client disables the microservice client's active registration function, avoiding issues such as dual registration of microservices, duplicate instance display, and chaotic cluster management, and ensuring the stability of the visualized management of microservice cluster operation and maintenance.

[0051] The multi-protocol unified access processing underlying module is compatible with multiple communication protocols of industrial gas sensors, adapts to complex interference scenarios in chemical industrial parks and mines, solves defects such as heterogeneous access of multi-protocol devices, data packet loss, transmission delay, and communication interruption, and ensures continuous, stable, and real-time acquisition and transmission of gas sensing data.

[0052] The multi-protocol unified access processing underlying module establishes data interaction connections with the single-instance MQTT bridging module and the multi-protocol message distribution module, respectively. The port forced release module serves as a global startup pre-support module. The registry center heartbeat fine-tuning module establishes service registration and health heartbeat connections with the single-instance MQTT bridging module, the multi-protocol message distribution module, the cold and hot dual-storage write module, and the multi-protocol unified access processing underlying module. The startup of the single-instance MQTT bridging module is supported by the port resources of the port forced release module, and its service registration and health status are managed by the registry center heartbeat fine-tuning module. Simultaneously, it receives MQTT protocol sensor data uploaded by the multi-protocol unified access processing underlying module, binds it with a unique client identifier, maintains the automatic reconnection link, and establishes a data forwarding connection with the multi-protocol message distribution module, normalizing the MQTT data. Uplink messages are pushed to the message distribution module; the multi-protocol message distribution module establishes connections with the single-instance MQTT bridge module, the multi-protocol unified access processing underlying module, and the cold and hot dual-storage write module respectively: it receives MQTT messages forwarded by the single-instance MQTT bridge module, receives other protocol messages gathered by the HTTP access layer and the underlying module, completes routing and message deduplication according to the device channel identifier, and pushes the valid collected data to the cold and hot dual-storage write module; the front end of the cold and hot dual-storage write module connects to the multi-protocol message distribution module to obtain compliant gas sensor collection data, and performs dual-path writing of Redis hot storage and relational database cold storage; at the same time, its own microservice connects to the governance system of the registry center heartbeat fine-tuning module, and relies on cluster deployment to achieve layered storage of real-time data and historical data and isolation of read and write requests.

[0053] The singleton MQTT bridge module includes a three-level process self-healing mechanism:

[0054] In the first stage, the watchdog daemon checks the MQTT Bridge process's liveness at fixed intervals (≤5 minutes) and performs a restart operation if an abnormality occurs.

[0055] At the second level, the operating system service manager (such as systemd) is configured to automatically restart within 10 seconds after a process exits;

[0056] The third level sets a limit on the number of times the device can be started. When the threshold is exceeded, an alarm is triggered to notify the operation and maintenance personnel. The three-level self-healing mechanism ensures that the MQTT Bridge service can run stably for a long time without human intervention, with an average fault recovery time (MTTR) of less than 15 seconds.

[0057] The singleton MQTT bridging module has a built-in three-level process self-healing mechanism. Through three layers of protection—daemon process inspection, system-level rapid self-start, and over-limit operation and maintenance alarm—the MQTT bridging service can be run unattended and automatically recover from faults, which greatly shortens the average fault recovery time and avoids the loss of gas sensor data transmission across the entire domain due to abnormal bridging process crashes, thereby improving the operational reliability and fault tolerance of the entire access layer.

[0058] The registry center heartbeat fine-tuning module sets `heart-beat-interval` to 3000ms, `heart-beat-timeout` to 9000ms, and `ip-delete-timeout` to 15000ms. Simultaneously, it configures the registry center gRPC communication timeout parameter to 3000ms. This ensures that if any 1-2 registry center nodes fail, healthy nodes can take over within 3 seconds, shortening the system downtime window from the default 30 seconds to 3 seconds. Further quantification and limitation of the registry center's complete set of key parameters for heartbeat, timeout, and cluster switching, precisely fixing the heartbeat interval, heartbeat timeout, instance deletion timeout, and gRPC communication timeout configurations, allows for rapid removal of faulty service instances, shortens the dwell time of faulty nodes, and enables seamless takeover within seconds when multiple registry center nodes fail. This significantly reduces the system downtime window and improves the high availability of the microservice cluster.

[0059] In the hot and cold storage dual-storage write module, hot storage is deployed in a Redis cluster mode with no fewer than 7 nodes, supporting data sharding and automatic failover; cold storage uses a managed relational database cluster, including read-write separated master nodes and read-only nodes; the message processing service writes synchronously to cold storage and asynchronously to hot storage. When hot storage writes fail, a failure log is recorded but the main process is not affected, ensuring the integrity of historical data. Specifically, the cluster deployment architecture, node size, read-write separation strategy, read-write asynchronous and synchronous mechanisms, and fault tolerance logic for the hot and cold storage are defined. Hot storage uses a multi-node Redis cluster to ensure real-time data sharding and failover; cold storage uses a master-slave read-write separated database cluster to ensure stable storage of massive amounts of historical data; differentiated read-write methods and exception log fault tolerance ensure both real-time query concurrency performance and that all historical data is not lost and business processes are not interrupted.

[0060] The multi-protocol message distribution module includes a message deduplication mechanism. Each uplink message carries a device-generated sequence number (seq). Before writing to storage, the message processing service checks if the seq already exists, discarding duplicate messages. The deduplication mechanism is based on a deduplication window (time range ≤ 5 minutes) in hot storage; historical messages exceeding the window time are not included in the deduplication process. Relying on the device's unique sequence number and the hot storage timed deduplication window, it automatically identifies and discards duplicate gas sensor messages, effectively filtering duplicate data caused by network jitter and device reconnection, reducing storage redundancy and backend computing pressure, and ensuring the uniqueness and validity of the collected data.

[0061] The Spring Boot Admin client disable module is configured by forcibly setting `spring.boot.admin.client.enabled=false` in the JVM startup parameters of all microservices. This prevents services from registering both through the registry center and directly with Spring Boot Admin, avoiding duplicate instance display issues caused by dual registration. By uniformly configuring JVM startup parameters to forcibly disable active client registration, dual registration of microservices, duplicate instance display, and inconsistent operation and maintenance monitoring interfaces are eliminated from the ground up, ensuring a unified microservice cluster topology display and simple and standardized operation and maintenance management.

[0062] The single-instance MQTT bridge module includes a single-instance deployment and scheduling unit, a unique client identifier binding unit, an automatic reconnection unit after disconnection, and a three-level process self-healing unit. The single-instance deployment and scheduling unit selects only a single node in the cluster to run the MQTTBridge service. The unique client identifier binding unit assigns a globally fixed client-id to each collection channel and exclusively binds it to the MQTT Broker. The automatic reconnection unit initiates reconnection within 3 seconds and backs off for a maximum of 60 seconds to ensure rapid link recovery.

[0063] The port forced release module includes a port detection unit, a residual process retrieval unit, a process forced cleanup unit, and a port status verification unit; it addresses issues such as zombie processes occupying TCP ports after service restart, port binding failures, and microservices failing to start normally.

[0064] The registry center heartbeat fine-tuning module includes a heartbeat parameter configuration unit, a timeout threshold control unit, a gRPC communication timeliness optimization unit, and a faulty node rapid removal and cluster takeover unit; it addresses issues such as the registry center's default lenient heartbeat timeliness, faulty instances remaining for too long, incorrect request routing, slow registry center node failover, and excessively long system unavailability windows.

[0065] The cold and hot dual storage write module includes an asynchronous hot storage write unit, a synchronous cold storage write unit, a Redis cluster deployment unit, a relational database read-write separation unit, and a write exception log recording and process fault tolerance unit; it addresses the problems of real-time query and historical analysis sharing the same database and competing for resources, high concurrency lag, slow real-time response, easy loss of historical full data, and mutual interference between data query services;

[0066] The multi-protocol message distribution module includes a multi-protocol unified access receiving unit, a channel identifier routing distribution unit, a device serial number verification unit, a short-time window message deduplication unit, and a duplicate data filtering and discarding unit;

[0067] The Spring Boot Admin client disables modules including a global JVM startup parameter unified configuration unit, a client registration switch forced disabling unit, and a microservice registration source unified management unit;

[0068] The multi-protocol unified access processing underlying module includes a multi-type industrial protocol compatibility and adaptation unit, a complex working condition signal anti-interference unit, a data transmission stability control unit, and a disconnection resumption and transmission delay suppression unit.

[0069] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or basic characteristics. Therefore, the embodiments should be considered exemplary and non-limiting in all respects. The scope of the invention is defined by the appended claims rather than the foregoing description. Therefore, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0070] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An Internet of Things (IoT) multi-protocol gas sensing data acquisition system, characterized in that, It includes a singleton MQTT bridging module, a port forced release module, a registry center heartbeat fine-tuning module, a cold and hot dual storage writing module, a multi-protocol message distribution module, and a multi-protocol unified access processing underlying module; The multi-protocol unified access processing underlying module establishes data interaction connections with the single-instance MQTT bridging module and the multi-protocol message distribution module respectively; the port forced release module is a global startup front-end support module; The registry center heartbeat fine-tuning module establishes service registration and health heartbeat connections with the singleton MQTT bridging module, the multi-protocol message distribution module, the cold and hot dual-storage write module, and the multi-protocol unified access processing underlying module. The singleton MQTT bridging module's startup is supported by port resources from the forced release module, and its service registration and health status are managed by the registry center heartbeat fine-tuning module. Simultaneously, it receives MQTT protocol sensor data uploaded by the multi-protocol unified access processing underlying module. After binding with a unique client identifier and maintaining automatic reconnection links, it establishes a data forwarding connection with the multi-protocol message distribution module, pushing the normalized MQTT uplink messages to the message distribution module. The multi-protocol message distribution module also establishes connections with the singleton MQTT bridging module. The bridging module, the multi-protocol unified access processing underlying module, and the cold / hot dual-storage write module establish a connection: It receives MQTT messages forwarded by the single-instance MQTT bridging module, receives other protocol messages aggregated by the HTTP access layer and the underlying module, completes routing and message deduplication according to the device channel identifier, and pushes the valid collected data to the cold / hot dual-storage write module; the front end of the cold / hot dual-storage write module connects to the multi-protocol message distribution module to obtain compliant gas sensor data, and performs dual-path writing of Redis hot storage and relational database cold storage; at the same time, its own microservice connects to the governance system of the registry center heartbeat fine-tuning module, relying on cluster deployment to achieve layered storage of real-time data and historical data and isolation of read / write requests.

2. The IoT multi-protocol gas sensing data acquisition system according to claim 1, characterized in that: The single-instance MQTT bridge module uniquely designates one node in the device access service cluster to run the MQTT Bridge service. The MQTT Bridge service maintains a globally unique and fixed MQTT client identifier (client-id) for each data acquisition channel, and establishes a connection with the MQTT Broker through an exclusive binding method, fundamentally eliminating the mutual disconnection problem caused by client-id conflicts among multiple instances. The MQTT Bridge service has a built-in automatic reconnection mechanism, which initiates a reconnection attempt within 3 seconds after a disconnection, with a maximum backoff time of no more than 60 seconds. Before the MQTT Bridge and other microservices start, the port forced release module performs the operation of forcibly releasing the target TCP port, cleans up all residual processes occupying the port, and ensures that the service process can successfully bind to the port; The registry center heartbeat fine-tuning module fine-tunes the combination of three parameters: heartbeat interval, heartbeat timeout, and instance deletion timeout, reducing the duration of faulty instances in the registry center from the default value to less than 15 seconds. The cold and hot dual-storage write module performs dual-path write operations simultaneously when processing each piece of collected data: writing recent window data for real-time querying to the Redis cluster; and writing full data for historical analysis to the relational database. Real-time API reads are stored hot, while historical API reads are stored cold, ensuring complete isolation between the two types of requests. The multi-protocol message distribution module receives uplink messages from the MQTT Bridge and the HTTP access layer, and routes them to the corresponding message processing service according to the device channel identifier, thereby realizing unified access and distribution for MQTT protocol devices and HTTP protocol devices. The multi-protocol unified access processing underlying module is compatible with multiple communication protocols of industrial gas sensors, adapts to complex interference scenarios in chemical industrial parks and mines, solves defects such as heterogeneous access of multi-protocol devices, data packet loss, transmission delay, and communication interruption, and ensures continuous, stable, and real-time acquisition and transmission of gas sensing data.

3. The IoT multi-protocol gas sensing data acquisition system according to claim 1, characterized in that: The singleton MQTT bridge module includes a three-level process self-healing mechanism: The first level involves the watchdog daemon checking the MQTT Bridge process's liveness at fixed intervals and performing a restart operation if an abnormality occurs. At the second level, the operating system service manager is configured to automatically restart within 10 seconds after a process exits; The third level sets a limit on the number of times the device can be started. When the threshold is exceeded, an alarm is triggered to notify the operation and maintenance personnel. The three-level self-healing mechanism ensures that the MQTT Bridge service can run stably for a long time without human intervention, and the mean time to recovery (MTTR) is less than 15 seconds.

4. The IoT multi-protocol gas sensing data acquisition system according to claim 1, characterized in that: The registry center heartbeat fine-tuning module sets heart-beat-interval to 3000ms, heart-beat-timeout to 9000ms, and ip-delete-timeout to 15000ms; it also configures the registry center gRPC communication timeout parameter to 3000ms, so that if any 1-2 registry center nodes fail, the healthy node can take over within 3 seconds, and the system unavailability time window is shortened from the default 30 seconds to 3 seconds.

5. The IoT multi-protocol gas sensing data acquisition system according to claim 1, characterized in that: In the hot and cold dual storage write module, the hot storage is deployed in Redis cluster mode, containing no less than 7 nodes, and supports data sharding and automatic failover; the cold storage adopts a managed relational database cluster, containing read-write separated master nodes and read-only nodes; the message processing service writes to the cold storage synchronously and to the hot storage asynchronously. When the hot storage write fails, a failure log is recorded but the main process is not affected, ensuring the integrity of historical data.

6. The IoT multi-protocol gas sensing data acquisition system according to claim 1, characterized in that: The multi-protocol message distribution module includes a message deduplication mechanism. Each uplink message carries a sequence number (seq) generated by the device. The message processing service checks whether the seq already exists before writing to storage, and duplicate messages are discarded directly. The deduplication mechanism is based on a deduplication window in hot storage, and historical messages that exceed the window time are not included in the deduplication judgment.

7. The IoT multi-protocol gas sensing data acquisition system according to claim 1, characterized in that: It also includes a Spring Boot Admin client disabling module, which uniformly disables the microservice client's active registration function, avoiding the problems of dual registration of microservices, duplicate instance display, and chaotic cluster management, and ensuring the stability of microservice cluster operation and maintenance visualization management; the Spring Boot Admin client disabling module forces the setting of spring.boot.admin.client.enabled=false in the JVM startup parameters of all microservices to prevent services from registering both through the registry center and directly with Spring Boot Admin, avoiding the problem of duplicate instance display caused by dual registration.

8. The IoT multi-protocol gas sensing data acquisition system according to claim 7, characterized in that: The singleton MQTT bridging module includes a single-instance deployment and scheduling unit, a unique client identifier binding unit, an automatic reconnection unit after disconnection, and a three-level process self-healing unit. The port forced release module includes a port detection unit, a residual process retrieval unit, a process forced cleanup unit, and a port status verification unit; The registration center heartbeat fine-tuning module includes a heartbeat parameter configuration unit, a timeout threshold control unit, a gRPC communication timeliness optimization unit, and a fault node rapid removal and cluster takeover unit. The cold and hot dual storage write module includes an asynchronous hot storage write unit, a synchronous cold storage write unit, a Redis cluster deployment unit, a relational database read-write separation unit, and a write exception log recording and process fault tolerance unit. The multi-protocol message distribution module includes a multi-protocol unified access receiving unit, a channel identifier routing distribution unit, a device serial number verification unit, a short-time window message deduplication unit, and a duplicate data filtering and discarding unit; The Spring Boot Admin client disable module includes a global JVM startup parameter unified configuration unit, a client registration switch forced disable unit, and a microservice registration source unified management unit; The underlying module for unified access processing of multiple protocols includes a multi-protocol compatibility and adaptation unit, a complex working condition signal anti-interference unit, a data transmission stability control unit, and a disconnection resumption and transmission delay suppression unit.

Images