IoT Gateway Data Segmentation for MQTT Transmission
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In IoT systems, network delays or infrastructure failures can cause data interruptions, leading to discarded or manually uploaded breakpoint data, which is time-consuming and may result in data integrity issues.
Innovation Solution
The method involves determining the type of data (real-time or historical) when a connection between a gateway device and a server is abnormal, storing it in separate message queues, and transmitting it over distinct MQTT channels upon connection restoration, ensuring timely and integral data upload without manual intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If breakpoint data are cached to a disk file or database and manually uploaded to the cloud after communication resumes, then data integrity is maintained, but the operation becomes complicated and time-consuming, and timeliness of data upload is not ensured
Solution Approach 1:
The patent segments the data transmission process by creating separate message queues for real-time data and historical data. Real-time data is transmitted through a real-time message queue with immediate upload capability, while historical data is stored in a separate historical message queue for batch processing. This segmentation allows critical real-time data to be uploaded promptly without waiting for manual intervention, thus reducing data upload time while maintaining integrity through structured queue management
Solution Approach 2:
The patent introduces a message-oriented middleware as an intermediary between the gateway device and the cloud. This middleware includes multiple message queues that automatically manage data buffering and transmission. When communication resumes, the system automatically uploads data from the message queues without requiring manual intervention, thus reducing both upload time and operational complexity while ensuring data integrity through the intermediary's structured handling
2Reliability
If breakpoint data are manually uploaded to the cloud according to a timestamp, then data completeness is achieved, but the operation is complicated and requires manual intervention
Solution Approach 1:
The patent implements self-service automation where the message-oriented middleware automatically manages the upload of breakpoint data without human intervention. The system monitors communication status, automatically buffers data in appropriate message queues during disruptions, and autonomously uploads data from these queues when communication resumes. This eliminates the need for manual timestamp-based uploads while ensuring data completeness through automatic tracking and transmission
Solution Approach 2:
The patent incorporates feedback mechanisms where the gateway device and message-oriented middleware continuously monitor communication status and data transmission states. When communication is interrupted, the system receives feedback about the disruption and automatically adjusts by buffering data in appropriate queues. When communication resumes, the feedback triggers automatic upload operations, eliminating manual intervention while ensuring complete data transmission through continuous status monitoring
3Device complexity
If real-time data and historical data are stored in the same message queue, then system complexity is reduced, but real-time data upload timeliness is compromised
Solution Approach 1:
The patent divides the single message queue into multiple specialized queues: a real-time message queue for immediate data transmission and a historical message queue for batch processing. This segmentation ensures that real-time data can be uploaded immediately without being blocked by historical data processing, thus improving real-time upload speed. The increased complexity of multiple queues is justified by the critical performance improvement in real-time data transmission
Data Source
AI summary
Disclosed are a method and apparatus for transmitting data in an IoT system. The method includes: determining a data type of device data sent by an IoT device when a connection between a gateway device and a server is abnormal; storing device data of a real-time data type into a first message queue of a message-oriented middleware, and storing device data of a historical data type into a second message queue of the message-oriented middleware; transmuting the device data in the first message queue to a server over a first MQTT channel and transmitting the device data in the second message queue to the server over a second MQTT channel when the connection resumes to a normal state.