Industrial Telemetry Streaming With Adaptive Queue And Payload Sizing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for transmitting industrial telemetry data from industrial sites to remote servers suffer from substantial storage requirements due to decoupled data collection and publication processes, leading to limited remote monitoring and significant delays between data transmissions.
Innovation Solution
A method for streaming industrial telemetry data using a data elements queue and data frames buffer, with adaptive control loops to optimize queuing time and frame size, incorporating machine learning algorithms to manage data flow and minimize storage needs, ensuring quasi-continuous data transmission with minimal delay and loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data elements are collected continuously and stored in a source database before periodic publishing, then data storage capacity is increased, but transmission delay and monitoring precision deteriorate due to the timespan between publications
Solution Approach 1:
The patent segments the data flow into two independent streams: a continuous collection stream that immediately stores data elements in a collecting database with minimal delay, and a periodic publishing stream that extracts and transmits data frames at scheduled intervals. This segmentation allows simultaneous achievement of continuous data capture (reducing transmission delay) and periodic efficient transmission (managing storage capacity).
Solution Approach 2:
The patent introduces a collecting database as an intermediary component between industrial devices and the periodic publishing process. This intermediary immediately receives and stores data elements from devices, eliminating the waiting time caused by periodic batch processing, while still allowing scheduled publishing to occur efficiently.
2Device complexity
If data elements are published periodically in batch files, then transmission process is simplified, but remote monitoring precision deteriorates due to non-negligible timespan between publications
Solution Approach 1:
The patent separates the collection function from the publishing function, allowing continuous collection to maintain simplicity in the publishing process while improving monitoring precision through immediate data availability in the collecting database.
Solution Approach 2:
The collecting database serves as an intermediary that enables both continuous data capture for precision monitoring and periodic batch publishing for process simplicity. Data elements are immediately available in the intermediary for rapid access, eliminating the need for complex real-time streaming mechanisms.
3Loss of time
If data elements are immediately transmitted from industrial devices to remote server, then transmission delay is reduced, but data loss increases due to congestion and buffer overflow
Solution Approach 1:
The patent performs preliminary action by pre-storing data elements in the collecting database before transmission is needed. This allows the system to immediately acknowledge data receipt (reducing perceived delay) while having data ready for transmission when bandwidth is available, preventing data loss during congestion.
Solution Approach 2:
The collecting database acts as a buffer intermediary between data sources and the transmission channel, absorbing congestion variability. Data elements are stored in the intermediary and transmitted at optimal moments, preventing both data loss and unnecessary delays.
4Productivity
If maximum frame size is constrained to optimize transmission efficiency, then bandwidth utilization is improved, but data element throughput decreases
Solution Approach 1:
The patent uses periodic publishing actions to accumulate multiple data elements into data frames over time. By periodically aggregating data elements into frames that respect maximum size constraints, the system optimizes bandwidth utilization while maintaining high throughput through continuous accumulation in the collecting database.
Solution Approach 2:
The system preliminarily collects and stores data elements in the collecting database before aggregating them into transmission frames. This preliminary accumulation allows optimal frame sizing that maximizes bandwidth efficiency without losing data elements, as the intermediary buffer ensures all elements are available for aggregation.
Data Source
Figure 1
AI summary
A method of streaming data elements (1) from an industrial site (3) to a remote server, comprising: receiving (S1) data elements (1) from industrial devices on the industrial site (3), storing (S2) the data elements in a queue (7), periodically extracting (S3) a number N of data elements from the queue (7) and aggregating (S4) the same into one data frame (13) whose size does not exceed a maximum size, storing (S5) each data frame in a buffer (17), and periodically extracting (S6) a data frame from the buffer (17) and sending (S6) the same to the remote server, wherein a data element is removed from the queue (7) and stored in a database (19) if the data element's time in the queue (7) exceeds a maximum queuing time, wherein a data frame is stored in the database (19) instead of being stored in the buffer (17) if the buffer is full, wherein the maximum queuing time is optimised as a function of the latest number of data elements removed from the queue (7) and the frequency at which the buffer (17) is full, and wherein the number N is optimised in view of the size of the latest resulting data frame (13) and of the number of data elements (11) aggregated therein, while respecting said maximum size.