Multimodal Data Reduction Agent for IIoT Time-Series Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Industrial automation systems generate vast amounts of near-real-time data, leading to data congestion and processing latency in cloud-based applications due to high data point density, with simple data truncation methods compromising data accuracy and losing associative links between reduced and original data sets.
Innovation Solution
Implementing a node system within the IIoT data pipeline that performs real-time distribution analysis to select appropriate data reduction algorithms based on the probability distribution of time-series data, reducing data volume while maintaining critical information and associative links between reduced and raw data sets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If simple data truncation methods are used to reduce data volume, then data processing load is reduced, but data accuracy is compromised and associative links between reduced and original data sets are lost
Solution Approach 1:
The system changes the parameter of data reduction from simple truncation to mode-based filtering. By analyzing the probability distribution of time-series data and identifying modes (most frequent values), the system selectively retains data points that represent significant patterns while removing redundant measurements, thus maintaining accuracy while reducing volume
Solution Approach 2:
The system incorporates feedback through mode detection and probability distribution analysis. By continuously analyzing the statistical characteristics of the data and using this feedback to guide the reduction process, the system ensures that only data points that deviate significantly from established patterns are retained, maintaining data integrity
2Measurement precision
If high data point density is maintained in cloud-based applications, then data accuracy is preserved, but data congestion and processing latency occur
Solution Approach 1:
The system performs preliminary data reduction at the edge or data collection point before data reaches the cloud-based application. By applying mode-based filtering in advance to remove redundant data points, the system reduces the data volume that needs to be processed in the cloud, thereby decreasing processing latency while maintaining accuracy
Solution Approach 2:
The system segments the data processing function into two parts: local mode-based filtering to reduce volume, and cloud-based analysis to maintain accuracy. This segmentation allows each component to operate optimally without overwhelming the other
3Quantity of substance
If data reduction algorithms are applied to reduce data volume, then data processing load is decreased, but complexity of the reduction process increases
Solution Approach 1:
The system uses self-service by automatically detecting the probability distribution of the time-series data and identifying modes without requiring manual configuration. The algorithm autonomously analyzes the data characteristics and applies appropriate reduction strategies, reducing the need for complex manual intervention
Data Source
AI summary
Data reduction services are implemented on one or more nodes of an IIoT data pipeline to intelligently determine an appropriate data reduction strategy based on characteristics of the incoming data. In one or more embodiments, data reduction components on the pipeline node or on an edge device define different data filtering rules or algorithms that are selectively applied to streaming time-series data based on a probability distribution of the data. The data pipeline node performs real-time distribution analysis on the streaming data to determine whether the data has a unimodal distribution, a multimodal distribution, or no mode, and selects one of the data filtering rules based on this determined probability distribution. In this way, the data is intelligently reduced in a manner that retains critical information within the reduced data set while achieving a high level of data reduction.