IoT Data Stream Classification Using Feature Engineering and Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in automatically classifying non-structured data streams from IoT devices without prior data categorization, especially in providing information about data sources, which hinders efficient processing and insights generation.
Innovation Solution
A computer-implemented method that uses feature engineering techniques, such as measurement summarization, histogram generation, and Fourier Transform, to extract features from IoT data streams, followed by neural network training with a predetermined topology, enabling the classification of non-structured data streams from unknown sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If lightweight messaging protocols are used for IoT devices, then communication efficiency is improved, but data source information is lost
Solution Approach 1:
The patent introduces an intermediary component that intercepts and processes MQTT messages before they reach the main application. This intermediary extracts and stores data source information (device ID, topic, payload metadata) separately from the minimal message body, allowing the system to maintain lightweight communication while preserving source information for later classification and analysis.
2Productivity
If automatic classification of non-structured data streams is implemented, then processing efficiency is improved, but system complexity increases
Solution Approach 1:
The patent implements preliminary action by pre-processing incoming data streams through feature extraction and transformation before classification. The system applies feature engineering techniques (statistical features, temporal features, domain-specific features) to transform raw non-structured data into structured feature vectors, which are then fed to pre-trained classification models. This preliminary transformation simplifies the classification task and improves processing efficiency while managing system complexity through modular architecture.
Solution Approach 2:
The classification system employs self-service mechanisms where the model automatically adapts to new data sources and types without manual intervention. The system performs self-training on incoming data streams, automatically updating classification boundaries and adapting to new IoT devices and protocols. This self-service capability reduces operational complexity while maintaining high processing efficiency for diverse non-structured data streams.
3Measurement precision
If feature engineering techniques are applied to extract features from partitioned data, then classification accuracy is improved, but computational overhead increases
Solution Approach 1:
The patent applies segmentation by dividing incoming data streams into fixed-size partitions or windows before feature extraction. Each partition is processed independently to extract statistical features (mean, variance, skewness), temporal features (trend, seasonality), and domain-specific features. This segmentation approach enables efficient batch processing, reduces computational overhead by processing smaller chunks, while maintaining high classification accuracy through comprehensive feature extraction from each partition.
Data Source
AI summary
Automated classification of non-structure data streams from a plurality of Internet of Things (IoT) devices includes receiving, by a computer, from the plurality of IoT devices a data stream including a set of labeled readings S with a predetermined sample size n and a predetermined partition size m. The received data stream is partitioned into a partition set S′ including m readings. The computer determines a set of features associated with the data stream based on the partition set S′ by applying feature engineering techniques. A vector representation of the obtained set of features is built by the computer to place each feature on a same range scale. A predetermined minimum number of layers and neurons is then selected based on the set of features for training a neural network. Finally, non-structured data streams from new or unknown data sources can be classified using the trained neural network.