IoT Data Compaction Using Attribute-Based Representative Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing volume of data from IoT devices poses economic and technical challenges in provisioning storage capacity and bandwidth, as existing compression methods require significant storage capacity and may not be suitable for devices with limited resources.
Innovation Solution
A data compaction system that dynamically selects compaction schemes based on attributes like bounded deviation, threshold sensitivity, and skew, using techniques such as function fitting, piecewise fitting, and bitmap indexing to represent data efficiently, allowing edge devices to retain more data without external storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If compression is used to reduce data volume, then storage capacity requirements are reduced, but appreciable storage capacity is still required and bandwidth provisioning challenges remain
Solution Approach 1:
The patent transforms data from its original format into a compacted representation by changing parameters such as aggregating multiple data points into single summary statistics (min, max, average, sum) and encoding temporal patterns. This parameter transformation reduces the quantity of data that needs to be stored and transmitted while preserving essential information characteristics.
Solution Approach 2:
The patent extracts only the most essential features and patterns from the original data stream, separating critical information (trend indicators, anomaly detections, summary statistics) from redundant details. This extraction process creates a condensed data representation that maintains analytical value while significantly reducing storage and bandwidth requirements.
2Quantity of substance
If data is compacted to reduce storage needs, then storage capacity is reduced, but data retrieval and processing capabilities must be maintained
Solution Approach 1:
The patent performs preliminary compaction and aggregation of data before storage, pre-processing the data stream to create compacted representations with embedded metadata and summary statistics. This preliminary action ensures that when data is retrieved, it is already in an optimized format that facilitates efficient processing and analysis, eliminating the need for complex post-retrieval processing.
Solution Approach 2:
The patent introduces intermediary data structures and compaction algorithms that serve as mediators between the original data stream and the storage system. These intermediaries transform raw data into compacted formats while preserving queryability and analytical capabilities, enabling efficient retrieval and processing without sacrificing data utility.
3Measurement precision
If lossless compression is used to maintain data accuracy, then data quality is preserved, but storage capacity requirements remain appreciable
Solution Approach 1:
The patent applies different levels of data representation quality to different portions of the data based on local characteristics. Critical data points and anomaly detections are preserved with high fidelity, while routine or predictable data patterns are represented with lower precision using aggregation and approximation techniques. This local quality differentiation maintains overall data accuracy while significantly reducing storage requirements.
Data Source
AI summary
Methods, apparatuses, and storage media associated with compaction of data from one or more computing devices are disclosed. In various embodiments, one or more Internet of Things (IoT) devices may transmit information to a computing system. The computing system may group together raw data received from these one or more IoT devices based on a shared attribute. The computing system may select a compaction scheme to represent the knowledge conveyed by a group of the raw data. The computing system may apply this compaction scheme to the group of raw data to generate data that is representative of the group of raw data. Other embodiments may be disclosed or claimed.