Sensor Data Reconstruction Using ML to Correct Lossy Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
IoT devices face challenges in transmitting data due to limited power resources, leading to the use of lossy compression algorithms that result in compression errors, where data quality is compromised to reduce power consumption.
Innovation Solution
A system and method that employs a machine learning model, trained on patterns of the compression algorithm and symbolic aggregate approximation metadata, to reconstruct lossy compressed data at a gateway, reducing compression errors without increasing energy expenditure on IoT devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If lossy compression is applied to reduce data transmission, then power consumption is reduced, but data quality deteriorates due to compression errors
Solution Approach 1:
The system segments the compression task between two stages: first, simple lossy compression at the IoT device to reduce power consumption; second, a reconstruction stage at the server using machine learning models to correct compression errors. This segmentation allows each stage to optimize for its specific function, resolving the contradiction between power savings and data quality.
Solution Approach 2:
A machine learning reconstruction model acts as an intermediary between the compressed data and the final reconstructed data. This intermediary corrects the compression errors introduced by lossy compression, thereby improving data quality without requiring the IoT device to perform power-intensive compression operations.
2Productivity
If simple lossy compression is used to save power, then compression speed increases and power consumption decreases, but compression error increases
Solution Approach 1:
The system performs preliminary simple lossy compression at the IoT device to achieve fast compression and low power consumption. Then, a machine learning model performs preliminary reconstruction at the server before final data usage, correcting compression errors in advance. This two-stage preliminary action approach maintains both speed and reliability.
Solution Approach 2:
The machine learning reconstruction model provides feedback correction to the compressed data. By analyzing the compressed data and applying learned correction patterns, the system feedbacks error corrections that reduce compression errors while maintaining the efficiency of the original simple compression algorithm.
3Quantity of substance
If data is compressed to reduce transmission amount, then bandwidth costs are reduced, but data accuracy is lost
Solution Approach 1:
The system substitutes traditional mechanical lossless compression (which would preserve accuracy but increase data volume) with a combination of lossy compression and machine learning-based reconstruction. The machine learning model replaces the need for exact data preservation by learning and correcting compression artifacts, achieving both reduced data volume and maintained accuracy.
Data Source
AI summary
Reconstructing compressed data to reduce compression loss. Data at a device is compressed using lossy compression and metadata values are added to the compressed data. A gateway receives a package including the compressed data and the metadata values. The data is decompressed and input to a machine learning model along with the metadata values. The machine learning model is trained to reduce the compression loss. The output of the model is an improved decompressed data. Actions may be performed based on the improved decompressed data.