Sensor Data Compression by Parameter Grouping and Normalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in efficiently compressing and storing large amounts of sensor data from various IoT sensors due to varying value ranges and the presence of disturbances, leading to resource wastage and reduced compression efficiency.
Innovation Solution
A storage system that groups sensors based on parameters, normalizes the data, and collectively compresses it using a neural network, with advanced data management techniques like normalization parameter databases and context-adaptive encoding to optimize compression without losing accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If existing compression methods are used on sensor data, then compression is performed, but compression accuracy deteriorates due to varying value ranges and disturbances in sensor data
Solution Approach 1:
The patent applies parameter changes by normalizing sensor data to a common range (0-1) using minimum and maximum values. This transformation changes the parameter range of the data, allowing effective compression while preserving accuracy. The normalization process converts varying sensor value ranges into a unified scale, making the data suitable for compression algorithms.
Solution Approach 2:
The patent segments sensor data into fixed-size blocks (e.g., 256 samples per block) before compression. This segmentation allows the compression algorithm to process data in manageable units, improving both compression efficiency and accuracy by applying compression techniques to homogeneous data segments rather than the entire dataset at once.
2Quantity of substance
If storage resources are allocated for large amounts of sensor data, then all sensor data can be stored, but storage resource waste increases
Solution Approach 1:
The patent changes the parameter of data representation by normalizing sensor values to a 0-1 range, which significantly reduces the number of bits required to represent each value. This parameter transformation enables more efficient storage allocation, allowing the system to store large volumes of sensor data without proportionally increasing storage resource consumption.
Solution Approach 2:
The patent performs preliminary normalization and compression of sensor data before storage. By preprocessing the data to reduce its size and optimize its format, the system prepares the data in advance for efficient storage, preventing resource waste that would occur if raw, unprocessed sensor data were stored directly.
3Productivity
If sensor data with varying value ranges is compressed collectively, then processing efficiency improves, but compression accuracy deteriorates
Solution Approach 1:
The patent applies parameter changes by normalizing all sensor data to a common 0-1 range using the minimum and maximum values across the dataset. This uniform parameter transformation enables different sensor types with varying value ranges to be processed together efficiently while maintaining compression accuracy, as all data now shares the same scale and distribution characteristics.
Solution Approach 2:
The patent creates a universal compression approach that handles multiple sensor types with different value ranges through a single normalization and compression pipeline. This universal method applies the same processing steps (normalization using global min-max, block segmentation, and compression) to all sensor data regardless of source, achieving both processing efficiency and accuracy.
Data Source
AI summary
Provided is a storage system that can store various types of and large amounts of sensor data while accurately compressing the sensor data without wasting storage resources. The storage system includes: a storage that records sensor data output from a plurality of sensors; a processor that controls recording of sensor data in the storage; and a memory that records parameters of the plurality of sensors. The processor reads parameters assigned to the sensors that output the sensor data from the memory, normalizes the sensor data based on the parameters, compresses the normalized sensor data, and records the compressed sensor data in the storage.