Mixed Sensor Data Compression for Bandwidth-Limited Edge Gateways
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in efficiently transmitting and processing the large volumes of mixed qualitative and quantitative sensing data from edge devices, such as vehicles, due to bandwidth limitations and the inability of existing gateways to handle the data size, leading to prohibitively expensive data collection and real-time streaming issues.
Innovation Solution
A device in a serial network de-multiplexes traffic streams, quantizes, and applies data-type-specific compression to segregate and transform data before sending it as Internet Protocol (IP) traffic, using a model-based, programmable compression mechanism that optimizes data flow and segregates data streams for analysis and compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is transmitted over existing network connections (LTE at Megabits range), then network infrastructure is maintained, but data transmission capacity is insufficient for large volumes of sensing data (Terabytes per day)
Solution Approach 1:
The patent applies preliminary compression and quantization to sensing data before transmission. The gateway device performs data reduction operations (compression algorithms and quantization) on raw sensing data from multiple sources, transforming terabytes of raw data into a compressed format that can be transmitted over existing Megabits-range LTE connections, thereby resolving the capacity mismatch between data generation and network transmission capabilities
Solution Approach 2:
The patent changes the parameters of data representation through quantization (reducing precision of numerical values) and compression (altering data structure). By transforming data from high-precision raw format to compressed quantized format, the system reduces data volume by orders of magnitude, enabling efficient transmission over constrained network infrastructure while preserving essential information for cloud-based analysis
2Loss of information
If all raw sensing data is transmitted to cloud-based services, then complete data availability is achieved, but bandwidth demands become prohibitively expensive
Solution Approach 1:
The patent extracts and transmits only the most relevant features and compressed representations of sensing data to cloud-based services, rather than transmitting all raw data. The gateway device performs feature extraction, event detection, and data summarization, sending only essential information (e.g., detected events, aggregated statistics, compressed representations) to the cloud, thereby maintaining data integrity for critical information while dramatically reducing bandwidth consumption
Solution Approach 2:
The gateway device performs preliminary data processing, compression, and filtering before transmission to the cloud. By pre-processing data locally (quantization, compression, feature extraction), the system preserves essential information while reducing the volume of data requiring expensive bandwidth, thus resolving the contradiction between data integrity and bandwidth consumption
3Device complexity
If existing gateways are used for data transmission, then infrastructure simplicity is maintained, but gateways cannot handle the size requirements of additional data (Terabytes per day)
Solution Approach 1:
The patent transforms the gateway device into a smart edge computing node by implementing advanced data compression algorithms, quantization engines, and feature extraction capabilities. These parameter changes in the gateway's processing abilities enable it to handle Terabytes of sensing data by reducing it to manageable compressed formats, thereby increasing data handling capacity without requiring complete infrastructure replacement
Solution Approach 2:
The patent segments the data processing function between the gateway device (edge) and cloud-based services (center). The gateway performs local compression, quantization, and preliminary analysis, while the cloud handles sophisticated analytics. This segmentation allows existing gateway hardware to handle large data volumes through distributed processing, maintaining infrastructure simplicity while dramatically increasing overall system capacity
4Measurement precision
If real-time streaming of all sensing data is implemented, then complete monitoring capability is achieved, but processing speed becomes insufficient due to data volume
Solution Approach 1:
The gateway device performs preliminary event detection, filtering, and compression before data leaves the edge device. By pre-processing data locally (identifying relevant events, filtering noise, compressing representations), the system enables faster processing speeds while maintaining monitoring accuracy, as the cloud receives pre-filtered, compressed data requiring less processing time
Solution Approach 2:
The system extracts and transmits only critical events and essential features rather than complete raw data streams. The gateway identifies and extracts meaningful events (anomalies, threshold violations, significant patterns) from terabytes of sensing data, sending only these extracted events to the cloud for real-time analysis, thereby achieving both high monitoring accuracy and fast processing speeds
Data Source
AI summary
In one embodiment, a device in a serial network de-multiplexes a stream of traffic in the serial network into a plurality of data streams. The device determines that data from a particular data stream should be reported to an entity external to the serial network based on an event indicated by the data from the particular data stream. The device quantizes the data from the particular data stream. The device applies compression to the quantized data to form a compressed representation of the particular data stream. The applied compression is selected based on a data type associated with the data. The device sends a compressed representation of the particular data stream to the external entity as Internet Protocol (IP) traffic.