Telemetry Data Compression via Johnson-Lindenstrauss Transform
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression techniques for hardware telemetry data result in large data sets due to high sampling rates, leading to significant storage and transmission challenges, with existing methods failing to efficiently reduce the dimensionality of time-series telemetry data while preserving its characteristics.
Innovation Solution
The implementation of a Johnson-Lindenstrauss transform (JLT) with linear mappings to project tabular telemetry data into aligned tensors, achieving a significant reduction in dimensionality while maintaining the integrity of the data for use in data science applications, resulting in a compressed form that is homomorphic and usable for data science operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high sampling rates are used for telemetry data collection, then measurement precision is improved, but data size increases leading to storage and transmission challenges
Solution Approach 1:
The patent transforms the data from its original high-dimensional format into a lower-dimensional representation by changing the dimensional parameters. This dimensionality reduction allows the data to be compressed by more than 20,000-fold while preserving the essential characteristics needed for accurate measurements and analysis.
2Quantity of substance
If dimensionality reduction is applied to telemetry data, then data size is reduced, but data characteristics may be lost
Solution Approach 1:
The patent applies dimensionality reduction techniques that project the high-dimensional telemetry data into a lower-dimensional space while preserving the essential characteristics. This transformation maintains the homomorphic properties of the data, ensuring that data science operations produce results within a predetermined threshold of the original data.
3Quantity of substance
If data compression is applied to telemetry data, then storage and transmission efficiency is improved, but computation complexity may increase
Solution Approach 1:
The patent performs dimensionality reduction and compression of the telemetry data before data science operations are applied. This preliminary transformation reduces the computational complexity of subsequent operations while maintaining the accuracy of results, as the compressed data retains the essential characteristics needed for meaningful analysis.
Data Source
AI summary
A system can identify a group of time-series telemetry data that represents performance metrics of computing devices, wherein the group of time-series telemetry data is represented according to a first number of dimensions. The system can compress the group of time-series telemetry data, wherein the compressed group of time-series telemetry data is represented according to a second number of dimensions that is less than the first number of dimensions, wherein the compressed group of time-series telemetry data is homomorphic. The system can perform a data science operation on the compressed group of time-series telemetry data to produce a first result, wherein the first result is within a predetermined threshold value of a second result of performing the data science operation on the group of time-series telemetry data.