Telemetry Tensor Compression for Irregular Time-Series Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression techniques for time-series telemetry data are inefficient, particularly for irregular and high-frequency data, leading to large file sizes and brittle data pipelines that hinder the collection and transmission of valuable telemetry data, which is crucial for anomaly detection and user support.
Innovation Solution
The system compresses time-series telemetry data by creating a tensor with regular numerical distances, combining values, and storing only nonzero values in sparse formats, resulting in a significantly reduced memory footprint while maintaining near-lossless data integrity for data science operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If current data compression techniques are used for irregular high-frequency telemetry data, then data transmission and storage become possible, but the file sizes remain large and data pipelines become brittle
Solution Approach 1:
The patent transforms irregular time-series data into regular intervals by resampling, changing the temporal parameter structure. This allows the data to be represented in a standardized tensor format with regular time steps, enabling more efficient compression while maintaining data integrity for anomaly detection operations.
Solution Approach 2:
The patent applies different processing strategies to different parts of the data: irregular intervals are resampled to regular intervals, and the resulting tensor is stored in sparse format where only nonzero values are retained. This localized optimization reduces overall data size while preserving critical information structure.
2Measurement precision
If telemetry data is collected at high frequency with irregular intervals, then more detailed performance metrics are captured, but the data size increases significantly
Solution Approach 1:
The patent changes the temporal sampling parameter from irregular high-frequency intervals to regular intervals through resampling. This transformation maintains the precision of performance metric capture while reducing the total data volume by eliminating redundant samples and standardizing the time structure.
Solution Approach 2:
The patent extracts only the essential nonzero values from the resampled tensor and stores them in sparse format, removing redundant zero values. This extraction process significantly reduces data size while preserving the critical performance metric information needed for analysis.
3Quantity of substance
If data is compressed to reduce file size, then storage and transmission efficiency improve, but data integrity for scientific operations may be compromised
Solution Approach 1:
The patent applies a deterministic resampling transformation that converts irregular intervals to regular intervals without loss of information. This parameter transformation maintains data fidelity because it is a reversible mathematical operation that preserves the underlying performance metric relationships while enabling efficient compression.
Solution Approach 2:
The patent performs resampling and tensor transformation as a preliminary action before compression and storage. By establishing the regular interval structure upfront, the system enables subsequent sparse storage that achieves compression while maintaining the mathematical properties needed for accurate scientific operations and anomaly detection.
Data Source
AI summary
A system can identify a first group of time-series telemetry data that represents performance metrics of a computing device, wherein the first group of time-series telemetry data identifies respective first values and corresponding respective first timestamps. The system can create a second group of time-series telemetry data that identifies second timestamps. The system can populate the second group of time-series telemetry data with the respective first values at respective first locations of the second group of time-series telemetry data that correspond to the respective first timestamps of the respective first values. The system can create a tensor that identifies third timestamps. The system can populate the tensor with the respective first values at respective second locations of the tensor that correspond to the respective first timestamps of the respective first values, wherein populating the tensor comprises combining two values of the respective first values.