Telemetry Data Compression Using Differential Dependent Records
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for managing telemetry data are limited in addressing intra-sample redundancies, leading to inefficient storage and management of massive amounts of telemetry data.
Innovation Solution
A lossless compression technique is employed that utilizes component identifiers (CIDs) to differentiate between telemetry samples, storing the first sample independently and subsequent samples as differential dependent records, which only include data fields that have changed, using a key-length-type-value (KLTV) encoding scheme and structured data information to maintain data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional compression methods are used that only address intra-sample redundancies, then the compression process is simple, but the compression ratio is limited and storage efficiency is poor
Solution Approach 1:
The patent segments the compression approach into two distinct record types: independent records that contain complete telemetry sample data, and differential dependent records that contain only changes from the previous sample. This segmentation allows the system to exploit inter-sample redundancies while maintaining a manageable compression protocol that processes samples sequentially.
Solution Approach 2:
The patent applies different compression strategies to different portions of the data stream. Independent records provide complete data for reconstruction purposes, while differential dependent records provide compact representations for incremental updates. This local quality differentiation optimizes the balance between compression ratio and decompression complexity.
2Quantity of substance
If all telemetry data fields are stored for every sample, then data integrity is maintained, but storage space is wasted due to redundant data
Solution Approach 1:
The patent stores complete independent records at regular intervals (e.g., every N samples) to serve as baseline references. This preliminary action ensures that data integrity is maintained by having full reconstruction points, while differential dependent records between these baselines capture only changes, thereby reducing redundant storage without risking information loss.
Solution Approach 2:
Instead of storing complete copies of every telemetry sample, the patent uses differential dependent records that copy only the changed fields from the previous sample. This copying strategy dramatically reduces storage requirements while maintaining the ability to reconstruct complete data by applying differences sequentially from the last independent record.
3Quantity of substance
If differential dependent records are used to store only changed fields, then storage efficiency is improved, but the complexity of data retrieval and reconstruction increases
Solution Approach 1:
The patent replaces complex random access operations with a simpler sequential processing mechanism. By organizing data as a chain of independent and differential dependent records, the system substitutes the need for random access to multiple complete records with a linear traversal that applies differences sequentially, simplifying the retrieval logic while maintaining compression benefits.
4Ease of operation
If the first telemetry sample is stored as an independent record, then data reconstruction is simplified, but storage efficiency is reduced compared to storing all samples as differential records
Solution Approach 1:
The patent changes the parameter of record completeness by alternating between independent records (complete data) and differential dependent records (change-only data). This parameter change strategy optimizes the balance between reconstruction simplicity and storage efficiency, as independent records provide easy reconstruction points while differential records maximize compression for samples with minimal changes.
Data Source
AI summary
Disclosed telemetry data management methods obtain, from a component of an information handling system, a component identifier (CID), uniquely indicative of the component, and telemetry samples including a first sample and one or more subsequent samples. The telemetry samples are encoded to create telemetry records, which are stored in a telemetry database. The first sample is stored as an independent record while subsequent samples are stored as dependent records. Independent records contain information sufficient to reconstruct the sample without accessing any other records whereas dependent records are not generally able to recreate the corresponding sample without referencing another record. The dependent records may be differential dependent records that indicate differences between two telemetry samples and omits any telemetry data field for which the two telemetry samples have the same value. Differential data records may include structured data information associating each telemetry data value with the correct telemetry data field.