Telemetry Time-Series Compression Using Bit Prediction Masks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current compression methods for telemetry data in space missions are inefficient, requiring significant processing power and not achieving sufficient compression ratios, especially for housekeeping data, which limits bandwidth and accuracy in monitoring remote systems.
Innovation Solution
A lossless compression method that predicts bit states in telemetry data packets based on historical data, using statistical analysis to create mask packets and perform bit operations that minimize processing requirements, allowing for real-time compression and transmission with improved compression ratios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If complicated compression software (e.g. zip) is loaded to the transmitter, then compression ratio is improved, but device complexity and processing power requirements increase
Solution Approach 1:
The compression method is segmented into two distinct phases: an offline training phase that analyzes historical telemetry data to build prediction models, and an online compression phase that uses these pre-built models for real-time encoding. This segmentation allows complex analysis to be performed beforehand when processing power is available, while runtime operations remain simple and efficient.
Solution Approach 2:
The system performs preliminary analysis of telemetry data patterns during an offline training phase, building statistical models and prediction tables before actual compression is needed. These pre-computed models capture the temporal and spatial correlations in telemetry data, enabling simple lookup-based compression during runtime without requiring complex real-time processing.
2Use of energy by moving object
If simple lossless compression techniques (e.g. RICE) are used, then processing power consumption is reduced, but compression ratio deteriorates
Solution Approach 1:
The compression system is self-adaptive, automatically learning the statistical characteristics of the specific telemetry data being compressed during the offline training phase. By analyzing the actual data patterns, temporal correlations, and bit-state transitions, the system builds customized prediction models that are optimized for that particular data source, enabling simple encoding operations to achieve high compression ratios tailored to the specific application.
Solution Approach 2:
The system dynamically adjusts compression parameters based on the learned statistical properties of the telemetry data. Different prediction models are selected and applied based on the specific patterns detected in the data, such as temporal autocorrelation, bit-state transition probabilities, and field-specific characteristics, allowing the simple RICE encoder to achieve variable compression ratios optimized for different data conditions.
3Loss of information
If run length substitution algorithm with multiple passes and frequent table access is used, then compression is achieved, but processing speed deteriorates
Solution Approach 1:
All complex compression logic, statistical analysis, and table generation are performed in advance during an offline training phase. The resulting prediction models and lookup tables are then used during real-time compression, eliminating the need for frequent table access and multiple passes during actual encoding operations.
Solution Approach 2:
The system extracts and separates the complex analytical functions from the real-time compression process. The offline training phase extracts statistical patterns and builds prediction models, while the online phase only performs simple encoding operations based on these pre-extracted models, removing the computational burden from the time-critical path.
4Loss of information
If compression algorithms requiring data storage before compression are used, then compression ratio is improved, but memory requirements and processing delay increase
Solution Approach 1:
The system performs preliminary statistical analysis on historical data to build prediction models offline, then uses these pre-built models for real-time compression of individual packets or small batches of data. This eliminates the need to accumulate large amounts of data in memory before compression can begin, enabling immediate compression with minimal memory requirements.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention relates to a method for compressing time series data and to a device and system for processing telemetric time series data. Based on historical time series data, the predictability of the state of each bit in a data packet is determined. A mask packet is provided with bits in a ONE state for all predictable bit positions of the data packet and with bits in a ZERO state for all unpredictable bit positions. Bit operations are carried out on a new data packet to bring all predictable bit positions of the new data packet into a ZERO state if an actual bit state matches the predicted bit state and into a ONE state if the actual bit state does not match the predicted bit state. A processed packet is generated by applying an AND operation between the new data packet on which the bit operations have been carried out and the mask packet to change all unpredictable bits to a ZERO state. A series of counters is provided by calculating the number of bits in the processed packet between the first bit and the first bit with a state ONE; and between the following bits with a state ONE and then between the last bit with a state ONE and the last bit in the processed packet. The provided series of counters are written into an output data packet which is stored or transmitted.