Floating-Point Time-Series Compression with Spatial-Temporal Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing compression techniques for floating point numbers in time-series data are inefficient due to the difficulty in compressing data formats like IEEE 754, which results in high overhead and suboptimal throughput in high performance computing environments.
Innovation Solution
A spatial-temporal compression technique that leverages both spatial and temporal information using an offset predictor to achieve higher compression ratios by predicting floating point values and storing differences, reducing processing overhead and improving data throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If standard compression techniques are used for floating point numbers, then data can be stored, but compression ratios are low and processing overhead is high
Solution Approach 1:
The patent applies preliminary action by predicting future floating point values before they are actually generated in the simulation. The predictor uses temporal and spatial patterns from historical data to estimate upcoming values, allowing the compression system to pre-calculate expected values and prepare compression schemes in advance, thereby reducing real-time processing overhead while maintaining high compression ratios
2Quantity of substance
If floating point data is compressed without prediction, then implementation is simple, but compression efficiency is poor
Solution Approach 1:
The patent implements feedback mechanisms where the compression system continuously monitors actual simulation output values against predicted values, calculates prediction errors, and uses this feedback to adjust and refine future predictions. This feedback loop enables the system to adapt to changing simulation patterns, maintaining high compression ratios while improving throughput through increasingly accurate predictions that require less data to represent
3Measurement precision
If IEEE 754 floating point format is used, then precision is maintained, but data size is large and compression is difficult
Solution Approach 1:
The patent extracts and separates the prediction component from the actual data storage. Instead of storing complete IEEE 754 floating point values, the system extracts only the prediction error (difference between actual and predicted values). This extraction approach maintains full precision when needed while dramatically reducing data size, as the errors are typically much smaller and can be represented with fewer bits than the original floating point values
Data Source
AI summary
Computer-implemented methods, systems, and devices to perform lossless compression of floating point format time-series data are disclosed. A first data value may be obtained in floating point format representative of an initial time-series parameter. For example, an output checkpoint of a computer simulation of a real-world event such as weather prediction or nuclear reaction simulation. A first predicted value may be determined representing the parameter at a first checkpoint time. A second data value may be obtained from the simulation. A prediction error may be calculated. Another predicted value may be generated for a next point in time and may be adjusted by the previously determined prediction error (e.g., to increase accuracy of the subsequent prediction). When a third data value is obtained, the adjusted prediction value may be used to generate a difference (e.g., XOR) for storing in a compressed data store to represent the third data value.