Time Series Compression Using Interpolation Error Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing lossless compression techniques for time series data are impractical for publicly accessible data that requires rapid decoding and provisioning, necessitating significant refactoring to meet real-time data access demands.
Innovation Solution
A lossy compression method that discards data points based on interpolation error scores, maintaining important changes while reducing data size by removing points with minimal impact on interpolation accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If lossless compression techniques are used for time series data, then data integrity is preserved, but decoding speed and real-time access performance deteriorate due to significant refactoring requirements
Solution Approach 1:
The patent changes the fundamental parameter of data representation by transitioning from lossless compression (preserving all original data points) to lossy compression (removing selected data points). This parameter change enables rapid provisioning and real-time access while maintaining acceptable data integrity through intelligent selection of which data points to remove based on interpolation error analysis.
2Quantity of substance
If time series data size is reduced through compression, then storage efficiency improves, but data precision and interpolation accuracy worsen
Solution Approach 1:
The patent implements a feedback mechanism by calculating interpolation error scores for each data point and using this information to determine which points to remove. The system continuously evaluates the impact of removing each data point on overall interpolation accuracy, allowing it to make informed decisions that minimize precision loss while achieving significant data size reduction.
Solution Approach 2:
The patent changes the parameter of data point selection by introducing interpolation error scoring as a criterion for determining which data points to remove. This parameter change enables the system to selectively reduce data size while maintaining interpolation accuracy in critical regions and accepting larger errors in less important regions.
3Loss of information
If all data points are retained in time series data, then complete information is preserved, but storage requirements and processing overhead increase
Solution Approach 1:
The patent applies the extraction principle by selectively removing data points that have minimal impact on the overall information content. By calculating interpolation error scores and removing points with low scores, the system extracts and eliminates redundant data while preserving the essential information needed for accurate time series representation and analysis.
Data Source
AI summary
A method includes obtaining time series data that includes a series of data points listed in temporal order. The method includes determining that a size of the time series data fails to satisfy a threshold size. In response, the method includes determining a range of the series of data points and determining, using the range of the series of data points, a respective score for each respective data point in the series of data points. The method also includes removing, using the respective scores for each data point in the series of data points, a plurality of data points from the series of data points. After removing the plurality of data points from the series of data points, the method includes determining an updated size of the series of data points and determining that the updated size of the series of data points satisfies the threshold size.