Multidimensional Time Series Compression Using Local Correlation Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current compression methods are ineffective in managing large multi-dimensional spectroscopic data, resulting in significant storage and transfer challenges due to limited compression ratios and potential file size expansion, especially with datasets exceeding 1 GB in size, and fail to retain high fidelity in compressed data.
Innovation Solution
The proposed method compresses multi-dimensional data by predicting and eliminating correlated portions across local regions, using a predictor to scale and subtract correlated data from current data series, allowing for high-fidelity restoration without additional bits in the encoded stream, achieving up to 330-fold compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If standard compression methods (e.g., ZIP) are used on multi-dimensional spectroscopic data, then the data can be stored and transferred, but the compression ratio is limited and file size may even expand
Solution Approach 1:
The patent divides the multi-dimensional spectroscopic data into multiple local regions, which are then processed independently using compression algorithms. This segmentation allows the system to apply different compression strategies to different portions of the data, achieving better overall compression ratios while maintaining data fidelity.
Solution Approach 2:
The patent transforms the data representation by changing parameters such as converting spectral data into different coordinate systems or applying mathematical transformations (e.g., wavelet transforms, principal component analysis) that enable more efficient compression while preserving the essential information content.
2Quantity of substance
If high compression ratios are achieved through aggressive compression, then data size is reduced, but data fidelity is lost
Solution Approach 1:
The patent implements feedback mechanisms where compression parameters are adjusted based on the measured fidelity of compressed data. The system continuously monitors data quality metrics and adapts compression settings to maintain acceptable fidelity thresholds while maximizing compression efficiency.
Solution Approach 2:
The patent applies compression selectively to different portions of the data based on their importance or redundancy. Critical regions receive lighter compression to preserve fidelity, while less critical regions undergo more aggressive compression, achieving an optimal balance between overall data size reduction and maintained quality.
3Productivity
If data is compressed to reduce storage and transfer costs, then efficiency improves, but processing and restoration complexity increases
Solution Approach 1:
The patent performs preliminary organization and preprocessing of the spectroscopic data before compression, structuring it in a way that facilitates efficient compression and subsequent restoration. This preliminary action includes sorting, filtering, and organizing data by spectral features, which simplifies the compression process and reduces restoration complexity.
Data Source
AI summary
Described herein are computer-implemented methods for compressing sparse multidimensional ordered series data. In particular, these methods and apparatuses for performing them (including software) may be particularly well suited to efficiently compressing spectrographic data.