Multidimensional Time Series Compression Using Local Residual Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing compression methods are ineffective in managing large multi-dimensional spectroscopic data, leading to difficulties in storage, transfer, and analysis due to significant file size and loss of fidelity.
Innovation Solution
The proposed method compresses multi-dimensional time series data by dividing it into local regions, predicting correlated portions from previous data series, and eliminating these correlations to achieve high-fidelity compression up to 330-fold.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional compression methods are applied to multi-dimensional spectroscopic data, then file size is reduced, but data fidelity is lost
Solution Approach 1:
The patent divides the multi-dimensional spectroscopic data into multiple local regions based on spatial coordinates. Each local region is processed independently through prediction and residual calculation, allowing compression while preserving local data characteristics and fidelity.
Solution Approach 2:
The patent transforms the data representation by calculating residuals between predicted and actual values in each local region. This parameter transformation enables more efficient encoding of the data, achieving compression while maintaining the ability to reconstruct high-fidelity data through inverse transformation.
2Measurement precision
If data is stored in original format, then data fidelity is maintained, but storage and transfer costs increase
Solution Approach 1:
The patent extracts the predictable correlated portions of data from each local region and represents them through prediction models. Only the residuals (unpredictable portions) and necessary prediction parameters are stored, significantly reducing storage requirements while maintaining data fidelity through reconstruction.
Solution Approach 2:
The patent performs prediction and residual calculation during the data acquisition or preprocessing stage. By preparing the data in this compressed residual form beforehand, the system achieves both space efficiency and fidelity preservation without requiring complex operations during storage or transfer.
3Volume of stationary object
If conventional compression is applied, then storage space is reduced, but processing time increases due to loss and reprocessing
Solution Approach 1:
The patent performs the prediction and residual calculation in advance during data acquisition. This preliminary processing transforms the data into a compressed form that requires minimal processing during analysis, reducing overall processing time while maintaining storage efficiency.
Solution Approach 2:
The compressed residual data structure is designed to be self-descriptive, containing all necessary information for reconstruction without requiring external reference data. This self-contained format enables fast processing and random access to specific regions without decompressing entire datasets.
4Quantity of substance
If high compression ratios are achieved, then storage efficiency improves, but data quality deteriorates
Solution Approach 1:
The patent applies different processing strategies to different local regions based on their specific characteristics. Each region's prediction model is tailored to its local data patterns, ensuring optimal compression while preserving the unique quality requirements of each region. This localized approach maintains overall data quality even at high compression ratios.
Data Source
AI summary
Described herein are computer-implemented methods for compressing sparse multidimensional ordered series data. In particular, these methods and apparatuses for performing them (including software) may be particularly well suited to efficiently compressing spectrographic data.