Time-Series Data Layering for Variable-Fidelity Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for compressing time-series data often compromise accuracy and data quality, as they rely on interpolation or fixed sampling, which can miss important features and degrade the fidelity of the data, especially in scenarios requiring variable resolution and high fidelity.
Innovation Solution
A computer-implemented method that iteratively selects data points from a time series dataset based on multiple threshold criteria, creating layered subsets that meet specific accuracy thresholds, allowing for flexible data retrieval and storage, ensuring data quality and fidelity without relying on interpolation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If fixed time sampling is used to compress time-series data, then the data size is reduced, but important features may be missed and accuracy is degraded
Solution Approach 1:
The patent changes the sampling parameter from fixed time intervals to variable intervals based on data characteristics. It introduces adaptive sampling rates that adjust according to the complexity and importance of data features, allowing higher sampling rates for critical features and lower rates for stable periods, thus maintaining accuracy while reducing overall data size
Solution Approach 2:
The patent implements dynamic sampling where the sampling rate is not static but adapts over time based on detected features and data variability. The system continuously monitors data characteristics and adjusts sampling parameters dynamically, enabling the compression algorithm to respond to changing data conditions and preserve important features
2Quantity of substance
If interpolation methods are used to compress time-series data, then storage requirements are reduced, but data fidelity is lost due to averaging
Solution Approach 1:
The patent extracts only the essential and important features from the time-series data rather than attempting to represent all data points. It identifies and retains key characteristics such as peaks, troughs, and significant transitions, discarding redundant information, thus maintaining data fidelity for critical features while reducing storage requirements
Solution Approach 2:
The patent applies different compression strategies to different portions of the data based on their local characteristics. Important features receive higher preservation priority with minimal compression, while less critical segments undergo more aggressive compression, creating a non-uniform compression quality that optimizes overall data fidelity relative to importance
3Quantity of substance
If duplicate data points are removed to compress data, then data size is reduced, but data resolution is lost
Solution Approach 1:
The patent applies partial deduplication by removing only exact or near-duplicate data points while preserving unique and varied data points. It uses a threshold-based approach that removes redundant copies but maintains sufficient data points to represent the full range of values and features, avoiding excessive compression that would lose resolution
4Quantity of substance
If conventional compression algorithms are used, then data is compressed into a single output, but the original data series cannot be recovered and variable fidelity queries are not supported
Solution Approach 1:
The patent segments the compressed data into multiple layers or levels of fidelity, where each layer represents a different level of compression and detail. This hierarchical structure allows users to selectively retrieve data at different quality levels and enables progressive refinement from low to high fidelity, supporting flexible queries and partial data recovery
Solution Approach 2:
The patent adds a quality/fidelity dimension to the compressed data structure, organizing data not just by compression level but by multiple dimensions including time ranges, feature types, and quality thresholds. This multi-dimensional organization enables versatile querying and retrieval strategies that conventional single-pass compression cannot support
Data Source
AI summary
A computer-implemented method includes: obtaining input data comprising a plurality of data values, wherein each data value is associated with a time value; obtaining a plurality of threshold criteria; and (a) selecting, or generating, a dataset or a plurality of datasets that are different to one another from the input data, and determining whether each selected, or generated dataset meets a threshold criterion of the plurality of threshold criteria; and (b) causing each selected, or generated, dataset that meets the threshold criterion to be stored in memory in association with the threshold criterion, repeating (a) and (b) for each threshold criterion of the plurality of threshold criteria, thus causing a plurality of datasets to be stored in the memory, wherein each stored dataset meets a threshold criterion of the plurality of threshold criteria.