Time Series Compression Using Sorted Regression Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current time series compression techniques are inefficient for larger intervals, waste computing resources, and prevent reordering of data points, leading to suboptimal storage and retrieval of large volumes of time series data.
Innovation Solution
A device that sorts time series data, processes it with a regression model to generate compressed data and parameters, encodes the index, and stores these in a data structure, allowing for efficient compression and retrieval by reordering data points without conforming to time sequencing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If delta compression is used for time series data, then compression is effective for small intervals, but compression ratio deteriorates for larger intervals
Solution Approach 1:
The patent transforms the time series data by applying a mathematical transformation (converting from time-domain to a transformed domain) that changes the parameters of the data representation. This transformation enables effective compression across larger intervals by altering how the data is structured, allowing the compression algorithm to work efficiently regardless of the time interval between data points.
2Stability of the object's composition
If data points are stored in time series order, then temporal sequence is preserved, but data reordering is prevented
Solution Approach 1:
The patent inverts the traditional approach by not storing data in chronological order, but rather in an optimized order that facilitates compression. The transformation reorders the data points in a way that maximizes compression efficiency while still allowing temporal queries to be answered by reversing the transformation process, thus achieving both compression efficiency and temporal accessibility.
3Device complexity
If current compression techniques are used, then simple processing is maintained, but computing resources are wasted
Solution Approach 1:
The patent applies a preliminary transformation to the time series data before compression, which pre-processes the data into a form that is more amenable to compression. This preliminary action (the mathematical transformation) is performed once during data ingestion, and subsequent compression operations become more efficient, reducing the overall computing resources required for processing large volumes of time series data.
Data Source
AI summary
A device may receive, from a source device, original time series data to be stored in a data structure associated with the device, and may sort the original time series data to generate sorted time series data. The device may identify an index for the original time series data based on the sorted time series data. The device may process the sorted time series data, with a regression model, to generate compressed time series data and parameters associated with the compressed time series data. The device may encode the index to generate an encoded index, and may store the encoded index, the compressed time series data, and the parameters in the data structure.