Multi-representation Time Series Storage for Memory and Query Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing volume and frequency of time-series data from industrial processes and IoT devices pose challenges in efficient storage and analysis, particularly in managing memory consumption, accuracy, and query response times, as existing approaches often conflict between execution efficiency and memory usage.
Innovation Solution
A multi-representation approach for storing time-series data, which selectively applies different compression models, error bounds, and storage media along the time axis (horizontal) and across the same period (vertical), optimizing memory consumption and query performance by identifying 'hot zones' and adapting representations based on query workloads and data aging strategies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a single compression model is used for all time series data, then memory consumption is reduced, but query response time deteriorates
Solution Approach 1:
The patent segments time series data into different temporal zones (hot zones, warm zones, cold zones) and applies different compression models to each zone. Recent data in hot zones uses lighter compression for fast access, while older data in cold zones uses heavier compression to save memory, thus resolving the contradiction between memory efficiency and query performance
Solution Approach 2:
Different regions of the time series data are assigned different quality characteristics through zone-based compression. Each zone has optimized compression parameters tailored to its access patterns, allowing local optimization that balances memory usage and query response time across different data regions
2Measurement precision
If high accuracy is maintained for all time series data, then measurement precision is improved, but memory consumption increases
Solution Approach 1:
The patent dynamically changes accuracy parameters (compression ratios, error bounds) based on the temporal zone of the data. Hot zone data maintains higher accuracy with lower compression, while cold zone data accepts lower accuracy with higher compression, optimizing the balance between precision and memory consumption
3Productivity
If multiple representations are stored for the same time period, then query performance is improved, but memory consumption increases
Solution Approach 1:
Multiple representations are segmented and stored only in specific temporal zones rather than uniformly across all data. This allows the system to maintain multiple formats for recent data where queries are frequent, while using single representations for older data, thus improving query performance without proportionally increasing memory consumption
Data Source
AI summary
A flexible approach for storing time series data, utilizes multiple representations in order to achieve optimization amongst various dimensions such as covered time period, accuracy, compression model, and storage medium. A first aspect selectively provides different representations for various portions of a time series along the time axis (horizontal). In this horizontal aspect, a first compression model may be applied to store data occurring during one period in a time series, while a different compression model is applied to store data during another period. In a separate (vertical) aspect, time series data over a same time period may be saved in multiple representations using different compression models. While increasing demands on memory, such vertical storage can afford efficient access for specific purposes (i.e., analytical). Storing time series data according to horizontal and/or vertical distributions, may be useful for applications such as data aging and the optimization of operator execution patterns.